Interference Light Removal Method, Device and Storage Medium Based on Grayscale Camera

Abstract

The present invention relates to a method, device and storage medium for removing interfering light based on a grayscale camera. The method includes: first, setting the camera exposure parameters for different strip temperature ranges; then establishing a background model based on a continuous plurality of input images; then changing the camera exposure according to the strip temperature, obtaining an input image and multiplying it by an image compensation gain to obtain a gain image; then, based on the background model, performing foreground extraction on the gain image to remove long-term interfering light and obtaining a background mask image; performing spark connected component extraction and connected component geometric information calculation on the obtained foreground image, and removing short-term interfering light based on the geometric information; and finally updating the background model. Compared with the prior art, the present invention has the advantages of high precision and strong environmental adaptability, etc.

Description

Technical Field

[0001] The present invention relates to the field of hot-rolled strip production and detection, and particularly to a method, device, and storage medium for removing interfering light based on a grayscale camera. Background Technique

[0002] Steel is one of the important basic application materials in the world. The steel industry is an important basic industrial sector, the material basis for the development of the national economy and national defense construction, and a symbol of a country's industrialization. In recent years, in order to improve the intelligent level of the steel industry, the application of machine vision technology in traditional steel manufacturing has been one of the research hotspots in universities and research institutions.

[0003] The camera and imaging quality are the key to machine vision technology. According to the imaging color, cameras can be divided into color cameras and grayscale cameras. In terms of light flux and detail expressiveness, grayscale cameras are significantly superior to color cameras. Therefore, in the industrial detection field where precision is required, grayscale cameras are often used as measurement "sensors". However, in actual applications, metal surface reflection, direct sunlight, and external light sources (such as sunlight and lighting) will interfere with the details of the detection object, ultimately resulting in a decrease in detection accuracy. A common solution is to abandon the detail expressiveness of the grayscale camera and select a color camera to remove interfering light in the color space layer.

[0004] In addition, in the field of hot-rolled strip production, grayscale cameras have another defect. According to the camera imaging principle, different from color cameras, the sensors of grayscale cameras collect photons of all wavelengths for photoelectric conversion. Therefore, the sensors of grayscale cameras will also absorb thermal radiation in the environment. However, in the production of hot-rolled strips, different processing technologies will result in different temperatures of the strips, and the temperature difference between different parts of the same steel plate can be as high as fifty degrees Celsius, which will cause a drastic change in the global grayscale value of the image and ultimately affect the detection accuracy. Summary of the Invention

[0005] The purpose of the present invention is to provide a method, device, and storage medium for removing interfering light based on a grayscale camera.

[0006] The purpose of the present invention can be achieved through the following technical solutions:

[0007] A method for removing interfering light based on a grayscale camera includes:

[0008] Step S1: Set the camera exposure parameters for different strip temperature ranges;

[0009] Step S2: Establish a background model based on multiple consecutive input images. The background model is a picture with the same size as the input image, and the gray value of each pixel point in the background model is the average value of the gray values of the pixel points at the same position in the multiple consecutive input images.

[0010] Step S3: Adjust the exposure of the camera according to the strip temperature, obtain the image to be detected with the same size as the background model, and obtain the average value M0 of the global gray value of the image to be detected.

[0011] Step S4: Calculate the ratio G = M B / M0 of the average values of the global gray values of the background model and the image to be detected as the image compensation gain.

[0012] Step S5: Multiply the image to be detected by the image compensation gain to obtain the gain image.

[0013] Step S6: Subtract the background model from the gain image to obtain the difference image, and obtain the background mask image based on the pre-configured background threshold.

[0014] Step S7: Extract the foreground image from the difference image based on the background mask image.

[0015] Step S8: Perform spark connected component extraction and connected component geometric information calculation on the obtained foreground image, and remove short-term interfering illumination based on the geometric information.

[0016] Step S9: Update the background model according to the background mask image.

[0017] The specific steps of Step S1 include:

[0018] Step S11: Determine the upper and lower limits of the strip temperature according to the processing technology.

[0019] Step S12: Divide the upper and lower temperature limits into multiple temperature intervals at equal intervals, and determine the exposure parameters for each temperature interval.

[0020] The gray value of each pixel point in the background model is the weighted average value of the gray values of the pixel points at the same position in the multiple consecutive input images, and the weights of each input image are as follows:

[0021]

[0022] Where: w N-k is the weight of the (N - k)-th input image, and w N is the weight of the N-th input image.

[0023] In Step S3, adjusting the exposure of the camera according to the strip temperature specifically means: adjusting the exposure of the camera according to the strip temperature collected in real time.

[0024] Step S6 specifically includes:

[0025] Step S61: Subtract the background model from the gain image, and set all pixel points with gray value differences lower than 0 to 0 to obtain a difference image;

[0026] Step S62: Traverse the difference image pixel by pixel. When the gray value of a certain pixel point is greater than the preset background threshold, this pixel point is the foreground pixel, and all the remaining pixel points are used as background pixels to generate a background mask image:

[0027]

[0028] Where: I S (pi) is the gray value of the pixel point pi in the difference image, and T B is the background threshold, and I Mask (pi) is the gray value of the pixel point pi in the background mask image.

[0029] Step S7 is specifically: Set all background pixels in the difference image to 0 to obtain a foreground image.

[0030] Step S8 specifically includes:

[0031] Step S81: Perform fixed-threshold binarization on the foreground image to obtain a fixed-threshold binary image;

[0032] Step S82: Perform maximum entropy threshold binarization on the foreground image to obtain a maximum entropy threshold binary image;

[0033] Step S83: Solve the intersection of the fixed-threshold binary image and the maximum entropy threshold binary image to obtain the final binarization result;

[0034] Step S84: Perform a morphological closing operation on the binarization result to obtain a morphological processing result;

[0035] Step S85: Use the seed filling algorithm to extract all spark connected components with a gray value of 1 from the morphological processing result;

[0036] Step S86: Calculate the geometric information of the connected component in terms of pixels, including the center point coordinates, area, and image moments.

[0037] In step S8, the process of removing short-term interfering light specifically includes:

[0038] Step S801: Set the geometric information cache length to n, indicating that the geometric information of consecutive n morphological processing results will be stored. Among them, 1 ≤ n ≤ R, where R is the sampling frequency, and the background model is updated every R morphological processing results;

[0039] Step S802: When there is new morphological processing result input, calculate the spatial moments of all its connected components as shown in the following formula:

[0040]

[0041] Where:

[0042] I(x, y) is the pixel value of the pixel point with abscissa x and ordinate y in the pixel coordinate system. x and y can be regarded as two discrete random variables. Therefore, m ji is the (i + j)-th mixed origin moment of the random variables x and y. In addition, w and h are the length and width of the image;

[0043] Step S803: Calculate the center points of all connected components:

[0044]

[0045] Where:

[0046] and represent the center point coordinates of the morphological processing result corresponding to the current frame. m 10 is the 1st origin moment of the random variable x (image abscissa), m 00 is the 0th moment of the image. Since the specific meaning of I(x, y) is the pixel value of the pixel point with abscissa x and ordinate y, the 0th moment represents the sum of the pixel values of the image at this time. m 01 is the 1st origin moment of the random variable y (image ordinate);

[0047] Step S804: Assume that the area of the connected component is approximately equal to the area S of its circumscribed bounding box, and calculate the feature deviation between the current frame and the cached frame:

[0048]

[0049] Where: and represent the center point coordinates of the morphological processing result corresponding to the k-th frame in the cache. S and S k represent the areas of the current frame and the k-th frame respectively;

[0050] Step S805: When the feature deviation D of a certain connected component is less than the pre-configured threshold T dist , it is considered that there is a connected component with similar features in the k-th frame for this connected component, and this connected component is determined to have short-term interference light removed; otherwise, it is considered that there is no similar connected component in the k-th frame.

[0051] Step S806: Store the connected component information of the current frame in the cache and replace the frame information with the farthest time.

[0052] An interference light removal device based on a grayscale camera, comprising a memory, a processor, and a program, wherein when the processor executes the program, the above-mentioned method is implemented.

[0053] A storage medium, on which a program is stored, and when the program is executed, the above-mentioned method is implemented.

[0054] Compared with the prior art, the present invention has the following beneficial effects:

[0055] 1. It is implemented based on a grayscale camera. Compared with a color camera, it has high imaging quality and strong detail expression ability, thereby reducing image loss and reducing the systematic error of image detection.

[0056] 2. The input of all image processing algorithms is a single-channel grayscale image. Compared with a three-channel RGB image, the data volume is reduced by two-thirds, and the processing speed is significantly improved;

[0057] 3. The background model is updated iteratively frame by frame, and the algorithm has good robustness and strong environmental adaptability. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 It is a schematic flow chart of the interference light removal method for the grayscale camera in the present invention;

[0059] Figures 2 to 4 It is a schematic diagram of three consecutive frames for establishing the background model in the embodiment of the present invention;

[0060] Figure 5 It is a schematic diagram of the background model in the embodiment of the present invention;

[0061] Figure 6 It is a schematic diagram of the input image for gain compensation in the embodiment of the present invention;

[0062] Figure 7 It is a schematic diagram of the image after gain compensation in the embodiment of the present invention;

[0063] Figure 8 It is a schematic diagram of the foreground image after removing the background in the embodiment of the present invention;

[0064] Figure 9 It is a schematic diagram of the extraction result of the spark connected domain in the embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0065] The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. This embodiment is implemented on the premise of the technical solution of the present invention, and detailed implementation manners and specific operation processes are given, but the protection scope of the present invention is not limited to the following embodiments.

[0066] A method for removing interfering light based on a grayscale camera, which is in the form of a computer program and executed by a computer system. It can be implemented by a device for removing interfering light based on a grayscale camera that executes the program and a corresponding storage medium that stores the program.

[0067] As Figure 1 shown, the method includes:

[0068] Step S1: Set the camera exposure parameters for different strip temperature ranges, specifically including:

[0069] Step S11: Determine the upper and lower limits of the strip temperature according to the processing technology;

[0070] Step S12: Divide the upper and lower temperature limits into multiple equal intervals to determine the exposure parameters for each temperature range.

[0071] In this embodiment, the measurement end of the thermocouple sensor is set on the strip inlet side of the camera view. Then, the DC voltage signal generated by the thermocouple sensor is transmitted to the upper computer through A / D conversion as the exposure control signal. Based on the principle that the steel plate and the spark can be clearly distinguished, the exposure time is adjusted according to the real-time temperature measured by the thermocouple, and finally the good exposure time corresponding to each temperature range is determined;

[0072] Step S2: Establish a background model based on a series of consecutive input images. Among them, the background model is a picture with the same size as the input image, and the gray value of each pixel point in the background model is the average value of the gray values of the pixel points at the same position in a series of consecutive input images;

[0073] The gray value of each pixel point in the background model is the weighted average value of the gray values of the pixel points at the same position in a series of consecutive input images, and the weights of each input image are as follows:

[0074]

[0075] Among them: w N-k is the weight of the (N - k)th input image, and w N is the weight of the Nth input image.

[0076] In this embodiment, a total of three consecutive frames of images are selected as Figures 2 to 4 shown for background modeling. The established background model is as Figure 5 shown. The background model is a grayscale image with the same size as the input image, and the gray value of each coordinate point is the weighted average of the gray values of the three consecutive frames of images at this coordinate point. The setting of the weights is related to the time sequence. Usually, the weight of the frame farther from the current frame is lower, and it can also be customized, but the sum of the weights of each frame needs to be 1.

[0077] Step S3: Adjust the exposure of the camera according to the strip temperature, obtain the image to be detected with the same size as the background model, and obtain the average value M0 of the global gray value of the image to be detected; among them, adjusting the exposure of the camera according to the strip temperature specifically means: adjusting the exposure of the camera according to the strip temperature collected in real time.

[0078] Step S4: Calculate the ratio G = M B / M0 of the average values of the global gray values of the background model and the image to be detected as the image compensation gain;

[0079] Step S5: Multiply the image to be detected by the image compensation gain to obtain the gain image I G ;

[0080] First, use the measured real-time strip temperature as the exposure control signal to change the exposure time of the camera and take an image. Due to the change of the exposure time, as Figure 6 shown, the global brightness of the input image is quite different from that of the background model as Figure 5 shown. Considering that the illumination has a multiplicative gain on the global brightness of the image, therefore, solve the ratio G of the average gray values of the two images as the compensation gain of the input image, that is, G = M B / M0. Finally, multiply the input image by the compensation gain G to obtain the gain image I G , as Figure 7 shown.

[0081] Step S6: Subtract the background model from the gain image to obtain the difference image, and obtain the background mask image I Mask , specifically including:

[0082] Step S61: Subtract the background model from the gain image, and set all pixel points with gray value differences lower than 0 to 0 to obtain the difference image I S ;

[0083] Step S62: Traverse the difference image pixel by pixel. When the gray value of a certain pixel point is greater than the preset background threshold T B , this pixel point is the foreground pixel, and all the remaining pixel points are used as background pixels to generate the background mask image:

[0084]

[0085] where: I S (pi) is the gray value of the pixel point pi in the difference image, T B is the background threshold, and I Mask (pi) is the gray value of the pixel point pi in the background mask image.

[0086] Step S7: Based on the background mask image, extract the foreground image I Fg, specifically: in the difference image, all background pixels are set to 0 to obtain the foreground image, and the rest is used as the background image I Bg

[0087] Step S8: Extract the spark connected components and calculate the geometric information of the connected components from the obtained foreground image, and remove short-term interference illumination based on the geometric information, specifically including:

[0088] Step S81: Set a fixed threshold T Fixed For the foreground image I Fg Perform fixed-threshold binarization. If the gray value is greater than T Fixed , it is set to 1; otherwise, it is set to 0. Obtain the fixed-threshold binary image I Fixed ;

[0089] Step S82: Use the maximum entropy threshold algorithm to solve the maximum entropy threshold T of the foreground image Auto , and obtain the maximum entropy threshold binary image I Auto ; Solve the intersection of the two binary images as the final binarization result I Binary ;

[0090] Step S83: Solve the intersection of the fixed-threshold binary image and the maximum entropy threshold binary image to obtain the final binarization result I Binary ;

[0091] Step S84: Perform a morphological closing operation on the binarization result I Binary to obtain the morphological processing result I m ;

[0092] Step S85: For the morphological processing result, use the seed filling algorithm to extract all spark connected components with a gray value of 1;

[0093] Step S86: Calculate the geometric information of the connected components in terms of pixels, including the center point coordinates, area, and image moments.

[0094] In step S8, the process of removing short-term interference illumination specifically includes:

[0095] First, set the geometric information cache length to n (1 ≤ n ≤ R), indicating that the geometric information of consecutive n images will be stored. Here, the images are the morphological processing results I obtained in step 5-2 m .

[0096] Then the steps are as follows:

[0097] Step S801: Set the geometric information cache length to n, indicating that the geometric information of consecutive n morphological processing results will be stored, where 1 ≤ n ≤ R, and R is the sampling frequency. The background model is updated every R morphological processing results;

[0098] Step S802: When new morphological processing results are input, calculate the spatial moments of all its connected components as shown in the following formula:

[0099]

[0100] Where:

[0101] I(x, y) is the pixel value of the pixel point with abscissa x and ordinate y in the pixel coordinate system. x and y can be regarded as two discrete random variables. Therefore, m ji is the (i + j)-th mixed origin moment of the random variables x and y. In addition, w and h are the length and width of the image;

[0102] Step S803: Calculate the center points of all connected components:

[0103]

[0104] Where:

[0105] and represent the center point coordinates of the morphological processing results corresponding to the current frame. m 10 is the 1st origin moment of the random variable x (image abscissa), m 00 is the 0th moment of the image. Since the specific meaning of I(x, y) is the pixel value of the pixel point with abscissa x and ordinate y, the 0th moment represents the sum of the pixel values of the image at this time. m 01 is the 1st origin moment of the random variable y (image ordinate);

[0106] Step S804: Assume that the area of the connected component is approximately equal to the area S of its circumscribed bounding box, and calculate the feature deviation between the current frame and the cached frame:

[0107]

[0108] Where: and represent the center point coordinates of the morphological processing results corresponding to the k-th frame in the cache. S and S k represent the areas of the current frame and the k-th frame respectively;

[0109] Step S805: When the feature deviation D of a certain connected component is less than the pre-configured threshold T dist , it is considered that there is a connected component with similar features in the k-th frame for this connected component, and this connected component is determined to be short-term interference light removed; otherwise, it is considered that there is no similar connected component in the k-th frame for this connected component.

[0110] Step S806: Store the connected component information of the current frame in the cache and replace the frame information with the farthest time.

[0111] Step S9: Update the background model according to the background mask image, specifically:

[0112] During initialization, the size of the background model cache needs to be set to N, indicating that the gray values of N images will be stored. The update of the background model is pixel-by-pixel, that is, the gray value corresponding to the coordinate of the background gray map is updated with the weighted average of all the gray values cached at a certain pixel.

[0113] In addition, according to the background mask image, the pixels in the input image can be divided into foreground pixels and background pixels. For foreground pixels, that is, the pixels whose corresponding points in the background mask image are 1, a probability P needs to be preset as the update probability during the initialization of the background model. When updating the background, a random number X in the range of [0, 1] is randomly selected. When the random number X is greater than the update probability P, the foreground pixels are updated; otherwise, they are not updated. For background pixels, their corresponding background models are updated.

Claims

1. A method for removing interfering light based on a grayscale camera, characterized in that, Including: Step S1: Set the camera exposure parameters for different strip temperature ranges; Step S2: Establish a background model based on a series of consecutive input images. Here, the background model is a picture with the same size as the input images, and the gray value of each pixel point in the background model is the average of the gray values of the pixel points at the same position in the series of consecutive input images; Step S3: Adjust the exposure of the camera according to the strip temperature, obtain the image to be detected with the same size as the background model, and obtain the average value M0 of the global gray value of the image to be detected; Step S4: Calculate the ratio G = M B / M0 of the average global grayscale values of the background model and the image to be detected as the image compensation gain; Step S5: Multiply the image to be detected by the image compensation gain to obtain the gain image; Step S6: Subtract the background model from the gain image to obtain the difference image, and obtain the background mask image based on the pre-configured background threshold; Step S7: Extract the foreground image from the difference image based on the background mask image; Step S8: Perform spark connected component extraction and connected component geometric information calculation on the obtained foreground image, and remove short-term interfering light based on the geometric information; Step S9: Update the background model according to the background mask image; The specific content of step S8 includes: Step S81: Perform fixed threshold binarization on the foreground image to obtain a fixed threshold binary image; Step S82: Perform maximum entropy threshold binarization on the foreground image to obtain a maximum entropy threshold binary image; Step S83: Solve the intersection of the fixed threshold binary image and the maximum entropy threshold binary image to obtain the final binarization result; Step S84: Perform a morphological closing operation on the binarization result to obtain the morphological processing result; Step S85: Use the seed filling algorithm to extract all spark connected components with a gray value of 1 from the morphological processing result; Step S86: Calculate the geometric information of the connected components in pixels, including the center point coordinates, area, and image moments; In step S8, the process of removing short-term interfering light specifically includes: Step S801: Set the geometric information cache length to n, indicating that the geometric information of consecutive n morphological processing results will be stored, where 1 ≤ n ≤ R, and R is the sampling frequency. The background model is updated every R morphological processing results; Step S802: When a new morphological processing result is input, calculate the spatial moments of all its connected components, as shown in the following formula: Where: I(x, y) is the pixel value of the pixel point with abscissa x and ordinate y in the pixel coordinate system. x and y are regarded as two discrete random variables, so m ji is the (i + j)-th mixed origin moment of the random variables x and y. In addition, w and h are the length and width of the image; Step S803: Calculate the center points of all connected components: Wherein: and represents the center point coordinates of the morphological processing result corresponding to the current frame, m 10 is the first-order origin moment of the random variable x (image abscissa), m 00 is the zero-order moment of the image. Since the specific meaning of I(x, y) is the pixel value of the pixel point with abscissa x and ordinate y, the zero-order moment at this time represents the sum of the pixel values of the image, m 01 is the first-order origin moment of the random variable y (image ordinate); Step S804: Assume that the area of the connected component is approximately equal to the area S of its circumscribed bounding box, and calculate the feature deviation between the current frame and the cached frame; Wherein: and represent the center point coordinates of the morphological processing result corresponding to the k-th frame in the cache, and S and S k respectively represent the areas of the current frame and the k-th frame; Step S805: When the feature deviation D of a certain connected component is less than a pre-configured threshold T dist , it is considered that there is a connected component with similar features in the k-th frame for this connected component, and this connected component is determined to have short-term interference light removed; otherwise, it is considered that there is no similar connected component in the k-th frame; Step S806: Store the connected component information of the current frame in the cache and replace the information of the frame with the farthest time; 2. The interference light removal method based on a grayscale camera according to claim 1, characterized in that, The specific content of step S1 includes: Step S11: Determine the upper and lower limits of the strip temperature according to the processing technology; Step S12: Divide the temperature range into multiple equal intervals, and determine the exposure parameters for each temperature interval; 3. A method for removing interfering light based on a grayscale camera according to claim 1, characterized in that, The gray value of each pixel point in the background model is the weighted average of the gray values of the pixel points at the same position in the series of consecutive input images, and the weights of each input image are as follows: where: w N-k is the weight of the (N - k)-th input image, and w N is the weight of the N-th input image.

4. A method for removing interfering light based on a grayscale camera according to claim 1, characterized in that, In step S3, adjusting the exposure of the camera according to the strip temperature specifically means: adjusting the exposure of the camera according to the strip temperature collected in real time; 5. A method for removing interfering light based on a grayscale camera according to claim 1, characterized in that, The specific content of step S6 includes: Step S61: Subtract the background model from the gain image, and set all pixel points with gray value differences lower than 0 to 0 to obtain a difference image; Step S62: Traverse the difference image pixel by pixel. When the gray value of a certain pixel point is greater than a preset background threshold, this pixel point is a foreground pixel, and all the remaining pixel points are used as background pixels to generate a background mask image: Where: I S (pi) is the gray value of the pixel point pi in the difference image, and T B is the background threshold, and I Mask (pi) is the gray value of the pixel point pi in the background mask image.

6. The interference light removal method based on a grayscale camera according to claim 1, wherein The specific content of step S7 is: Set all background pixels in the difference image to 0 to obtain a foreground image.

7. An interference light removal device based on a grayscale camera, comprising a memory, a processor, and a program, characterized in that When the processor executes the program, it implements the method described in any one of claims 1-6.

8. A storage medium, on which a program is stored, characterized in that, When the program is executed, it implements the method described in any one of claims 1-6.

Images