Hair Foreign Object Detection Method for Coated Surfaces Based on Fast Local Threshold Segmentation

Abstract

The present invention discloses a method for detecting hair foreign bodies on a coated surface based on fast local threshold segmentation. The coated surface is photographed to obtain a surface grayscale image, the surface grayscale image is combined with Gaussian filtering and subtraction operations to obtain a grayscale difference image, a large threshold is used to segment the grayscale difference image to obtain the outline of suspected obvious hair, and a preliminary judgment is made, and then a fine further image analysis is performed based on the grayscale difference image to determine whether there is a hair defect. The present invention is accurate and practical in detection, and can effectively filter out background interference similar to hair. The image calculation speed is fast, and it can be widely used in similar scenes, and has great application value.

Description

Technical Field

[0001] The present invention relates to a computer vision image processing method, and specifically relates to a method for detecting hair-like foreign objects on a coated surface based on fast local threshold segmentation. Background Art

[0002] With the rapid development of the industrial field, the requirements for coating quality are getting higher and higher. The traditional method of using manual inspection for coating surface defects not only affects the production speed but also easily misses small defects. In recent years, with the gradual improvement of computer vision in the industrial field, image algorithms are gradually replacing manual labor as the main means of coating surface defect detection.

[0003] Computer vision in coating detection is mainly divided into two types:

[0004] 1) Defect detection methods based on deep learning. The common method is to manually calibrate images with defects, train them through neural networks, and use the trained weights to process the images to obtain defects. This technology can accurately detect obvious and larger-sized defects. However, in order to improve the training and detection speed, the image needs to be compressed before being processed by the neural network, resulting in the easy loss of small defects such as hair-like filaments after compression. If the compression ratio is reduced, it will lead to too long processing time or the need to stack graphics cards, resulting in a large increase in costs.

[0005] 2) Defect detection methods using traditional image algorithms. The common method is to obtain defects by the difference from the background. This method can detect defects well when the background is uniform, but it is difficult to extract defects when the background changes significantly, especially when the defects are similar to the background.

[0006] Existing methods have problems such as limited types of detected surface defects, complex calculation methods, slow speed, difficulty in being used for online detection, or reliance on complex and costly hardware imaging technologies. Therefore, a new method for detecting hair-like foreign objects on a coated surface is needed. Summary of the Invention

[0007] In order to solve the problem that it is difficult to detect defects such as hair-like filaments on the coating surface that are slender and similar to the background, the purpose of the present invention is to provide a method for detecting hair-like foreign objects on a coated surface based on fast local threshold segmentation. Compared with the background art, the recognition method is simple, can quickly and accurately detect hair-like foreign objects, and is suitable for online detection occasions.

[0008] As Figure 1 shown, the steps of the technical solution adopted by the present invention to solve its technical problems are as follows:

[0009] 1) Use a camera or a camera to capture an image of the coated surface to obtain a surface grayscale image P(i, j);

[0010] 2) Combine Gaussian filtering and subtraction operation on the surface grayscale image P(i, j) to obtain the grayscale difference image D(i, j);

[0011] 3) Use a large threshold to segment the grayscale difference image D(i, j), traverse each pixel point of the segmented binary image to obtain the connected regions and use them as the contours of suspected obvious hair strands, and calculate the length and circularity of the contours to further determine whether it is a hair strand:

[0012] If it is a hair strand, it is determined that there is a hair strand, that is, it is considered that there is a hair strand foreign object, return the hair strand data and exit to save calculation, otherwise jump to the next step;

[0013] 4) Perform further image analysis and processing on the grayscale difference image D(i, j) to determine whether there is a hair defect, and further determine whether there is a hair strand foreign object.

[0014] The specific steps of step 2) are as follows:

[0015] 2.1) Perform Gaussian filtering on the surface grayscale image P(i, j) to form a blurred image;

[0016] 2.2) Use the blurred image to perform subtraction operation on the surface grayscale image P(i, j) to obtain the grayscale difference image D(i, j).

[0017] In step 2.1), filter the original surface grayscale image according to the formula:

[0018]

[0019] In the formula, f(x, y) is the filtered signal value, x and y respectively represent the horizontal and vertical coordinates of the image, σ represents the standard deviation of the Gaussian distribution, and e represents the natural constant.

[0020] The specific steps of step 4) are as follows:

[0021] 4.1) Calculate the variance of the pixel values of all pixel points in each column of the grayscale difference image D(i, j) and save it as the variance of each pixel column, and obtain the maximum and minimum values of the variances of all pixel columns;

[0022] 4.2) Calculate the high segmentation threshold according to the variance of each pixel column and the maximum and minimum values of the variances of each pixel column, and segment the grayscale difference image D(i, j) using the high segmentation threshold to obtain the binary image B(i, j) of the darker region;

[0023] 4.3) Perform dilation processing on the binary image B(i, j) of the darker region to obtain the suspected region template image M(i, j);

[0024] 4.4) Calculate the low segmentation threshold based on the variance of each column of pixels, as well as the maximum and minimum values of the variance of each column of pixels. Segment the grayscale difference image D(i,j) using the low segmentation threshold to obtain a binary image of the darker region;

[0025] 4.5) Perform an AND operation on the binary image of the darker region and the suspected region template image M(i,j) to obtain a binary contour image C(i,j);

[0026] 4.6) Obtain all connected components in the binary contour image C(i,j) as contours through connected component recognition. One connected component is regarded as one contour, and filter all contours according to the circularity and length of each contour to obtain all slender contours suspected of being hair;

[0027] 4.7) Connect adjacent contours among all the slender contours in step 4.6), filter non-hair defects according to the actual hair length, and determine whether there are hair defects.

[0028] In step 3) or step 4.6), use the following formula to calculate the circularity of the contour:

[0029]

[0030] In the formula, degree is the circularity of the contour, area is the enclosed area of the contour, and r is the radius of the smallest circle enclosing the contour.

[0031] In step 4.1), the variance of the pixel values of all pixel points in every five columns is used as the variance of each column of pixel points among these five columns of pixel points, so as to calculate the variance of the compressed image to compress the image, saving calculation and preventing deviation.

[0032] In step 4.1), use the following formula to calculate the variance of each column of pixel points:

[0033]

[0034] In the formula, S 2 is the variance of an m*n-sized image, is the average value of the pixel values of all pixel points in the region where each column of pixel points is located, and p(i,j) is the pixel value of the pixel point at coordinate (i,j).

[0035] In step 4.2) and step 4.4), both the high segmentation threshold and the low segmentation threshold used for image segmentation are calculated using the following formula:

[0036] thresh = thresh max *θ + thresh min *(1 - θ)

[0037] Wherein, thresh is the segmentation threshold actually used, thresh max and thresh min are respectively the upper and lower limits of the preset segmentation threshold, and θ is the proportionality coefficient;

[0038] The high segmentation threshold and the low segmentation threshold are set according to the upper and lower limits of the preset segmentation threshold according to the above formula. Both use the same formula. Due to different positions or light fields or other purposes, the empirical values of threshmax and threshmin are different.

[0039] And the proportionality coefficient θ is set according to the following formula:

[0040]

[0041] d = M - m

[0042] Wherein, v is the variance of each column of pixel columns, m and M are respectively the minimum and maximum values of the variances of each column of pixel columns in the image, and d represents the maximum difference of the variances of each column of pixel columns in the image.

[0043] In the steps 4.2) and 4.4), the following formula is used to segment the image by using the segmentation threshold:

[0044]

[0045] Wherein, v i,j is the pixel value after segmentation processing of the pixel point at the coordinate (i, j), p i,j is the original pixel value of the pixel point at the coordinate (i, j), thresh is the segmentation threshold, which is the high segmentation threshold or the low segmentation threshold.

[0046] The step 4.7) is specifically as follows:

[0047] 4.7.1) Judge each slender contour:

[0048] If the length of the slender contour is greater than or equal to the preset hair length, then the slender contour is a hair defect, and it is considered that there is a hair foreign object, and the next step is not carried out;

[0049] If the length of the slender contour is less than the preset hair length, then the slender contour is used as a suspected contour, and the next step is carried out;

[0050] 4.7.2) Initially connect the suspected contour with other suspected contours within the actual adjacent distance around it to form a connected contour. Specifically, first calculate the actual adjacent distance between the contours by using the outer bounding rectangle between the contours, and perform the initial connection and save within the actual adjacent distance to save calculation. The actual adjacent distance is determined according to the following formula:

[0051] dis = min(dis i,j )

[0052]

[0053] where dis is the actual adjacent distance, and dis i,j is the distance between the i-th pixel of the first contour and the j-th pixel of the second contour, and p i and p j are the pixel values of the pixels on the two contours respectively;

[0054] 4.7.3) For all the connected contours obtained in 4.7.2), calculate the length of the minimum bounding rectangle of the connected contour and make a judgment:

[0055] If the length is less than the preset hair length, discard it;

[0056] If the length is greater than or equal to the preset hair length, the connected contour is a hair defect, and it is considered that there is a hair foreign object.

[0057] The coated surface described in the present invention refers to the surface formed by coating an object, the ground or the wall with a certain material. Usually, this material is a polymer material.

[0058] The beneficial effects of the present invention are:

[0059] The present invention utilizes the characteristic that the image brightness is uneven on the coated surface due to light or process reasons, performs segmented processing on the image through an adaptive threshold, uses a high threshold to eliminate most of the coating interference, and uses a low threshold to connect the hair strands.

[0060] The detection of the present invention is accurate and practical, and can effectively filter the background interference similar to hair. The method is simple, which is not only conducive to engineering implementation, but also has a fast processing speed and high accuracy. The image calculation speed is fast. In the detection of hair strands, it can be widely applied to similar scenarios and has great application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 is the flow chart of the method of the present invention.

[0062] Figure 2 is the original image in Embodiment 1 of the present invention.

[0063] Figure 3 is the blurred image after Gaussian filtering.

[0064] Figure 4 is Figure 3 the difference image from the original image.

[0065] Figure 5It is the binary image after the difference image is segmented using a high threshold.

[0066] Figure 6 is Figure 5 the binary image after dilation.

[0067] Figure 7 It is the binary image after the difference image is segmented using a low threshold.

[0068] Figure 8 is Figure 5 and Figure 6 the binary image after AND operation.

[0069] Figure 9 is Figure 2 the detection result image of hair foreign objects.

[0070] Figure 10 is the original image with hair.

[0071] Figure 11 is Figure 10 the detection result image of hair foreign objects.

[0072] Figure 12 is the original image with hair.

[0073] Figure 13 is Figure 12 the detection result image of hair foreign objects. Specific implementation manner

[0074] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments.

[0075] As Figure 1 shown, the embodiments of the present invention and their implementation processes are as follows:

[0076] 1) Take an image of the coating surface through a camera or a webcam to obtain the surface grayscale image P(i,j); take a grayscale image of the coating surface. As Figure 2 shown, for the convenience of display, only the hair and the surrounding area are shown.

[0077] 2) Use a 31×31 window size to perform Gaussian filtering on the original surface grayscale image P(i,j) according to the following formula to form a blurred image as Figure 3 shown;

[0078]

[0079] In the formula, f(x,y) is the filtered signal value, x and y respectively represent the horizontal and vertical coordinates of the image, σ represents the standard deviation of the Gaussian distribution, and e represents the natural constant.

[0080] 3) Subtract the original image from the filtered image, that is, perform a subtraction operation on the surface gray-scale image P(i, j) using the blurred image, to obtain the gray-scale difference image D(i, j) that only contains darker regions as shown in Figure 4 Figure Figure 4 .

[0081] 4) Segment the gray-scale difference image D(i, j) using a large threshold to obtain a binary image. The segmentation threshold is shown in formula (2). Here, thresh max is the upper limit of the threshold for subsequent use, and the segmentation method is shown in formula (3).

[0082] Traverse each pixel point of the segmented binary image to obtain connected regions and use them as the contours of suspected obvious hair strands, and calculate the length and circularity of the contours to further determine whether they are hair strands:

[0083] The circularity of the contour is calculated using the following formula:

[0084] thresh = 1.5 * thresh max (2)

[0085]

[0086] In the formula, degree is the circularity of the contour, area is the enclosed area of the contour, and r is the radius of the smallest circle enclosing the contour.

[0087] If it is a hair strand, it is determined that there is a hair strand, that is, it is considered that there is a hair strand foreign object, return the hair strand data and exit to save calculations, otherwise jump to the next step;

[0088] 5) Perform further image analysis and processing on the gray-scale difference image D(i, j) to determine whether there is a hair defect, and then determine whether there is a hair strand foreign object.

[0089] 5.1) Calculate the variance of the pixel values of all pixel points in each column of the gray-scale difference image D(i, j) and save it as the variance of each pixel column, and obtain the maximum and minimum values of the variances of all pixel columns;

[0090] In specific implementation, the variance of the pixel values of all pixel points in every five columns is used as the variance of each pixel column in these five columns, so as to calculate the variance of the compressed image to compress the image, saving calculations and preventing deviation.

[0091] The following formula is used to calculate the variance of each pixel column:

[0092]

[0093] In the formula, S 2 is the variance of an m * n sized image, is the average value of the pixel values of all pixel points in the area where each column of pixels is located, and p(i, j) is the pixel value of the pixel point at coordinates (i, j).

[0094] 5.2) Calculate the high segmentation threshold and the low segmentation threshold according to the variance of each column of pixel columns and the maximum and minimum values of the variances of each column of pixel columns, and use the high segmentation threshold to segment the grayscale difference image D(i, j) to obtain Figure 5 the binary image B(i, j) of the darker area as shown

[0095] In the above processing, the following formula is used to calculate the high segmentation threshold and the low segmentation threshold for image segmentation:

[0096] thresh = thresh max * θ + thresh min *(1 - θ) (6)

[0097] In the formula, thresh is the actual segmentation threshold used, thresh max and thresh min are respectively the upper and lower limits of the preset segmentation threshold, and θ is the proportionality coefficient;

[0098] And the proportionality coefficient θ is set according to the following formula:

[0099]

[0100] d = M - m (7)

[0101] In the formula, v is the variance of each column of pixel columns, m and M are respectively the minimum and maximum values of the variances of each column of pixels in the image, and d represents the maximum difference of the variances of each column of pixel columns in the image.

[0102] Then use the segmentation threshold to segment the image using the following formula:

[0103]

[0104] In the formula, v i,j is the pixel value after segmentation processing of the pixel point at coordinates (i, j), p i,j is the original pixel value of the pixel point at coordinates (i, j), thresh is the segmentation threshold, and is the high segmentation threshold or the low segmentation threshold.

[0105] 5.3) Perform dilation processing on the binary image B(i, j) of the darker area using a 5×5 window to obtain Figure 6 the suspected area template image M(i, j) as shown

[0106] 5.4) Segment the grayscale difference image D(i, j) using the low segmentation threshold to obtainFigure 7 The binarized image of the darker area shown

[0107] 5.5) Perform an AND operation on the binarized image of the darker area and the suspected area template image M(i, j) to obtain a binarized contour image C(i, j) as shown Figure 8 in the figure

[0108] 5.6) Obtain all connected regions in the binarized contour image C(i, j) as contours through connected component recognition. One connected region is taken as one contour, and all contours are filtered according to the circularity and length of each contour to obtain all slender contours suspected of being hair

[0109] The circularity of the contour is calculated using the following formula

[0110]

[0111] In the formula, degree is the circularity of the contour, area is the enclosed area of the contour, and r is the radius of the smallest circle enclosing the contour

[0112] 5.7) Connect adjacent contours among all the slender contours in step 5.6), and the result is as shown Figure 9 in the figure, and filter non-hair defects according to the actual hair length to determine whether there are hair defects

[0113] 5.7.1) Judge each slender contour

[0114] If the length of the slender contour is greater than or equal to the preset hair length, then the slender contour is a hair defect, and it is considered that there is a hair foreign object, and the next step is not performed

[0115] If the length of the slender contour is less than the preset hair length, then the slender contour is used as a suspected contour, and the next step is performed

[0116] 5.7.2) Initially connect the suspected contour with other suspected contours within the actual adjacent distance around it to form a connected contour. Specifically, first calculate the actual adjacent distance between the contours using the outer bounding rectangle between the contours, and perform an initial connection and save it within the actual adjacent distance to save calculation. The actual adjacent distance is determined according to the following formula

[0117] dis = min(dis i,j )

[0118]

[0119] In the formula, dis is the actual adjacent distance, dis i,j is the distance between the i-th pixel point of the first contour and the j-th pixel point of the second contour, p i and pj They are the pixel values of the pixel points on the two contours respectively;

[0120] 5.7.3) For all the connected contours obtained in 5.7.2), calculate the length of the minimum bounding rectangle of the connected contour and make a judgment:

[0121] If the length is less than the preset hair length, discard it;

[0122] If the length is greater than or equal to the preset hair length, the connected contour is a hair defect, and it is considered that there is a hair foreign object.

[0123] Embodiment 2

[0124] The original image with hair is as Figure 10 shown, and the result after being processed in the same way as in Embodiment 1 is as Figure 11 shown.

[0125] Embodiment 3

[0126] The original image with hair is as Figure 12 shown, and the result after being processed in the same way as in Embodiment 1 is as Figure 13 shown.

[0127] After being implemented multiple times by multiple embodiments of the present invention, the accuracy rate of the method of the present invention reaches 93%.

[0128] As can be seen from the comparison between the original images and the detection result images of the above respective embodiments, the present invention can accurately locate and detect hair foreign objects on the coating surface. In addition, the image processing speed is fast. On a desktop computer with an i7-12700 processor as the CPU, the detection time of a coating grayscale image with a size of 8192×3000 per frame is not more than 150 ms, and if there are obvious hairs, the time can be controlled within 80 ms, which has great application value in the detection of hair foreign objects on the coating surface.

[0129] The above specific embodiments are used to explain and illustrate the present invention, rather than to limit the present invention. Any modifications and changes made to the present invention within the spirit and scope of the protection of the claims of the present invention fall within the protection scope of the present invention.

Claims

1. A method for detecting hair foreign bodies on coated surfaces based on fast local threshold segmentation, characterized in that The method comprises the following steps: 1) Taking a photo of the coated surface to obtain a surface grayscale image P(i, j); 2) The surface grayscale image P(i,j) is processed by Gaussian filtering and subtraction to obtain the grayscale difference image D(i,j); 3) Use a large threshold to segment the grayscale difference image D(i,j), traverse each pixel point of the segmented binary image to obtain the connected domain and use it as the outline of the suspected obvious hair, and calculate the length and circularity of the outline to determine whether it is a hair: If it is a hair, it is judged that there is a hair foreign body, otherwise jump to the next step; 4) Further image analysis and processing is performed on the grayscale difference image D(i,j) to determine whether there is a hair defect, and then determine whether there is a hair foreign body; The step 4) is specifically as follows: 4.1) Calculate the variance of the pixel values ​​of all pixels in each column of the grayscale difference image D(i,j) as the variance of each pixel column, and obtain the maximum and minimum values ​​of the variances of all pixel columns; 4.2) Calculate the high segmentation threshold according to the variance of each pixel column and the maximum and minimum values ​​of the variance of each pixel column, and segment the grayscale difference image D(i, j) using the high segmentation threshold to obtain the darker area binary image B(i, j); 4.3) Dilate the darker area binary image B(i,j) to obtain the suspected area template image M(i,j); 4.4) Calculate the low segmentation threshold according to the variance of each pixel column and the maximum and minimum values ​​of the variance of each pixel column, and segment the grayscale difference image D(i,j) using the low segmentation threshold to obtain a binary image of the dark area; 4.5) Perform an AND operation on the dark area binary image and the suspected area template image M(i,j) to obtain a binary contour image C(i,j); 4.6) Obtain all contours in the binary contour image C(i,j), and filter all contours according to circularity and length to obtain all slender contours suspected to be hair; 4.7) Connecting all the elongated contours in step 4.6) and filtering non-hair defects according to the actual hair length to determine whether there are hair defects; The step 4.7) is specifically as follows: 4.7.1) Judge each slender contour: If the length of the thin and long contour is greater than or equal to the preset hair length, the thin and long contour is a hair defect and is considered to contain a hair foreign body, and the next step is not performed; If the length of the slender contour is less than the preset hair length, the slender contour is taken as a suspected contour and the next step is performed; 4.7.2) The suspected contour is preliminarily connected with other suspected contours around it to form a connected contour. Specifically, the actual adjacent distance between the contours is calculated using the outer enclosing rectangle between the contours, and the preliminary connection is performed and saved within the actual adjacent distance. The actual adjacent distance is determined by the following formula: dis=min(dis i,j ) In the formula, dis is the actual adjacent distance, dis i,j is the distance between the i-th pixel of the first contour and the j-th pixel of the second contour, p i and p j are the pixel values ​​of the pixels on the two contours respectively; 4.7.3) For all connected contours obtained in 4.7.2), calculate the length of the minimum enclosing rectangle of the connected contours and make a judgment: If the length is shorter than the preset hair length, it will be discarded; If the length is greater than or equal to the preset hair length, the connected contour is a hair defect and it is considered that there is a hair foreign body.

2. The method for detecting hair foreign bodies on a coating surface based on fast local threshold segmentation according to claim 1, characterized in that: The step 2) is specifically as follows: 2.1) Perform Gaussian filtering on the surface grayscale image P(i,j) to form a blurred image; 2.2) Use the blurred image to subtract the surface grayscale image P(i,j) to obtain the grayscale difference image D(i,j).

3. The method for detecting hair foreign bodies on a coating surface based on fast local threshold segmentation according to claim 2, characterized in that: In the step 2.1), the original surface grayscale image is filtered according to the formula: Where f(x,y) is the signal value after filtering, x and y represent the horizontal and vertical coordinates of the image respectively, s represents the standard deviation of the Gaussian distribution, and e represents the natural constant.

4. The method for detecting hair foreign bodies on a coating surface based on fast local threshold segmentation according to claim 1, characterized in that: In step 3) or step 4.6), the circularity of the contour is calculated using the following formula: Where degree is the circularity of the contour, area is the area enclosed by the contour, and r is the minimum radius of the circle enclosing the contour.

5. The method for detecting hair foreign bodies on a coating surface based on fast local threshold segmentation according to claim 1, characterized in that: In the step 4.1), the variance of the pixel values ​​of all the pixel points in every five columns is used as the variance of each pixel column in the five columns of pixel points.

6. The method for detecting hair foreign bodies on a coating surface based on fast local threshold segmentation according to claim 1, characterized in that: In step 4.1), the variance of each pixel column is calculated using the following formula: In the formula, S 2 is the variance of the m*n size image, is the average value of all pixel points in the area where each pixel column is located, and p(i,j) is the pixel value of the pixel point with coordinates (i,j); In step 4.2) and step 4.4), the high segmentation threshold and the low segmentation threshold used for image segmentation are calculated using the following formula: thresh=thresh max *q+thresh min *(1-q) In the formula, thresh is the actual segmentation threshold, thresh max and thresh min are the preset upper and lower limits of the segmentation threshold, respectively, and q is the proportional coefficient; And the proportional coefficient q is set according to the following formula: d=Mm Where v is the variance of each pixel column, m and M are the minimum and maximum variances of each pixel column of the image, and d represents the maximum difference of the variances of each pixel column of the image; In step 4.2) and step 4.4), the image is segmented using the following formula using the segmentation threshold: In the formula, v i,j is the pixel value after the pixel point segmentation at coordinate (i, j), p i,j is the original pixel value of the pixel at coordinate (i, j), and thresh is the segmentation threshold.

Images