Bottle cap multi-defect detection method based on regular machine vision

Abstract

According to the bottle cap multi-defect detection method based on regular machine vision, four defect detection models which do not need to be trained are constructed, and the purposes that the number of image samples is small, the relative positions of a lens and the bottle caps in the horizontal direction are not fixed, and the bottle caps are numerous in type are achieved; and bottle cap defect detection under the conditions of no uniform normal bottle cap template and the like is realized. The method comprises the following steps: (1) graying an image; (2) deformation detection; (3) scratch detection; (4) rubber ring damage detection; (5) stain detection; and (6) summarizing detection results.

Description

Technical Field

[0001] The present invention aims to identify and locate different defects of bottle caps (deformation, scratches, rubber ring damage and stains), and proposes corresponding identification and positioning methods for each defect type, involving a series of machine vision algorithms such as corrosion, expansion, Gaussian filtering and Hough circle transform, and belongs to the field of defect detection technology. Background Art

[0002] Bottle caps are widely used in packaging liquid products. Whether they are qualified or not has a great impact on the quality of the products in the container. This is not only related to the sealing of the product container, but also directly related to the safety of consumers. During the production process, defects on the bottle caps (such as deformation, scratches, stains, and damaged rubber rings) may seriously affect the overall appearance and sealing performance of the container, thereby reducing the quality of the product, and even causing product contamination or leakage, which in turn affects the safety of consumers.

[0003] In order to effectively deal with these potential problems, it is particularly necessary to adopt automated defect detection algorithms. Automated detection algorithms can significantly improve the accuracy and consistency of detection, avoid the subjectivity and errors of manual detection, and ensure that every bottle cap shipped out of the factory meets quality standards. However, automated detection algorithms often face problems such as a small number of image samples, poor results of deep learning detection algorithms, unfixed relative positions of the lens and the bottle cap in the horizontal direction, and a large variety of bottle caps without a unified normal bottle cap template.

[0004] Therefore, the present invention proposes a method for detecting multiple defects of bottle caps based on rule-based machine vision. The method has the following advantages: 1) the model in the method does not need to be trained, and only the parameters related to the rubber ring size need to be manually adjusted according to a very small number of pictures, which greatly reduces the model's demand for the number of samples; 2) the center of the bottle cap can be automatically located, solving the problem of the relative position of the lens and the bottle cap in the horizontal direction not being fixed; 3) based on the principle of detecting the characteristics of different defects, defects of different types of bottle caps can be detected, and no template matching operation is required, so a unified normal bottle cap template is not required.

[0005] The process of the proposed method can reflect the above advantages: first, the original image is converted into a grayscale image, and then the grayscale image is input into the four defect detection models designed by the present invention. Each model does not need to be trained. After automatically locating the bottle cap and its different areas, the features of various defects are extracted according to the image processing rules determined by the model structure. Among them, the deformation detection model will output a binary label to determine whether the bottle cap is deformed. The scratch, rubber ring damage and stain detection model outputs a binary label to determine whether the corresponding defect occurs and a mask image for locating the defect position and measuring the defect size. The labels output by the four models will be spliced ​​into a vector of length four as the overall label for determining whether the bottle cap has defects. The mask image output by the scratch, rubber ring damage and stain detection model will be superimposed on the original image to mark the location of different types of defects, so as to realize the recognition and positioning of multiple defects of the bottle cap. Summary of the invention

[0006] The purpose of the present invention is to propose a method for detecting multiple defects of bottle caps based on rule-based machine vision. This method is an automated detection algorithm based on python. By constructing four defect detection models that do not require training, it realizes bottle cap defect detection under conditions such as a small number of image samples, an unfixed horizontal relative position between the lens and the bottle cap, a large number of bottle cap types, and no unified normal bottle cap template. However, this method requires that the vertical distance between the camera lens and the bottle cap is fixed and the rubber ring sizes of different bottle caps are approximately the same (the average absolute percentage error of the inner and outer ring radii of the group of bottle caps (the true value is regarded as the average value of the inner and outer radii, and the predicted value is regarded as the inner and outer radii of each bottle cap rubber ring) is less than or equal to 4%). The steps of this method are as follows:

[0007] S1 image reading and grayscale conversion

[0008] The three-channel original image is read in as a three-dimensional array and converted to grayscale, as shown in Figure A.

[0009] S2 Bottle Cap Deformation Detection

[0010] Copy image A, assign A to B, input B into the deformation detection model to detect whether the bottle cap is deformed.

[0011] 1. Use Gaussian filtering on B to reduce the impact of noise.

[0012] 2. Perform gradient operation on B to find its edge.

[0013] 3. Use Otsu's method to binarize B.

[0014] 4. Perform three consecutive gradient operations on B with different kernel sizes to expand the edge outward to reduce the impact of uneven brightness around the bottle cap on edge detection.

[0015] 5. Perform three consecutive closing operations with different kernel sizes on the B output image to smooth the contour of the edge curve.

[0016] 6. Perform contour search on the B image to obtain the contour containing the largest number of pixels, which is recorded as B1.

[0017] 7. Get the convex hull shape of B1, perform a fitting operation on the convex hull, and record the output convex hull shape as B2.

[0018] 8. Use a smooth spline curve to fit B2 to obtain the contour line B3 of the bottle cap.

[0019] 9. Project B3 onto multiple straight lines with different angles to obtain a set consisting of a set of projection lengths, and calculate the variance B4 of the set.

[0020] 10. Determine whether B4 exceeds the set threshold. If it exceeds, it is considered as deformation.

[0021] S3 Bottle Cap Scratch Detection

[0022] The scratch detection model consists of three sub-models: the original image mask model, the pattern mask model, and the text mask model. Therefore, copy image A, assign A to C, D, and E, input the three sub-models respectively, and perform AND operations on the outputs of the three sub-models to obtain the detection results of the bottle cap scratches.

[0023] 1. Original image mask model

[0024] (1) Assign C to C0 and use Gaussian filtering on C0 to reduce the impact of noise.

[0025] (2) Use Hough circle transform to detect the center of the circle in C0, and draw a circle with a radius equal to the radius of the inner area of ​​the bottle cap rubber ring based on the detected center of the circle. The pixel value of the inner area of ​​the circle is set to 255, and the pixel value of the outer area is set to 0, thus obtaining the mask C1 of the inner area of ​​the bottle cap rubber ring.

[0026] (3) Use Gaussian filtering on C to reduce the impact of noise.

[0027] (4) Use a convolution filter to sharpen C.

[0028] (5) Perform an AND operation on C and C1, assign the value to C, and obtain the grayscale image under the mask area.

[0029] (6) Use adaptive Gaussian binarization for C.

[0030] (7) Perform an AND operation on C and C1, assign the value to C, and obtain the binary image under the mask area.

[0031] (8) Invert the binary image C to obtain the original image mask.

[0032] 2. Pattern Mask Model

[0033] (1) Use Gaussian filtering on D to reduce the impact of noise.

[0034] (2) Assign the result of the AND operation of D and C1 to D.

[0035] (3) Perform a reverse threshold operation on D to obtain a binary image.

[0036] (4) Perform corrosion and dilation operations on D for different times.

[0037] (5) Invert the binary image D to obtain the pattern mask.

[0038] 3. Text Mask Model

[0039] (1) Assign the result of the AND operation of E and C1 to E.

[0040] (2) Determine the relationship between the overall brightness of E and the set value, and increase or decrease the overall pixel value according to the set ratio.

[0041] (3) Use Gaussian filtering on E to reduce the impact of noise.

[0042] (4) Use a convolution filter to sharpen E.

[0043] (5) Assign the result of the AND operation of E and C1 to E.

[0044] (6) Perform a reverse threshold operation on E to obtain a binary image.

[0045] (7) Invert the binary image E.

[0046] (8) Perform corrosion operation on E.

[0047] (9) The text is detected by detecting and retrieving the outermost contour of E, and a list E1 containing a series of contours is obtained.

[0048] (10) Traverse E1, obtain the minimum circumscribed rectangle of all contours, and filter E1 according to the shape of the rectangle. The filtered contours will be covered with rectangles of a specific size. The set of all rectangles used for covering is recorded as E2.

[0049] (11) Reset all pixels of E to 255, traverse E2, and set the pixel value of each pixel covered by the rectangle of E2 in image E to 0 to obtain the text mask.

[0050] 4. Perform an AND operation on C, D, and E, and assign the result to C.

[0051] 5. Perform corrosion and expansion operations on C with different numbers and kernel sizes.

[0052] 6. Perform connected component detection on C to obtain a connected component set, recorded as C2.

[0053] 7. Set an initial threshold C3, determine the number of connected components in C2 whose area exceeds the threshold C3, and if the number exceeds a certain value C4, expand C3.

[0054] 8. Filter the connected components in C2 according to the area threshold C3, select the components whose area is larger than C3, and reassign the set of these components to C2.

[0055] 9. Set all pixel values ​​of C to 0, and set the pixel values ​​of the overlapping pixels in the connected areas of C and C2 to 255.

[0056] 10. The model outputs a binary number Label_s that reflects whether C2 is empty, 0 if empty, 1 if not empty, and the scratch mask C.

[0057] S4 Bottle Cap Rubber Ring Inspection

[0058] Copy image A, assign A to F, input F into the rubber ring detection model to detect whether the bottle rubber ring is damaged.

[0059] 1. Assign F to F0 and use Gaussian filtering on F0 to reduce the impact of noise.

[0060] 2. Use Hough circle transform to detect the center of the circle in F0, and based on the detected center, draw concentric circles consisting of a circle with a radius equal to the radius of the inner area of ​​the bottle cap rubber ring and a circle with a radius equal to the radius of the outer area of ​​the bottle cap rubber ring. The pixel value of the inner area of ​​the concentric circles is set to 255, and the pixel value of the outer area is set to 0, and the mask F1 of the bottle cap rubber ring area is obtained.

[0061] 3. Use Gaussian filtering on F to reduce the impact of noise.

[0062] 4. Perform an AND operation on F and F1, assign the value to F, and obtain the grayscale image under the mask area.

[0063] 5. Calculate the grayscale histogram of the mask area in F, find the 3σ interval of the pixel grayscale values ​​in the mask area, and take the value at the right end of the interval as the threshold F2.

[0064] 6. For the pixels in the mask area inside F, the pixel values ​​greater than F2 will be set to 255, and the pixel values ​​less than or equal to F2 will be set to 0.

[0065] 7. Perform a closed operation on F.

[0066] 8. Perform connected area detection on F to obtain a connected component set, denoted as F3.

[0067] 9. Filter the connected regions in F3 whose area is less than or equal to a certain value.

[0068] 10. Set all pixel values ​​of F to 0, and set the pixel values ​​of the overlapping pixels in the connected areas of F and F3 to 255.

[0069] 11. The model outputs a binary number Label_r that reflects whether F3 is empty, 0 if empty, 1 if not empty, and the rubber ring damage mask F.

[0070] S5 Bottle Cap Internal Stain Detection

[0071] The stain detection model consists of the stain detection model of the inner area of ​​the rubber ring and the stain detection model on the rubber ring. Therefore, copy graph A, assign A to G and H, input the two sub-models respectively, and perform an OR operation on the outputs of the two sub-models to obtain the detection result of the stain inside the bottle cap.

[0072] 1. Stain detection model for the inner area of ​​the rubber ring

[0073] (1) Assign G to G0 and use Gaussian filtering on G0 to reduce the impact of noise.

[0074] (2) Use Hough circle transform to detect the center of the circle in G0, and draw a circle with a radius equal to the radius of the inner area of ​​the bottle cap rubber ring based on the detected center of the circle. The pixel value of the inner area of ​​the circle is set to 255, and the pixel value of the outer area is set to 0, thus obtaining the mask G1 of the inner area of ​​the bottle cap rubber ring.

[0075] (3) Use Gaussian filtering on G to reduce the impact of noise.

[0076] (4) Obtain the pixel values ​​of all pixels in the mask area G1 of the image G, and record the set of them as G2.

[0077] (5) Convert G2 into a matrix and use the Otsu method to find the threshold G3 for subsequent binarization.

[0078] (6) Assign G and G1 and the operation result to G.

[0079] (7) The mask area of ​​G is binarized according to the threshold G3. For pixels whose values ​​are less than the threshold, their pixel values ​​are set to 255, otherwise they are set to 0.

[0080] (8) Perform corrosion operation on G.

[0081] (9) Perform connected area detection on G and obtain a set of connected areas, denoted as G4.

[0082] (10) Filter the connected regions in G4 whose area is less than or equal to a certain value.

[0083] (11) Set all pixel values ​​of G to 0, and set the pixel values ​​of the overlapping pixels in the connected regions of G and G4 to 255.

[0084] (12) The model outputs a binary number Label_s_in that reflects whether G4 is empty, 0 for empty and 1 for non-empty, as well as a mask G that reflects the stain condition of the inner area of ​​the rubber ring.

[0085] 2. Stain detection model on rubber ring

[0086] (1) Assign H to H0 and use Gaussian filtering on H0 to reduce the impact of noise.

[0087] (2) Use Hough circle transform to detect the center of the circle in H0, and based on the detected center, draw concentric circles consisting of a circle with a radius equal to the radius of the inner area of ​​the bottle cap rubber ring and a circle with a radius equal to the radius of the outer area of ​​the bottle cap rubber ring. The pixel value of the inner area of ​​the concentric circles is set to 255, and the pixel value of the outer area is set to 0, thereby obtaining the mask H1 of the bottle cap rubber ring area.

[0088] (3) Use Gaussian filtering on H to reduce the impact of noise.

[0089] (4) Perform an AND operation on H and H1, assign the value to H, and obtain the grayscale image under the mask area.

[0090] (5) Calculate the grayscale histogram of the masked area in H, find the 3σ interval of the pixel grayscale values ​​in the masked area, and take the value at the left end of the interval as the threshold H2.

[0091] (6) For the pixels in the mask area inside H, the pixel values ​​less than H2 will be set to 255, and the pixel values ​​greater than or equal to H2 will be set to 0.

[0092] (7) Perform connected region detection on H and obtain a set of connected regions, denoted as H3.

[0093] (8) Filter the connected areas in H3 whose area is less than or equal to a certain value.

[0094] (9) Set all pixel values ​​of H to 0, and set the pixel values ​​of the overlapping pixels in the connected areas of H and H3 to 255.

[0095] (10) The model outputs a binary number Label_s_on that reflects whether H3 is empty, 0 for empty and 1 for non-empty, as well as a mask H that reflects the stain on the rubber ring.

[0096] 3. Add G and H and assign the result to G. Perform an OR operation on Label_s_in and Label_s_on and assign the result to Label_s.

[0097] 4. The model outputs a binary number Label_s that reflects whether the bottle cap has stains. If there is no stain, it is 0, if there is stain, it is 1, and a mask G that reflects the stain status of the bottle cap.

[0098] S6 test results summary

[0099] 1. Superimpose C, F, and G to obtain the total defect area mask.

[0100] 2. Concatenate the labels output by each model to get the total label.

[0101] 3. Superimpose the three-channel color original image with the total defect area mask and mark the location where the defect occurs. BRIEF DESCRIPTION OF THE DRAWINGS

[0102] Figure 1 It is a schematic diagram of the overall process of the invention.

[0103] Figure 2 It is a flow chart of the deformation detection model.

[0104] Figure 3 It is a flow chart of the scratch detection model.

[0105] Figure 4 It is a flow chart of the rubber ring detection model.

[0106] Figure 5 It is a flow chart of the stain detection model.

[0107] Figure 6 It is a schematic diagram of some test results. DETAILED DESCRIPTION

[0108] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments. The overall process of defect detection is as follows: Figure 1 In addition, Figure 1 The four defect detection models shown will be used as four parallel threads in the program to improve the detection speed.

[0109] S1 image reading and grayscale conversion

[0110] The three-channel original image of size (3, 2448, 2048) is read in as a three-dimensional array, and then grayscaled to obtain a two-dimensional array A. The grayscale formula is as follows:

[0111] A=(38×red+75×green+15×blue)>>7 (1)

[0112] Among them, red, green, and blue represent the pixel values ​​of the red, green, and blue channels of the original image respectively.

[0113] S2 Bottle Cap Deformation Detection

[0114] A is assigned to B, and B is input into the deformation detection model to detect whether the bottle cap is deformed. The steps are as follows Figure 2 shown.

[0115] 1. To reduce the impact of noise, first perform Gaussian filtering on B. The two-dimensional Gaussian distribution formula is as follows:

[0116]

[0117] Among them, x and y represent the horizontal and vertical coordinates of each element in the Gaussian kernel matrix Gaussian, with the center of the matrix as the reference. The Gaussian kernel is a square matrix with an odd length, σ 1 and σ 2 They are used to control the expansion degree of the Gaussian kernel in the horizontal and vertical directions respectively.

[0118] When Gaussian filtering is performed on B, the Gaussian kernel size is set to (57, 57), and the parameter σ that controls the degree of expansion in the horizontal and vertical directions is set 1 and σ 2 The value of is 12. Gaussian filtering uses the Gaussian kernel matrix Gaussian as the convolution kernel to convolve the original image. The formula is as follows:

[0119]

[0120] Among them, B 1 represents the grayscale image after Gaussian filtering, w G and h G They represent the length and width of the Gaussian kernel matrix, both of which are set to 57. 11 and b 12 Respectively represent B 1 The horizontal and vertical coordinates of the pixel point.

[0121] 2. For B 1 Perform a gradient operation, that is, subtract the image after the expansion operation from the image after the erosion operation, set the kernel shape to an ellipse, and the size to (30, 30). The image obtained after the gradient operation is recorded as B 2 .

[0122] 3. Use Otsu's method to find the threshold t 1 To B 2 Binarization is performed, the formula is as follows:

[0123]

[0124] Among them, p t1 The pixel value of all pixels is less than the threshold t 1 The probability, p iis the probability that the pixel value is i. According to the threshold t 1 , for grayscale image B 2 Binarize and get B 3 , the formula is as follows:

[0125]

[0126] Among them, b 31 and b 32 They are B 3 The horizontal and vertical coordinates of the pixel point.

[0127] 4. For B 3 Perform three consecutive gradient operations with different kernel sizes to expand the edge outward to reduce the impact of uneven brightness on edge detection. Set the kernel shape to ellipse. In the three gradient operations, the kernel sizes are (30, 30), (15, 15) and (5, 5) respectively. 3 After gradient calculation, we get B 4 .

[0128] 5. For B 4 Perform three consecutive closing operations with different kernel sizes, that is, dilation followed by erosion, to smooth the contour of the edge curve. Set the kernel shape to an ellipse. In the three closing operations, the kernel sizes are (30, 30), (15, 15), and (3, 3) respectively. 4 After the closed operation, we get B 5 .

[0129] 6. For B 5 The image uses the cv2.findcounters method to find the contour, only detecting the outermost contour and finding the contour with the largest number of pixels, which is denoted as B. 6 , is a set of points.

[0130] 7. Use cv2.convexHull method to calculate B 6 The convex hull shape of the point is recorded as the point coordinate set B 7 .

[0131] 8. To B 7 The abscissa and ordinate of are fitted using cubic polynomials, respectively, and the formula is as follows:

[0132]

[0133] in and Represent the fitting curves in the horizontal and vertical directions respectively, n x and n y Represents the independent variables of the fitting curve in the horizontal and vertical directions, j = 0, 1, 2, ..., nB7 -1,n B7 Representative B 7 The number of midpoints, a j , b j , c j , d j , e j , f j , g j ,h j Represents the parameters in a cubic polynomial, determined by the following formula:

[0134]

[0135] Formula (7) represents the interpolation condition, B 7 (j) represents the point coordinate set B 7 The coordinates of the point with index j in , formula (8) is the continuity condition, formula (9) is the first-order derivative continuity condition, and They are and The first-order derivative of , formula (10) is the continuity condition of the second-order derivative, and They are and The second derivative of .

[0136] The number of fitting points on the horizontal and vertical axes is set to n e = 2000, the contour line of the bottle cap is obtained using the horizontal and vertical coordinates refined after fitting, that is, the point coordinate set B 8 , the formula is as follows:

[0137] B 8 (j e )=(X B7 (j e ),Y B7 (j e ))

[0138]

[0139] Among them, j e Represents the sampling points in the fitting curve, and also represents the set B 8 The index of the element in .

[0140] 9. For B 8 Projecting on multiple straight lines with different angles, the set of straight lines can be expressed by the following formula:

[0141]

[0142] Among them, x l and lrepresents the horizontal and vertical coordinates of the point on the line, o represents the index of the line in the set, and n l Indicates the number of lines in the set, set n l =100, a set of projection lengths is obtained, and the variance B of the set is calculated 9 .

[0143] 10. According to experience, the threshold of deformation is set to 100, and the 9 Whether it exceeds the set threshold, if it exceeds, it is considered as deformation. The model outputs the label Label_def representing whether the detected object is deformed. The deformation is 1 and the normal is 0.

[0144] S3 Bottle Cap Scratch Detection

[0145] The scratch detection model consists of three sub-models: original image mask model, pattern mask model and text mask model. Figure 3 When testing, copy Figure A, assign A to C, D, and E, input C, D, and E into the three sub-models respectively, and perform AND operations on the outputs of the three sub-models to obtain the test results of the bottle cap scratches.

[0146] 1. C input original image mask model

[0147] (1) Assign C to C 0 , for C 0 Use Gaussian filtering, set the Gaussian kernel size to (13, 13), and control the horizontal and vertical expansion parameters to 2. After Gaussian filtering, we get image C. 1 .

[0148] (2) Detect C using Hough circle transform 1 The center of the circle in the image is detected, and a circle with a radius equal to the radius of the inner area of ​​the rubber ring of the bottle cap is drawn based on the detected center of the circle. The pixel value of the inner area of ​​the circle is set to 255, and the pixel value of the outer area is set to 0, and the mask C of the inner area of ​​the rubber ring of the bottle cap is obtained. 2 , the formula is as follows:

[0149]

[0150] Among them, c 21 , c 22 Represents C 2 The horizontal and vertical coordinates of the pixel point in the middle, o c21 , o c22 Represents the horizontal and vertical coordinates of the detected center of the circle, r 0 Represents the inner radius of the rubber ring of the bottle cap. Since it is assumed that the vertical distance between the camera and the bottle cap is fixed and the rubber rings of different types of bottle caps are of the same size, this method only needs to measure the inner diameter of the rubber ring of one image before using it to determine r 0 .

[0151] (3) Use Gaussian filtering on C, set the Gaussian kernel size to (11, 11), and set the parameters controlling the horizontal and vertical expansion degree to 2 to obtain image C. 3 .

[0152] (4) For C 3 Use the convolution filter to sharpen the image, and get image C 4 , the convolution kernel is set as follows:

[0153]

[0154] (5) C 4 and C 2 Perform AND operation to get C 5 , and obtain the grayscale image under the mask area.

[0155] (6) For C 5 Use adaptive Gaussian binarization to get the binary image C 6 .

[0156] (7) C 6 and C 2 Perform AND operation to obtain the binary image C under the mask area 7 .

[0157] (8) Let C 8 =255-C 7 , reverse C 7 Get the original image mask C 8 .

[0158] 2. Pattern Mask Model

[0159] (1) Use Gaussian filtering on D, set the Gaussian kernel size to (11, 11), and set the parameters controlling the expansion degree in the horizontal and vertical directions to 2, and obtain D 0 .

[0160] (2) D 0 and C 2 Perform AND operation to get D 1 .

[0161] (3) For D 1 Perform a reverse threshold operation and set the threshold to 127 to obtain the binary image D 2 .

[0162] (4) Set the kernel to an ellipse with a size of (3, 3) and perform the binary image D 2 Perform 2 dilations and 5 erosions to get D 3 .

[0163] (5) Let D 4 =255-D3 , and obtain the pattern mask.

[0164] 3. Text Mask Model

[0165] (1) E and C 2 Perform AND operation to get E 0 .

[0166] (2) Set the standard brightness value to s b =159, according to s b Adjust the image brightness to get image E 1 , the formula is as follows:

[0167] E 1 (e 11 ,e 12 )=αE 0 (e 11 ,e 12 )+(s b -b a ) (14)

[0168] Among them, e 11 , e 12 They represent the horizontal and vertical coordinates of the pixels in the image, α represents the multiple of brightness increase, which is set to 1 here, and b a Represents E 0 The average brightness.

[0169] (3) To E 1 Use Gaussian filtering, set the Gaussian kernel size to (13, 13), and set the parameters controlling the expansion degree in the horizontal and vertical directions to 2, and get E 2 .

[0170] (4) To E 2 Use the convolution filter to sharpen and get E 3 , the convolution kernel is set as follows:

[0171]

[0172] (5) E 3 and C 2 Perform AND operation to get E 4 .

[0173] (6) To E 4 Perform a reverse threshold operation and set the threshold to the standard brightness value s b =159, and the binary image E is obtained. 5 .

[0174] (7) Let E 6 =255-E 5 , get the inverted binary image E6 .

[0175] (8) To E 6 Perform an erosion operation and set the core to an elliptical core of (2, 2) to obtain E 7 .

[0176] (9) To E 7 Use cv2.findcounters method to find contours and retrieve E by detection 7 Detect text using the outermost contour to obtain a set E containing a series of contours. 8 .

[0177] (10) Traverse E 8 , use the cv2.boundingRect method to obtain the minimum bounding rectangle of all contours and get the set E 9 , since the rectangle can be simply expressed as the reference point and the length and width, E 9 It can be written as:

[0178]

[0179] in, Respectively represent the horizontal and vertical coordinates of the rectangular reference point, Respectively represent the length and width of the rectangle, n e9 Represents set E 9 Length.

[0180] (11) Since the camera height is fixed, E can be adjusted according to the size range of the text. 9 Filter to select the bounding rectangles whose length or width is greater than 25 pixels and whose length and width are both less than 100 pixels. The set is recorded as E 10 .

[0181] (12) E 10 Take the reference point of each circumscribed rectangle as the reference and expand the length and width by 2s e pixels, set s e =9, draw a new rectangle, and its set is recorded as E 11 , the formula is as follows:

[0182] E 11 (j e11 )=Expend(E 10 (j e11 )) j e11 =0,1,2,...,n e11 -1 (16)

[0183]

[0184] n in formula (16) e11Represents set E 11 The length of E 10 To E 11 The mapping is specifically expressed as formula (17). In formula (17) Respectively represent E 11 The horizontal and vertical coordinates of the reference point of the rectangle, Respectively represent E 11 The length and width of the middle rectangle, Respectively represent E 10 The horizontal and vertical coordinates of the reference point of the rectangle, Respectively represent E 10 The length and width of the middle rectangle.

[0185] (13) Let E 12 represents an image of the same size as E, with all pixel values ​​255. 11 , image E 12 By E 11 The pixel values ​​of the pixels covered by the middle rectangle are set to 0, and the text mask E is obtained. 13 , the formula is as follows:

[0186]

[0187] Condition (e) in formula (18) 131 ,e 132 )in E 11 It is specifically expressed as formula (19), where e 131 , e 132 Represents image E 13 The horizontal and vertical coordinates of the pixel point.

[0188] (14) Let E 14 =255-E 13 , and get the text mask.

[0189] 4. Let C 9 =C 8 and D 4 and E 14 , and the binary image C after filtering the pattern and text is obtained 9 .

[0190] 5. Set the size of the ellipse kernel to (2, 2) and 9 Perform one erosion, eight dilations, and reset the size of the ellipse kernel to (3, 3). 9 Perform eight consecutive etching operations to obtain C 10 .

[0191] 6. Use the cv2.connectedComponentsWithStats method to C 10 Perform connected component detection to obtain a connected component set, denoted as C 11 In the connected component set, each connected component can be regarded as a set consisting of a set of pixel coordinates.

[0192] 7. Set the initial threshold C 12 =500, judge C 11 The area in the middle exceeds the threshold C 12 The number of connected components, if this number exceeds a certain set value C 13 =7, then let C 12 =C 12 × 6. The purpose of this step is to filter out scattered small-area patches and reduce the influence of the internal texture of the bottle cap on scratch detection.

[0193] 8. According to the area threshold C 12 C 11 Filter the components in the Unicom network and select the ones with an area larger than C 12 The components of 14 .

[0194] 9. Order C 15 represents an image of the same size as C with all pixel values ​​0, and C 15 With C 14 The pixel values ​​of the overlapping pixels in the connected region are set to 255, and C 16 , the formula is as follows:

[0195]

[0196] Among them, c 161 , c 162 Represents image C 16 The horizontal and vertical coordinates of the pixel point.

[0197] 10. Model output reflects C 14 Whether it is empty or not, the binary number Label_scr, empty is 0, non-empty is 1, and the scratch mask C 16 .

[0198] S4 Bottle Cap Rubber Ring Inspection

[0199] Assign A to F, and input F into the rubber ring detection model. The process is as follows Figure 4 As shown, check whether the bottle rubber ring is damaged.

[0200] 1. Assign F to F 0 , for F 0Use Gaussian filtering, set the Gaussian kernel size to (13, 13), and set the parameters controlling the horizontal and vertical expansion to 2 to get F 1 .

[0201] 2. Use Hough circle transform to detect F 1 The center of the circle in the bottle cap is detected, and based on the detected center of the circle, concentric circles are drawn, which are composed of a circle with a radius equal to the radius of the inner area of ​​the bottle cap rubber ring and a circle with a radius equal to the radius of the outer area of ​​the bottle cap rubber ring. The pixel value of the inner area of ​​the concentric circle is set to 255, and the pixel value of the outer area is set to 0, and the mask F of the bottle cap rubber ring area is obtained. 2 , the formula is as follows:

[0202]

[0203] Among them, f 21 , f 22 Represents F 2 The horizontal and vertical coordinates of the pixel point in the middle, o f21 , o f22 Respectively represent the horizontal and vertical coordinates of the detected center of the circle, r 0 , r 1 Respectively represent the inner and outer radius of the rubber ring of the bottle cap. Since the vertical distance between the camera and the bottle cap is fixed, and the rubber rings of different types of bottle caps have the same size, this method only needs to measure the inner diameter of the rubber ring of one picture before using it to determine r 0 With r 1 .

[0204] 3. Use Gaussian filtering on F, set the Gaussian kernel size to (11, 11), and set the parameters controlling the horizontal and vertical expansion degree to 0.8, and get F 3 .

[0205] 4. F 3 and F 2 Perform AND operation and record the result as F 4 , and obtain the grayscale image under the mask area.

[0206] 5. Calculate F 4 The grayscale histogram of the mask area is obtained, and the 3σ interval of the pixel grayscale value in the mask area is obtained. The value at the right end of the interval is taken as the threshold F 5 , the formula is as follows:

[0207] F 50 =min(255,a f4 +3×σ f4 )

[0208]

[0209] Among them, a f4 and σ f4 Represents F4 In the mask area F 2 The mean and standard deviation of the pixel values ​​of the internal pixels are set to prevent the right side of the 3σ interval from reaching 255. f50 , according to experience, f50 =5.

[0210] 6. For F 4 Pixels in the internal mask area, larger than F 5 The pixel value will be set to 255, which is less than or equal to F 5 The pixel value will be set to 0, resulting in a binary image F 6 , the formula is as follows:

[0211]

[0212] Among them, f 61 , f 62 Respectively represent F 6 The horizontal and vertical coordinates of the pixel point in the middle.

[0213] 7. Set the ellipse kernel size to (10, 10) and 6 Perform a closing operation and get F 7 .

[0214] 8. Use cv2.connectedComponentsWithStats method to 7 Perform connected area detection to obtain a connected component set, denoted as F 8 .

[0215] 9. Filter out F 8 The area is less than or equal to the threshold F 9 The connected area of ​​​​the 9 =1000, we get F 10 .

[0216] 10. Let F 11 represents an image of the same size as F, with all pixel values ​​​​set to 0. 11 With F 10 The pixel values ​​of the pixels at the overlapping positions of the connected regions are set to 255, and the F 12 The formula is as follows:

[0217]

[0218] 11. Model output reflects F 10 Whether it is empty or not, the binary number Label_rub, empty is 0, non-empty is 1, and the rubber ring damage mask F 12 .

[0219] S5 Bottle Cap Internal Stain Detection

[0220] The stain detection model consists of the stain detection model in the inner area of ​​the rubber ring and the stain detection model on the rubber ring, such as Figure 5 Therefore, copy Figure A, assign A to G and H, input the two sub-models respectively, and perform an OR operation on the outputs of the two sub-models to obtain the detection result of the stain inside the bottle cap.

[0221] 1. Stain detection model for the inner area of ​​the rubber ring

[0222] (1) Assign G to G 0 , for G 0 Use Gaussian filtering, set the Gaussian kernel size to (13, 13), and set the parameters controlling the horizontal and vertical expansion to 2, and get G 1 .

[0223] (2) Use Hough circle transform to detect G 1 The center of the circle in the image is detected, and a circle with a radius equal to the radius of the inner area of ​​the rubber ring of the bottle cap is drawn based on the detected center of the circle. The pixel value of the inner area of ​​the circle is set to 255, and the pixel value of the outer area is set to 0, and the mask G of the inner area of ​​the rubber ring of the bottle cap is obtained. 2 , as shown in formula (13).

[0224] (3) Use Gaussian filtering on G, set the Gaussian kernel size to (39, 39), and set the parameters controlling the expansion degree in the horizontal and vertical directions to 0.25, and obtain G 3 .

[0225] (4) Obtaining image G 3 Mask region G 2 The pixel values ​​of all pixels in the 4 .

[0226] (5) G 4 Convert to a matrix and use Otsu's method to transform the matrix G 4 Find the threshold G for subsequent binarization 5 .

[0227] (6) G 3 and G 2 Perform AND operation to obtain the grayscale image G under the mask area 6 .

[0228] (7) According to the threshold G 5 For G 6 The mask area is binarized. For pixels whose pixel values ​​are less than the threshold, their pixel values ​​are set to 255, otherwise they are set to 0, and the binary image G is obtained. 7 .

[0229] (8) Set the ellipse kernel size to (10, 10) and7 Perform corrosion operation to obtain G 8 .

[0230] (9) Use the cv2.connectedComponentsWithStats method to G 8 Perform connected area detection and obtain the set of connected areas, which is denoted as G 9 .

[0231] (10) Filter out G 9 The area is less than or equal to the threshold G 10 Unicom area, set G according to experience 10 =2000, the filtered set is denoted as G 11 .

[0232] (11) Let G 12 represents an image of the same size as G, with all pixel values ​​​​set to 0. 12 With G 11 The pixel values ​​of the pixels at the overlapping positions of the connected regions in the graph are set to 255, and the obtained value is G 13 , the formula is as follows:

[0233]

[0234] Among them, g 131 , g 132 Represents G 13 The horizontal and vertical coordinates of the pixel point.

[0235] (12) Model output reflects G 11 The binary number Label_sta_in indicating whether it is empty or not, 0 if empty and 1 if not empty, and the mask G reflecting the stain condition of the inner area of ​​the rubber ring 13 .

[0236] 2. Stain detection model on rubber ring

[0237] (1) Assign H to H 0 , for H 0 Use Gaussian filtering, set the Gaussian kernel size to (13, 13), and set the parameters controlling the horizontal and vertical expansion to 2 to get H 1 .

[0238] (2) Using Hough circle transform to detect H 1 The center of the circle in the bottle cap is detected, and based on the detected center of the circle, concentric circles are drawn, which are composed of a circle with a radius equal to the radius of the inner area of ​​the bottle cap rubber ring and a circle with a radius equal to the radius of the outer area of ​​the bottle cap rubber ring. The pixel value of the inner area of ​​the concentric circle is set to 255, and the pixel value of the outer area is set to 0, and the mask H of the bottle cap rubber ring area is obtained. 2 , as shown in formula (21).

[0239] (3) Use Gaussian filtering to process H, set the Gaussian kernel size to (13, 13), and set the parameters controlling the expansion degree in the horizontal and vertical directions to 1, and obtain H 3 .

[0240] (4) H 3 and H 2 Perform AND operation to obtain the grayscale image H under the mask area 4 .

[0241] (5) Calculate H 4 The grayscale histogram of the mask area is obtained, and the 3σ interval of the pixel grayscale value in the mask area is obtained. The value at the left end of the interval is taken as the threshold H 5 , the formula is as follows:

[0242] H 50 =max(0,a h4 -3×σ h4 )

[0243]

[0244] Among them, a h4 and σ h4 Represents H 4 In the mask area H 2 The mean and standard deviation of the pixel values ​​of the internal pixels are set to prevent the left side of the 3σ interval from reaching 0. h50 , according to experience, h50 =5.

[0245] (6) For H 4 Pixels in the internal mask area are smaller than H 5 The pixel value will be set to 255, which is greater than or equal to H 5 The pixel value will be set to 0, and the binary image H 6 .

[0246] (7) Use the cv2.connectedComponentsWithStats method to 6 Perform connected area detection and obtain the set of connected areas, which is recorded as H 7 .

[0247] (8) Filter H 7 The area is less than or equal to the threshold H 8 Unicom area, set H based on experience 8 =1500, and the filtered set H is obtained 9 .

[0248] (9) Let H 10 represents an image of the same size as H, with all pixel values ​​​​set to 0.10 With H 9 The pixel values ​​of the pixels at the overlapping positions of the connected regions in the figure are set to 255, and H 11 , the formula is as follows:

[0249]

[0250] (10) The model output reflects H 9 The binary number Label_sta_on indicating whether it is empty or not, 0 if empty and 1 if not empty, and the mask H reflecting the stain on the rubber ring 11 .

[0251] 3. Let G 14 =G 13 +H 11 , at the same time Label_sta=Label_sta_in or Label_sta_on.

[0252] 4. The model outputs a binary number Label_sta that reflects whether the bottle cap has stains. If there is no stain, it is 0, if there is stain, it is 1, and a mask G that reflects the stain status of the bottle cap. 14 .

[0253] S6 test results summary

[0254] Some test results and their summary process are as follows Figure 6 shown.

[0255] 1. Let Mask = C 16 +F 12 +G 14 , where Mask represents the total defect area mask.

[0256] 2. Let Label = [Label_def, Label_scr, Label_rub, Label_sta] to get the total label.

[0257] 3. Let O'=Mask+O, where O represents the three-channel color original image, and O' represents the original image after superimposing the total defect area mask.

Claims

1. A bottle cap multiple defect detection method based on rule-based machine vision, characterized in that: Here are the steps: S1 image reading and grayscale conversion The three-channel original image of size (3, 2448, 2048) is read in as a three-dimensional array, and then grayscaled to obtain a two-dimensional array A. The grayscale formula is as follows: A=(38×red+75×green+15×blue)>>7 (1) Among them, red, green, and blue represent the pixel values ​​of the red, green, and blue channels of the original image respectively; S2 Bottle Cap Deformation Detection A is assigned to B, and B is input into the deformation detection model to detect whether the bottle cap is deformed; 1) To reduce the impact of noise, first perform Gaussian filtering on B. The two-dimensional Gaussian distribution formula is as follows: Among them, x and y represent the horizontal and vertical coordinates of each element in the Gaussian kernel matrix Gaussian, with the center of the matrix as the reference. The Gaussian kernel is a square matrix with an odd length. σ1 and σ2 are used to control the expansion degree of the Gaussian kernel in the horizontal and vertical directions respectively; When performing Gaussian filtering on B, the Gaussian kernel size is set to (57, 57), and the values ​​of the parameters σ1 and σ2 that control the degree of expansion in the horizontal and vertical directions are set to 12; Gaussian filtering uses the Gaussian kernel matrix Gaussian as the convolution kernel to convolve the original image. The formula is as follows: Among them, B1 represents the grayscale image after Gaussian filtering, w G and h G They represent the length and width of the Gaussian kernel matrix, both of which are set to 57. 11 and b 12 Respectively represent the horizontal and vertical coordinates of the pixel points in B1; 2) Perform a gradient operation on B1, that is, subtract the image after the dilation operation from the image after the erosion operation, set the kernel shape to an ellipse, and the size to (30, 30). The image obtained after the gradient operation is recorded as B2; 3) Use the Otsu method to find the threshold t1 to binarize B2. The formula is as follows: Among them, p t1 is the probability that the pixel value of all pixels is less than the threshold t1, p i is the probability that the pixel value is i; according to the threshold t1, the grayscale image B2 is binarized to obtain B3, the formula is as follows: Among them, b 31 and b 32 They are the horizontal and vertical coordinates of the pixel points in B3; 4) Perform three consecutive gradient operations with different kernel sizes on B3 to expand the edge outward to reduce the impact of uneven brightness of the outer periphery of the bottle cap on edge detection; set the kernel shape to an ellipse, and in the three gradient operations, the kernel sizes are (30, 30), (15, 15) and (5, 5) respectively; B3 is subjected to gradient operation to obtain B4; 5) Perform three consecutive closing operations on B4 with different kernel sizes, i.e., dilation first and then erosion, to smooth the contour of the edge curve; set the kernel shape to an ellipse, and in the three closing operations, the kernel sizes are (30, 30), (15, 15) and (3, 3) respectively; after the closing operation of B4, B5 is obtained; 6) Use cv2.findcounters method to find the contour of B5 image, only detect the outermost contour, and find the contour with the largest number of pixels. This contour is recorded as B6, which is a set of points; 7) Use the cv2.convexHull method to calculate the convex hull shape of B6, and the output is recorded as the point coordinate set B7; 8) Use cubic polynomials to fit the horizontal and vertical coordinates of B7, respectively. The formula is as follows: in and Represent the fitting curves in the horizontal and vertical directions respectively, n x and n y Represents the independent variables of the fitting curve in the horizontal and vertical directions, j = 0, 1, 2, ..., n B7 -1,n B7 represents the number of midpoints in B7, a j , b j , c j , d j , e j , f j , g j ,h j Represents the parameters in a cubic polynomial, determined by the following formula: Formula (7) represents the interpolation condition, B7(j) represents the coordinates of the point with index j in the point coordinate set B7, formula (8) is the continuity condition, and formula (9) is the first-order derivative continuity condition. and They are and The first-order derivative of , formula (10) is the continuity condition of the second-order derivative, and They are and The second derivative of The number of fitting points on the horizontal and vertical axes is set to n e = 2000, and the contour line of the bottle cap is obtained using the horizontal and vertical coordinates refined after fitting, that is, the point coordinate set B8, and the formula is as follows: Among them, j e Represents the sampling point in the fitting curve, and also represents the index of the element in the set B8; 9) Project B8 onto multiple straight lines with different angles. The group of straight lines can be expressed by the following formula: Among them, x l and l represents the horizontal and vertical coordinates of the point on the line, o represents the index of the line in the set, and n l Indicates the number of lines in the set, set n l =100, a set consisting of a set of projection lengths is obtained, and the variance B9 of the set is calculated; 10) Set the threshold of deformation to 100, and determine whether B9 exceeds the set threshold. If it exceeds the threshold, it is considered deformed. The model outputs the label Label_def representing whether the detected object is deformed. The deformation is 1 and the normal is 0. S3 Bottle Cap Scratch Detection The scratch detection model consists of three sub-models: the original image mask model, the pattern mask model and the text mask model. During detection, image A is copied, and A is assigned to C, D and E. C, D and E are input into the three sub-models respectively, and the outputs of the three sub-models are ANDed to obtain the detection result of the bottle cap scratch. 3.

1. C Input original image mask model (1) Assign C to C0 and use Gaussian filtering to process C0. Here, the Gaussian kernel size is set to (13, 13), and the parameters controlling the expansion degree in the horizontal and vertical directions are both 2. After Gaussian filtering, image C1 is obtained; (2) Use Hough circle transform to detect the center of the circle in C1, and draw a circle with a radius equal to the radius of the inner area of ​​the bottle cap rubber ring based on the detected center of the circle. The pixel value of the inner area of ​​the circle is set to 255, and the pixel value of the outer area is set to 0. The mask C2 of the inner area of ​​the bottle cap rubber ring is obtained. The formula is as follows: Among them, c 21 , c 22 Respectively represent the horizontal and vertical coordinates of the pixel points in C2, o c21 , o c22 represents the horizontal and vertical coordinates of the detected center of the circle, and r0 represents the inner radius of the rubber ring of the bottle cap. Since it is assumed that the vertical distance between the camera and the bottle cap is fixed, and the rubber rings of different types of bottle caps have the same size, this method only needs to measure the inner diameter of the rubber ring of one picture to determine r0 before use; (3) Use Gaussian filtering on C, set the Gaussian kernel size to (11, 11), and set the parameters controlling the expansion degree in the horizontal and vertical directions to 2, and obtain image C3; (4) Use a convolution filter to sharpen C3 to obtain image C4. The convolution kernel is set as follows: (5) Perform an AND operation on C4 and C2 to obtain C5, and obtain the grayscale image under the mask area; (6) Adaptive Gaussian binarization is applied to C5 to obtain a binarized image C6; (7) Perform an AND operation on C6 and C2 to obtain a binary image C7 under the mask area; (8) Let C8 = 255 - C7, invert C7 to obtain the original image mask C8; 3.2 Pattern Mask Model (1) Use Gaussian filtering on D, set the Gaussian kernel size to (11, 11), and set the parameters controlling the expansion degree in the horizontal and vertical directions to 2, and obtain D0; (2) Perform an AND operation on D0 and C2 to obtain D1; (3) Perform a reverse threshold operation on D1, set the threshold to 127, and obtain the binary image D2; (4) Set the kernel to an ellipse with a size of (3, 3), dilate the binary image D2 twice and erode it five times to obtain D3; (5) Let D4 = 255-D3 to obtain the pattern mask; 3.3 Text Mask Model (1) Perform an AND operation on E and C2 to obtain E0; (2) Set the standard brightness value to s b =159, according to s b Adjust the image brightness and get image E1. The formula is as follows: E1(e 11 ,of 12 )=αE0(e 11 ,of 12 )+(s b -b a ) (14) Among them, e 11 , e 12 They represent the horizontal and vertical coordinates of the pixels in the image, α represents the multiple of brightness increase, which is set to 1 here, and b a represents the average brightness of E0; (3) Gaussian filtering is used on E1, the Gaussian kernel size is set to (13, 13), and the parameters controlling the expansion degree in the horizontal and vertical directions are both 2, and E2 is obtained; (4) Use the convolution filter to sharpen E2 to obtain E3. The convolution kernel is set as follows: (5) Perform an AND operation on E3 and C2 to obtain E4; (6) Perform a reverse threshold operation on E4 and set the threshold to the standard brightness value s b =159, and a binary image E5 is obtained; (7) Let E6 = 255 - E5, and obtain the inverted binary image E6; (8) E6 is corroded, and the core is set to an elliptical core of (2, 2), to obtain E7; (9) Use cv2.findcounters method to search for contours of E7, detect the text by detecting and retrieving the outermost contour of E7, and obtain a set E8 containing a series of contours; (10) Traverse E8 and use the cv2.boundingRect method to obtain the minimum bounding rectangle of all contours to obtain set E9. Since the rectangle can be simply represented by the reference point and length and width, E9 can be written as: in, Respectively represent the horizontal and vertical coordinates of the rectangular reference point, Respectively represent the length and width of the rectangle, n e9 Represents the length of set E9; (11) Since the camera height is fixed, we can filter E9 according to the size range of the text to select the bounding rectangles whose length or width is greater than 25 pixels and whose length and width are both less than 100 pixels. The set is denoted as E 10 ; (12) E 10 Take the reference point of each circumscribed rectangle as the reference and expand the length and width by 2s e pixels, set s e =9, draw a new rectangle, and its set is recorded as E 11 , the formula is as follows: E 11 (j e11 )=Expend(E 10 (j e11 )) j e11 =0,1,2,...,n e11 -1 (16) n in formula (16) e11 Represents set E 11 The length of E 10 To E 11 The mapping is specifically expressed as formula (17); in formula (17) Respectively represent E 11 The horizontal and vertical coordinates of the reference point of the rectangle, Respectively represent E 11 The length and width of the middle rectangle, Respectively represent E 10 The horizontal and vertical coordinates of the reference point of the rectangle, Respectively represent E 10 The length and width of the middle rectangle; (13) Let E 12 represents an image of the same size as E, with all pixel values ​​255. 11 , image E 12 By E 11 The pixel values ​​of the pixels covered by the middle rectangle are set to 0, and the text mask E is obtained. 13 , the formula is as follows: Condition (e) in formula (18) 131 ,e 132 )in E 11 Specifically expressed as formula (19), where e 131 , e 132 Represents image E 13 The horizontal and vertical coordinates of the middle pixel; (14) Let E 14 =255-E 13 , get the text mask; 3.

4. Let C9 = C8 and D4 and E 14 , and obtain the binary image C9 after filtering the pattern and text; 3.

5. Set the size of the ellipse kernel to (2, 2), erode C9 once, dilate it eight times, reset the size of the ellipse kernel to (3, 3), and perform eight consecutive erosion operations on C9 to obtain C 10 ; 3.6, Use cv2.connectedComponentsWithStats method to C 10 Perform connected component detection to obtain a connected component set, denoted as C 11 ; In the connected component set, each connected component is regarded as a set consisting of a set of pixel coordinates; 3.

7. Set the initial threshold C 12 =500, judge C 11 The area in the middle exceeds the threshold C 12 The number of connected components, if this number exceeds a certain set value C 13 =7, then let C 12 =C 12 ×6; 3.

8. According to the area threshold C 12 C 11 Filter the components in the Unicom network and select the ones with an area larger than C 12 The components of 14 ; 3.9, let C 15 represents an image of the same size as C with all pixel values ​​0, and C 15 With C 14 The pixel values ​​of the overlapping pixels in the connected region are set to 255, and C 16 , the formula is as follows: Among them, c 161 , c 162 Represents image C 16 The horizontal and vertical coordinates of the middle pixel; 3.10 Model output reflects C 14 Whether it is empty or not, the binary number Label_scr, empty is 0, non-empty is 1, and the scratch mask C 16 ; S4 Bottle Cap Rubber Ring Inspection 4.1 Assign A to F, input F into the rubber ring detection model to detect whether the bottle rubber ring is damaged; Assign F to F0, use Gaussian filtering on F0, set the Gaussian kernel size to (13, 13), and set the parameters controlling the expansion degree in the horizontal and vertical directions to 2, and get F1; 4.2 Use Hough circle transform to detect the center of the circle in F1, and based on the detected center of the circle, draw concentric circles consisting of a circle with a radius equal to the radius of the inner area of ​​the bottle cap rubber ring and a circle with a radius equal to the radius of the outer area of ​​the bottle cap rubber ring. The pixel value of the inner area of ​​the concentric circle is set to 255, and the pixel value of the outer area is set to 0. The mask F2 of the bottle cap rubber ring area is obtained, and the formula is as follows: Among them, f 21 , f 22 Respectively represent the horizontal and vertical coordinates of the pixel points in F2, o f21 , o f22 Respectively represent the horizontal and vertical coordinates of the detected center of the circle, r0 and r1 represent the inner and outer radii of the rubber ring of the bottle cap, respectively; 4.

3. Use Gaussian filtering on F, set the Gaussian kernel size to (11, 11), and control the expansion degree in the horizontal and vertical directions with the parameters of 0.8, and obtain F3; 4.

4. Perform an AND operation on F3 and F2, and record the result as F4 to obtain the grayscale image under the mask area; 4.

5. Calculate the grayscale histogram of the mask area in F4, find the 3σ interval of the pixel grayscale value in the mask area, and take the value at the right end of the interval as the threshold F5. The formula is as follows: Among them, a f4 and σ f4 Represent the mean and standard deviation of the pixel values ​​of F4 in the mask area F2. In order to prevent the right side of the 3σ interval from reaching 255, set δ f50 , according to experience, f50 =5; 4.

6. For the pixels in the mask area inside F4, the pixel values ​​greater than F5 will be set to 255, and the pixel values ​​less than or equal to F5 will be set to 0, and the binary image F6 is obtained. The formula is as follows: Among them, f 61 , f 62 Respectively represent the horizontal and vertical coordinates of the pixel points in F6; 4.

7. Set the ellipse kernel size to (10, 10), perform a closing operation on F6, and obtain F7; 4.

8. Use the cv2.connectedComponentsWithStats method to detect the connected area of ​​F7 and obtain the connected component set, which is recorded as F8. 4.

9. Filter out the connected regions in F8 whose area is less than or equal to the threshold F9. According to experience, set F9 = 1000 to obtain F 10 ; 4.

10. Let F 11 represents an image of the same size as F, with all pixel values ​​​​set to 0; and F 11 With F 10 The pixel values ​​of the pixels at the overlapping positions of the connected regions are set to 255, and the F 12 The formula is as follows: 4.

11. Model output reflects F 10 Whether it is empty or not, the binary number Label_rub, empty is 0, non-empty is 1, and the rubber ring damage mask F 12 ; S5 Bottle Cap Internal Stain Detection The stain detection model consists of a stain detection model for the inner area of ​​the rubber ring and a stain detection model on the rubber ring. Therefore, copy graph A, assign A to G and H, input the two sub-models respectively, and perform an OR operation on the outputs of the two sub-models to obtain the detection result of the stain inside the bottle cap. 5.

1. Stain detection model in the inner area of ​​the rubber ring (1) Assign G to G0, apply Gaussian filtering to G0, set the Gaussian kernel size to (13, 13), and set the parameters controlling the expansion degree in the horizontal and vertical directions to 2, and obtain G1; (2) Use Hough circle transform to detect the center of the circle in G1, and draw a circle with a radius equal to the radius of the inner area of ​​the bottle cap rubber ring based on the detected center of the circle. The pixel value of the inner area of ​​the circle is set to 255, and the pixel value of the outer area is set to 0. The mask G2 of the inner area of ​​the bottle cap rubber ring is obtained, as shown in formula (13); (3) Use Gaussian filtering on G, set the Gaussian kernel size to (39, 39), and set the parameters controlling the expansion degree in the horizontal and vertical directions to 0.25, and obtain G3; (4) Obtain the pixel values ​​of all pixels in the mask region G2 of the image G3, and record the set of pixels as G4; (5) Convert G4 into a matrix and use the Otsu method to obtain the threshold G5 for subsequent binarization of the matrix G4; (6) Perform an AND operation on G3 and G2 to obtain the grayscale image G6 under the mask area; (7) Binarization operation is performed on the mask area of ​​G6 according to the threshold value G5. For pixels whose pixel values ​​are less than the threshold value, their pixel values ​​are set to 255, otherwise they are set to 0, thereby obtaining a binary image G7. (8) Set the ellipse kernel size to (10, 10), perform an erosion operation on G7, and obtain G8; (9) Use cv2.connectedComponentsWithStats method to detect the connected areas of G8, and the set of connected areas is recorded as G9; (10) Filter out the G9 with an area less than or equal to the threshold G 10 Unicom area, set G according to experience 10 =2000, the filtered set is denoted as G 11 ; (11) Let G 12 represents an image of the same size as G and with all pixel values ​​0; and G 12 With G 11 The pixel values ​​of the pixels at the overlapping positions of the connected regions in the graph are set to 255, and the obtained value is G 13 , the formula is as follows: Among them, g 131 , g 132 Represents G 13 The horizontal and vertical coordinates of the middle pixel; (12) Model output reflects G 11 The binary number Label_sta_in indicating whether it is empty or not, 0 if empty and 1 if not empty, and the mask G reflecting the stain condition of the inner area of ​​the rubber ring 13 ; 5.

2. Stain detection model on rubber ring (1) Assign H to H0, apply Gaussian filtering to H0, set the Gaussian kernel size to (13, 13), and set the parameters controlling the expansion degree in the horizontal and vertical directions to 2, and obtain H1; (2) Use Hough circle transform to detect the center of the circle in H1, and based on the detected center of the circle, draw concentric circles consisting of a circle with a radius equal to the radius of the inner area of ​​the bottle cap rubber ring and a circle with a radius equal to the radius of the outer area of ​​the bottle cap rubber ring. The pixel value of the inner area of ​​the concentric circle is set to 255, and the pixel value of the outer area is set to 0, and the mask H2 of the bottle cap rubber ring area is obtained, as shown in formula (21); (3) Use Gaussian filtering on H, set the Gaussian kernel size to (13, 13), and set the parameters controlling the expansion degree in the horizontal and vertical directions to 1, and obtain H3; (4) Perform an AND operation on H3 and H2 to obtain the grayscale image H4 under the mask area; (5) Calculate the grayscale histogram of the mask area in H4, find the 3σ interval of the grayscale values ​​of the pixels in the mask area, and take the value at the left end of the interval as the threshold H5. The formula is as follows: Among them, a h4 and σ h4 Represent the mean and standard deviation of the pixel values ​​of H4 in the mask area H2. In order to prevent the left side of the 3σ interval from reaching 0, set δ h50 , according to experience, h50 =5; (6) For the pixels in the mask area inside H4, the pixel values ​​less than H5 will be set to 255, and the pixel values ​​greater than or equal to H5 will be set to 0, thus obtaining the binary image H6; (7) Use cv2.connectedComponentsWithStats method to detect the connected area of ​​H6, and the set of connected areas is recorded as H7; (8) Filter the connected areas in H7 whose area is less than or equal to the threshold H8, set H8 = 1500 based on experience, and obtain the filtered set H9; (9) Let H 10 represents an image of the same size as H, with all pixel values ​​​​being 0; and H 10 The pixel value of the pixel point overlapping with the connected area in H9 is set to 255, and H 11 , the formula is as follows: (10) The model outputs a binary number Label_sta_on that reflects whether H9 is empty, 0 for empty and 1 for non-empty, as well as a mask H that reflects the stain on the rubber ring. 11 ; 5.3, G 14 = G 13 + H 11 , while Label_sta = Label_sta_in or Label_sta_on; 5.

4. The model outputs a binary number Label_sta that reflects whether the bottle cap has stains. If there is no stain, it is 0, if there is stain, it is 1, and a mask G that reflects the stain status of the bottle cap. 14 ; S6 test results summary 6.

1. Let Mask = C 16 +F 12 +G 14 , where Mask represents the total defect area mask; 6.

2. Let Label = [Label_def, Label_scr, Label_rub, Label_sta] to get the total label; 6.

3. Let O'=Mask+O, where O represents the three-channel color original image and O' represents the original image after superimposing the total defect area mask.

Images