A method for detecting bottled product labeling quality based on machine vision

Abstract

The present invention discloses a method for detecting surface labeling quality defects of bottled products. The method comprises the following steps: irradiating the bottle top with a blue annular light source to obtain an image of the bottle body; utilizing multi-scale representation to detect and extract the target edge and obtain a ROI target image; calculating the grayscale mean and grayscale standard deviation of the ROI target image, and if the threshold condition is not met, determining that the label is missing; comparing the grayscale histogram distribution of the ROI target image and the standard image, and if there is obvious fluctuation, determining that the label is damaged; performing Hough line detection on the label edge of the ROI target image, and calculating the inclination angle of the label edge segment, and when the inclination angle is ≤2°, determining that the label is qualified; otherwise, the label is skewed. Labeling defect detection is performed using a machine vision method, and combined with a deep learning model for label edge extraction, the detection accuracy is higher and more robust, aiming to achieve high-precision real-time detection of label quality.

Description

Technical Field

[0001] The present invention relates to the technical field of labeling quality detection, in particular to a method for detecting quality defects of labeling of bottled products. Background Art

[0002] Product labeling involves attaching a label containing product information, such as the expiration date, production license number, and instructions, to the surface of the product. Consumers can use this information to select and use the product. Defects such as damaged labels can not only affect the consumer's purchasing experience but can also lead to improper use and safety issues.

[0003] With technological advancements, traditional manual labeling has been replaced by automated labeling machines. Automated labeling requires inspecting labeling defects on products after labeling. Manual inspection for labeling defects in automated labeling machines is significantly slower than automated ones, and inspection standards vary from individual to individual. Furthermore, long-term manual work can lead to visual fatigue, increasing missed and false positive rates. Therefore, the use of machine vision to inspect product labeling quality is essential. Summary of the Invention

[0004] The object of the present invention is to provide a method for detecting quality defects of labeling of bottled products.

[0005] In order to solve the above problems, a method for detecting labeling quality defects of bottled products is proposed, including:

[0006] Step (1), using a blue ring light source to illuminate the bottle top, adjusting the camera position and adjusting the lens focal length to enable the camera to clearly capture the image of the bottle body;

[0007] Step (2), using multi-scale representation to detect and extract the edge of the target in the captured image;

[0008] Step (3), cropping the label ROI target image according to the minimum circumscribed rectangle of the bottle target edge;

[0009] Step (4), calculating and comparing the grayscale mean, grayscale standard deviation, and grayscale histogram distribution of the ROI target image and the standard image;

[0010] Step (5), perform Hough line detection on the edge of the ROI target image and calculate the line inclination angle;

[0011] Step (6): When the target image satisfies the conditions that the grayscale mean and grayscale standard deviation are within the threshold range, the grayscale histogram distribution has no obvious fluctuation, and the label edge segment inclination angle of the ROI target image is ≤2°, the labeling quality of the bottled product is judged to be qualified.

[0012] Furthermore, the detailed steps of implementing the detection and extraction of the edge of the target in the captured image by using the multi-scale representation in step (2) include:

[0013] a. Use multi-scale representations of target image edges and learn multi-scale hierarchical features through deep learning models to achieve edge detection including detailed edges;

[0014] The image to be tested is used as the input of the VGG network. Convolutional feature maps of five different scales are obtained from layers 1 to 5. An output layer is inserted after the convolutional layer and deep supervision is applied to output the edge prediction results of the feature maps of each scale. The output results of the five layers are fused to achieve edge detection including detailed edges.

[0015] The fusion output Y of the edge prediction results is expressed as:

[0016]

[0017] Among them, h m Represents the output weight of each layer’s prediction value, A (m) Represents the output prediction value of the mth layer;

[0018] b. Extract the target contour based on the detected edge information. Perform two morphological erosion and dilation operations on the detected edge information binary image to achieve the segmentation of the target area:

[0019] The dilation operation is used to fill the gaps in the small edges of the label. In the dilation operation, for the pixel to be operated f(x, y), no matter which pixel value of f(x, y-1), f(x, y+1), f(x-1, y), or f(x+1, y) is 255, then f(x, y) = 255. The calculation formula is as follows:

[0020]

[0021] Where A is a set containing all pixels to be operated, B is a structural element, and expansion is understood as the center of B walking around the outer boundary of A;

[0022] The erosion operation is used to eliminate some small edge contours in the image. The erosion operation is used to eliminate the background part of the image. For the pixel f(x, y) to be operated, only f(x, y-1), f(x, y+1), f(x-1, y), and f(x+1, y) have a value of 255, then f(x, y) = 255. The calculation formula is as follows:

[0023]

[0024] The target label area is segmented by repeating the dilation and corrosion operation twice on the edge detected in step (2a).

[0025] Furthermore, in step (3), the detailed steps of cropping the label ROI target image according to the minimum bounding rectangle of the bottle target edge include:

[0026] a. Get the minimum bounding rectangle of the target outline and return the coordinates of the four vertices of the rectangle, which are (x1, y1), (x2, y2), (x3, y3), and (x4, y4), representing the lower left corner, upper left corner, upper right corner, and lower right corner, respectively.

[0027] b. Based on the coordinates of the four vertices of the minimum bounding rectangle, crop the horizontal rectangle containing the minimum bounding rectangle as the label ROI target image. The vertex coordinates of the horizontal rectangle are (m1, n1), (m2, n2), (m3, n3), and (m4, n4), which represent the lower left corner, upper left corner, upper right corner, and lower right corner respectively:

[0028] m1=m2=min(x1, x2, x3, x4),

[0029] n1=n4=min(y1, y2, y3, y4),

[0030] m3=m4=max(x1,x2,x3,x4),

[0031] n2=n3=max(y1, y2, y3, y4);

[0032] The internal area of ​​the horizontal rectangle is used as the label ROI target image, which contains all label edge information.

[0033] Furthermore, in step (4), the detailed steps of calculating and comparing the grayscale mean, grayscale standard deviation, and grayscale histogram distribution of the ROI target image and the standard image include:

[0034] a. Use the median filter algorithm to achieve noise reduction on the label ROI target image and the standard image;

[0035] Let the one-dimensional sequence be f1, f2…f n , assuming that the median filter is a window containing m pixels, m is an odd number. During the processing, the grayscale value of the pixel in the middle of the window is replaced by the median of the grayscale values ​​of all pixels in the window. The median filter for this one-dimensional sequence is to extract m numbers from the input sequence in succession, arrange the values ​​of these m points in order of size, and take the middle number of the sequence as the filter output. Let v represent the center value of the median filter window, then the median filter y of the i-th pixel in the image is i It can be expressed as:

[0036] y i =Med{f i-v ,...,f i,...,f i+v},

[0037] v = (m-1) / 2;

[0038] b. Randomly select d standard images (d ≥ 10 to avoid the randomness of threshold data results), calculate the grayscale mean and grayscale standard deviation of these images, and use them to determine the threshold range for judging missing labels. Observe the grayscale histogram distribution and determine the grayscale frequency threshold for judging label damage;

[0039] c. Calculate the grayscale mean and grayscale standard deviation of the target image of the label ROI, draw the grayscale distribution histogram, and compare it with the standard threshold;

[0040] d. When the grayscale mean and grayscale standard deviation of the ROI target image do not meet the standard threshold, the labeling quality is judged to be unqualified and the defect type is "missing label". When the grayscale distribution histogram of the ROI target image fluctuates greatly and exceeds the standard grayscale frequency threshold, the labeling quality is judged to be unqualified and the defect type is "label damage".

[0041] Furthermore, in step (5), the Hough line detection is performed on the edge of the ROI target image, and the detailed steps of calculating the line inclination angle include:

[0042] a. Perform Hough line detection on the edge of the ROI target image;

[0043] b. Based on the Hough line detection, obtain the coordinates of the longest line segment endpoint of the label edge of the ROI target image and calculate the inclination angle of the label edge segment;

[0044] c. When the tilt angle of the label edge segment in the ROI target image is ≤2°, the bottled product labeling quality is judged to be qualified; otherwise, it is unqualified and the defect type is "label skew".

[0045] Beneficial effects of the present invention:

[0046] The present invention uses machine vision methods to detect label defects and combines them with deep learning models to extract label edges. The detection accuracy is higher and more robust, aiming to achieve high-precision real-time detection of label quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to clearly illustrate the specific embodiments of the present invention, the drawings required for use in the specific embodiments are described as follows:

[0048] Figure 1 This is a flow chart of the real-time labeling quality detection of the present invention;

[0049] Figure 2 This is a sample image of qualified labeling quality of an embodiment of the present invention;

[0050] Figure 3 This is a sample picture of unqualified labeling quality in an embodiment of the present invention. DETAILED DESCRIPTION

[0051] The technical solutions in the embodiments of the present invention are described clearly and completely below with reference to the accompanying drawings. The embodiments described in the present invention are only some of the embodiments of the present invention, not all of them. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort are also within the scope of protection of the present invention.

[0052] Example

[0053] This embodiment provides a method for detecting quality defects in bottled product labels, including:

[0054] Step (1) using a blue ring light source to illuminate the bottle top, adjusting the camera position and adjusting the lens focal length to enable the camera to clearly capture the image of the bottle body;

[0055] Step (2) using multi-scale representation to detect and extract the edge of the target in the captured image;

[0056] Step (3) cropping the label ROI target image according to the minimum circumscribed rectangle of the largest edge of the bottle body;

[0057] Step (4) calculating and comparing the grayscale mean, grayscale standard deviation, and grayscale histogram distribution of the ROI target image and the standard image;

[0058] Step (5) performs Hough line detection on the edge of the ROI target image and calculates the tilt line angle;

[0059] Step (6) When the target image satisfies the conditions that the grayscale mean and grayscale standard deviation are within the threshold range, the grayscale histogram distribution has no obvious fluctuation, and the label edge segment inclination angle of the ROI target image is ≤2°, the labeling quality of the bottled product is judged to be qualified.

[0060] In this embodiment, a deep learning model is used to extract label edges, which has better extraction effect. An improved traditional label defect detection algorithm is used to classify label quality defects, which has higher detection accuracy and is more robust, aiming to achieve high-precision real-time detection of label quality.

[0061] Figure 1 The specific detection process of the embodiment is shown.

[0062] Specifically, in this embodiment, the step of using multi-scale representation to detect and extract the edge of the target in the captured image in step (2) includes:

[0063] a. The image to be tested is used as the input of the VGG network. Convolutional feature maps of five different scales are obtained from layers 1 to 5. An output layer is inserted after the convolutional layer. Deep supervision is applied to output the edge prediction results of the feature maps at each scale. The output results of the five layers are fused to achieve edge detection including detailed edges.

[0064] The fusion output Y of the edge prediction results can be expressed as:

[0065]

[0066] Among them, h m Represents the output weight of each layer’s prediction value, which can be obtained through network training. (m) Represents the output prediction value of the mth layer. After summing the output prediction values ​​of the m layers, the σ function is used to normalize them to obtain the edge result after multi-scale prediction fusion.

[0067] b. Perform two morphological corrosion and expansion operations on the detected edge information binary image to achieve the segmentation of the target area:

[0068] The dilation operation is used to fill the gaps in the small edges of the label. In the dilation operation, for the pixel to be operated f(x, y), no matter which pixel value of f(x, y-1), f(x, y+1), f(x-1, y), or f(x+1, y) is 255, then f(x, y) = 255. The calculation formula is as follows:

[0069]

[0070] Where B is the structural element, and the expansion can be understood as the center of B moving around the outer boundary of A;

[0071] The erosion operation is used to eliminate some small edge contours in the image. Using the erosion operation to eliminate the background part of the image can also produce a filter effect. For the pixel f(x, y) to be operated, only f(x, y-1), f(x, y+1), f(x-1, y), and f(x+1, y) have a pixel value of 255, then f(x, y) = 255. The calculation formula is as follows:

[0072]

[0073] Where A is a set of all pixels to be operated (in this patent, it refers to the label edge image), B is the structural element, and erosion can be understood as the center of B walking around the inner boundary of A;

[0074] The target label area is segmented by repeating the dilation and corrosion operation twice on the edge detected in step (2a).

[0075] In step (3), the step of cropping the label ROI target image according to the minimum circumscribed rectangle of the target edge of the bottle body includes:

[0076] a. Get the minimum bounding rectangle of the segmented target outline and return the coordinates of the four vertices of the rectangle, which are (x1, y1), (x2, y2), (x3, y3), and (x4, y4), representing the lower left corner, upper left corner, upper right corner, and lower right corner, respectively;

[0077] b. Based on the coordinates of the four vertices of the minimum bounding rectangle, crop the horizontal rectangle containing the minimum bounding rectangle as the label ROI target image. The vertex coordinates of the horizontal rectangle are (m1, n1), (m2, n2), (m3, n3), and (m4, n4), which represent the lower left corner, upper left corner, upper right corner, and lower right corner respectively:

[0078] m1=m2=min(x1, x2, x3, x4),

[0079] n1=n4=min(y1, y2, y3, y4),

[0080] m3=m4=max(x1,x2,x3,x4),

[0081] n2=n3=max(y1, y2, y3, y4);

[0082] The internal area of ​​the horizontal rectangle is used as the label ROI target image, which contains all label edge information.

[0083] In step (4), the step of calculating and comparing the grayscale mean, grayscale standard deviation, and grayscale histogram distribution of the ROI target image and the standard image includes:

[0084] a. Use the median filter algorithm to reduce the noise of the label ROI target image and the standard image to make the image smoother;

[0085] Let the one-dimensional sequence be f1, f2...f n , assuming that the median filter is a window containing m pixels, m is an odd number. During the processing, the grayscale value of the pixel in the middle of the window is replaced by the median of the grayscale values ​​of all pixels in the window. The median filter for this one-dimensional sequence is to extract m numbers from the input sequence in succession, arrange the values ​​of these m points in order of size, and take the middle number of the sequence as the filter output. Let v represent the center value of the median filter window, then the median filter y of the i-th pixel in the image is i It can be expressed as:

[0086] y i =Med{f i-v ,...,f i ,...fi+v},

[0087] v = (m-1) / 2;

[0088] b. Randomly select d standard images (d ≥ 10 to avoid the randomness of threshold data results). Calculate the grayscale mean and grayscale standard deviation of these images after processing such as target area extraction and median filtering. Use this to determine the threshold range for judging missing labels. Observe the grayscale histogram distribution and determine the grayscale frequency threshold for judging label damage.

[0089] c. Calculate the grayscale mean and grayscale standard deviation of the target image of the label ROI, draw the grayscale distribution histogram, and compare it with the standard threshold;

[0090] Taking the label ROI target image as an example, assuming that the size of the target image is w×h and the grayscale value of each pixel is s i ∈[0, 255], calculate the grayscale mean Grayscale standard deviation σ, and draw a grayscale histogram, the horizontal axis of the grayscale histogram is the grayscale level, and the vertical axis is the frequency p of the grayscale level:

[0091]

[0092]

[0093] p(r k )=n k / w×h;

[0094] where r k is the gray level of the pixel, n k is a grayscale r k The number of pixels;

[0095] d. When the grayscale mean and grayscale standard deviation of the ROI target image do not meet the standard threshold, the labeling quality is judged to be unqualified and the defect type is "missing label". When the grayscale distribution histogram of the ROI target image fluctuates greatly and exceeds the standard grayscale frequency threshold, the labeling quality is judged to be unqualified and the defect type is "label damage".

[0096] In step (5), the steps of performing Hough line detection on the edge of the ROI target image and calculating the angle of the inclined line include:

[0097] a. Preset the minimum length threshold of the edge segment of the ROI target image and the maximum distance between parallel segments and perform Hough line detection on it;

[0098] b. Based on the Hough line detection, obtain the coordinates of the longest line segment endpoint of the label edge of the ROI target image, and calculate the average tilt angle of the label edge segment based on the line segment endpoints;

[0099] Assuming that the endpoint coordinates of the line segment are (x1, y1) and (x2, y2), the inclination angle of the line segment can be expressed as:

[0100]

[0101] c. When the tilt angle of the label edge segment in the ROI target image is ≤2°, the bottled product labeling quality is judged to be qualified; otherwise, it is unqualified and the defect type is "label skew".

Claims

1. A method for inspecting the quality of bottled product labeling based on machine vision, characterized in that The steps include: (1) Use a blue ring light source to illuminate the bottle top, adjust the camera position and adjust the lens focal length to achieve a clear image of the bottle body; (2) Using multi-scale representation to detect and extract the edge of the target in the captured image; (3) Crop the label ROI target image according to the minimum circumscribed rectangle of the bottle target edge; (4) Calculate and compare the grayscale mean, grayscale standard deviation, and grayscale histogram distribution of the ROI target image and the standard image; (5) Perform Hough line detection on the edge of the ROI target image and calculate the inclination angle of the edge segment; The specific steps of step (5) include: 5a. Perform Hough line detection on the edge of the ROI target image; 5b. According to Hough line detection, obtain the coordinates of the longest line segment endpoint of the label edge of the ROI target image and calculate the inclination angle of the label edge segment; 5c. When the tilt angle of the label edge line segment in the ROI target image is ≤2°, the bottled product labeling quality is judged to be qualified; otherwise, it is unqualified and the defect type is "label skew"; (6) When the target image satisfies the grayscale mean and grayscale standard deviation within the threshold range, the grayscale histogram distribution has no obvious fluctuation, and the label edge segment inclination angle of the ROI target image is ≤2°, the labeling quality of the bottled product is judged to be qualified.

2. The method for detecting the quality of bottled product labeling based on machine vision according to claim 1, characterized in that The specific steps of step (3) include: 3a. Get the minimum bounding rectangle of the target outline and return the coordinates of the four vertices of the rectangle, which are (x1, y1), (x2, y2), (x3, y3), and (x4, y4), representing the lower left corner, upper left corner, upper right corner, and lower right corner, respectively. 3b. Based on the coordinates of the four vertices of the minimum bounding rectangle, crop the horizontal rectangle containing the minimum bounding rectangle as the label ROI target image. The vertex coordinates of the horizontal rectangle are (m1,n1), (m2,n2), (m3,n3), and (m4,n4), which represent the lower left corner, upper left corner, upper right corner, and lower right corner respectively: m1=m2=min(x1,x2,x3,x4), n1=n4=min(y1,y2,y3,y4), m3=m4=max(x1,x2,x3,x4), n2=n3=max(y1,y2,y3,y4); The internal area of ​​the horizontal rectangle is used as the label ROI target image, which contains all label edge information.

3. The method for detecting the quality of bottled product labeling based on machine vision according to claim 2, characterized in that The specific steps of step (4) include: 4a. Use the median filter algorithm to reduce the noise of the label ROI target image and the standard image; Let the one-dimensional sequence be f1,f2…f n , assuming that the median filter is a window containing m pixels, m is an odd number. During the processing, the grayscale value of the pixel in the middle of the window is replaced by the median of the grayscale values ​​of each pixel in the window; the median filter of the one-dimensional sequence is to extract m numbers from the input sequence in succession, arrange the values ​​of these m points in order of size, and take the middle number of the sequence as the filter output. Let v represent the center value of the median filter window, then the median filter y of the i-th pixel in the image is i Expressed as: y i =Med{f i-v ,…,f i ,…,f i+v }, 4b. Randomly select d standard images (d ≥ 10), calculate the grayscale mean and grayscale standard deviation of these images, and use them to determine the threshold range for judging missing labels. Observe the grayscale histogram distribution and determine the grayscale frequency threshold for judging label damage. 4c. Calculate the grayscale mean and grayscale standard deviation of the target image of the label ROI, draw a grayscale distribution histogram, and compare it with the standard threshold; 4d. When the grayscale mean and grayscale standard deviation of the ROI target image do not meet the standard threshold, the labeling quality is judged to be unqualified and the defect type is "missing label". When the grayscale distribution histogram of the ROI target image fluctuates greatly and exceeds the standard grayscale frequency threshold, the labeling quality is judged to be unqualified and the defect type is "label damage".

Images