A quality inspection method for glass containers based on AI deep learning technology

Abstract

This invention relates to the fields of image recognition and deep learning technology, and provides a method for quality inspection of glass containers based on AI deep learning technology. The method includes: Step 1, inputting image data of glass containers with defect labels as a training set into a deep learning model for training; Step 2, preprocessing and normalizing the training set; Step 3, extracting features from the training set; Step 4, generating a loss function using the features and the training set to bring the deep learning model to converge; Step 5, inputting image data of the glass container to be inspected into the trained deep learning model; Step 6, using the deep learning model to predict defects in the image data of the glass container to be inspected, generating bounding boxes and defect types; Step 7, performing secondary feature extraction on the defects corresponding to the bounding boxes to further determine the defect type. This invention, by training on complex background regions of glass containers, can solve defects that traditional algorithms cannot detect.

Description

Technical Field

[0001] This invention relates to the fields of image recognition and deep learning technology, and in particular to a method for quality inspection of glass containers based on AI deep learning technology. Background Technology

[0002] With the continuous development of glass container manufacturing technology, my country's daily-use glass industry has become increasingly sophisticated, narrowing the gap with international advanced technologies. However, as manufacturing processes continue to evolve, customers are now placing increasingly higher demands on the testing of glass containers.

[0003] In the glass container industry, the main existing inspection methods are based on various traditional machine vision processing techniques, including blob analysis, morphological processing, and feature matching. Patents such as "CN201720728586.2 A Visual Inspection System for Neck Crack Defects in Glass Container Manufacturing," "CN201510427763.9 A Visual Inspection System and Method for Manufacturing Defects in Transparent Glass Containers," and "CN202020271276.4 A Rotating Device for Online Visual Inspection of Pharmaceutical Glass Bottles" all employ traditional machine vision processing techniques to address the problems requiring inspection.

[0004] As the shapes and patterns of glass containers become more diverse and complex, more and more defects appear around or even within the patterns. The inability to detect defects around the patterns is a major pain point and challenge for traditional algorithms. For humans, object detection is a very simple task. However, computers can only "see" the encoded numbers of an image. Traditional algorithms struggle to understand high-level semantic concepts such as product defects as defined by humans, making it even more difficult to pinpoint the location of the target within the image. This ultimately leads to low detection efficiency and low detection accuracy. Summary of the Invention

[0005] This invention provides a method for quality inspection of glass containers based on AI deep learning technology, which includes the following steps:

[0006] Step 1: Input the image data of glass containers with defect labels as the training set into the deep learning model for training;

[0007] Step 2: Preprocess and normalize the training set;

[0008] Step 3: Extract features from the training set;

[0009] Step 4: Generate a loss function using features and the training set to bring the deep learning model to converge.

[0010] Step 5: Input the image data of the glass container to be detected into the trained deep learning model;

[0011] Step 6: Use a deep learning model to predict defects in the image data of the glass container to be inspected, and generate regression boxes and defect types;

[0012] Step 7: Perform secondary feature extraction on the defects corresponding to the regression boxes to further determine the defect type.

[0013] Furthermore, in step 1, the defect label includes common defects at the bottle mouth: bright scissor marks, bright stones, dark scissor marks, and dark stones; and common defects at the bottle bottom: stones, air bubbles, stains, and cracks.

[0014] Furthermore, in step 7, the detection method further includes the following steps:

[0015] Step 71: Filter the regression boxes corresponding to multiple defects based on confidence level to generate true defects;

[0016] Step 72: Segment the regression boxes corresponding to the multiple defects predicted by the deep learning model.

[0017] Step 73: Perform dynamic clustering of defects based on regression boxes;

[0018] Step 74: Perform secondary feature extraction on the pattern area at the bottom of the bottle using a defect filter to determine the defect type and precise location.

[0019] Furthermore, step 73 also includes the following steps:

[0020] Step 731, calculate the feature vector of each pixel within the regression bounding box:

[0021] f(x,y)=[I(x,y),G sigma I(x,y),G sigma I 2 (x,y)]

[0022] Where I(x,y) represents the gray value of pixel (x,y) in the image, and G sigma I and G sigma I 2 Representing images I and I respectively 2 The Gaussian blur result, where sigma represents the standard deviation of the Gaussian kernel;

[0023] Step 732: Cluster the feature vectors of all pixels within the regression bounding box to obtain the set of cluster centers:

[0024] C = {c1, c2, ..., c} k}

[0025] Among them, ck Let k be the number of cluster centers;

[0026] Step 733: For all pixels (x, y) within the regression bounding box, calculate the similarity between that pixel and the cluster center.

[0027] S k (x,y)=f(x,y)·c k

[0028] Where f(x,y)·c k The feature vector f(x,y) and the cluster center c represent the relationship between the feature vector f(x,y) and the cluster center c. k The dot product;

[0029] Step 734, calculate c for each cluster center k Threshold:

[0030] T k =max{f(x,y)·c k}

[0031] Step 735, and set the threshold of pixel (x,y) to the threshold of the cluster center closest to it;

[0032] Step 736: Extract valid defects from the regression box using a threshold.

[0033] Furthermore, step 74 also includes the following steps:

[0034] Step 741, Construct the feature set:

[0035] F={area,width,height,anisometry,…m}

[0036] area represents the area of ​​the cluster region, width represents the length of the cluster region, height represents the width of the cluster region, anisometry represents the ratio of the major semi-axis to the minor semi-axis, and m represents the number of items in the constructed feature set.

[0037] Step 742: Construct the feature weight vector:

[0038] L = {a, b, c, d, ..., m′}

[0039] Where a, b, c, d, ..., m′ represent the weights of the corresponding constructed features in the constructed feature set F, respectively;

[0040] Step 743: Convolution operation of feature set and feature weights outputs sample feature values.

[0041]

[0042] Where i∈(1,m) represents the i-th item in the constructed feature set.

[0043] Step 744: Compare the calculated sample feature values ​​with preset feature conditions. If the preset feature conditions are met, the sample is judged to have a defect; otherwise, it is not considered a defect.

[0044] The beneficial effects achieved by this invention are:

[0045] This invention addresses the shortcomings of traditional algorithms in detecting complex background regions of glass containers by enhancing training on these regions.

[0046] This invention enhances the network's ability to extract features from image texture information by supervising the training of shallow texture information in an AI network model, constructs a detection network, improves the image's ability to resist interference from complex backgrounds, and demonstrates strong model robustness. Attached Figure Description

[0047] Figure 1 This is a flowchart of a glass container quality inspection method based on AI deep learning technology.

[0048] Figure 2 This is a simplified schematic diagram of the image neural network structure in a glass container quality inspection method based on AI deep learning technology.

[0049] Figure 3 This is a schematic diagram of a deep learning convolutional layer in a glass container quality inspection method based on AI deep learning technology.

[0050] Figure 4 This is a schematic diagram of the feature fusion operation in a glass container quality inspection method based on AI deep learning technology.

[0051] Figure 5 This is a schematic diagram of the dimensional expansion module in a glass container quality inspection method based on AI deep learning technology.

[0052] Figure 6 This is a schematic diagram of crack detection in the patterned area at the bottom of a glass container using a quality inspection method based on AI deep learning technology.

[0053] Figure 7 This is a schematic diagram illustrating the detection of air bubbles in a complex background on the bottom anti-slip block of a glass container using an AI deep learning-based quality inspection method.

[0054] Figure 8 This is a schematic diagram of chipping in the inner ring area of ​​the bottle mouth in a glass container quality inspection method based on AI deep learning technology.

[0055] Figure 9 This is a schematic diagram of bottle bottom positioning and mold reading in a glass container quality inspection method based on AI deep learning technology.

[0056] Figure 10 This is a schematic diagram of the user interface for detecting defects in glass containers using an AI deep learning-based quality inspection method.

[0057] Figure 11 This is a schematic diagram of the user interface for detecting defects in glass containers using an AI deep learning-based quality inspection method.

[0058] Figure 12 This is a schematic diagram of the operation interface for manual re-inspection after the glass container quality inspection method based on AI deep learning technology detects container defects. Detailed Implementation

[0059] The technical solution of the present invention will be described in more detail below with reference to the accompanying drawings. The present invention includes, but is not limited to, the following embodiments.

[0060] As attached Figure 1 As shown, this invention provides a method for quality inspection of glass containers based on AI deep learning technology, comprising the following steps:

[0061] Step 1: Input the image data of glass containers with defect labels as the training set into the deep learning model for training;

[0062] Image input is as shown below. Figure 2 Before training a neural network, data needs to be collected and cleaned to determine the category set. It's also important to ensure that the image content is clear and complete, and that the amount of data for each category is roughly the same.

[0063] Common defects at the bottle mouth include bright scissor marks, bright stones, dark scissor marks, and dark stones; common defects at the bottle bottom include stones, air bubbles, stains, and cracks. Custom detection can also be performed for other possible defects.

[0064] Specifically, common defects at the bottle mouth include pitting, double mouth, broken mouth, insufficient mouth, air bubbles on the mouth surface, and blocked neck; common defects on the bottle body include air bubbles, stones, oil stains, vertical axis deviation, internal thread overlap, dead material, quenching marks, airplane wings, and thick mold seam lines; common defects at the bottle bottom include oil stains on the bottle bottom, misaligned printing at the top, and exposed bottom.

[0065] Step 2: Preprocess and normalize the training set;

[0066] Data preprocessing is the image input stage of a neural network. Its main function is to preprocess and normalize images using specific methods, making image features more apparent and accelerating the convergence of the neural network model. Common data preprocessing methods include image filtering and image edge detection.

[0067] Step 3: Extract features from the training set;

[0068] Convolutional neural network models are commonly used for feature extraction, as shown in the attached image. Figure 4 .

[0069] The input layer is the input to the entire neural network. Through the connection and action of each neuron, it extracts the feature information of the input image layer by layer, and analyzes and abstracts the obtained feature information to obtain higher-dimensional and more abstract image features.

[0070] As attached Figure 5 As shown, in the process of feature extraction in a neural network, the features are expanded in dimension, and then the expanded dimensional features are concatted to ensure that high-dimensional information of the recognized features can be obtained.

[0071] Step 4: Generate a loss function using features and the training set to bring the deep learning model to converge.

[0072] The loss function is a computational function used to represent the degree of difference between the model's prediction results and the actual data. The smaller the loss function, the better the robustness of the model.

[0073] During model training, the loss function continuously evaluates the model's progress, reflecting its current performance on the training set. After obtaining the loss value, the model updates its parameters through backpropagation to reduce the discrepancy between the true and predicted values, thus bringing the model's predictions closer to the true values ​​and achieving the learning objective.

[0074] Step 5: Input the image data of the glass container to be detected into the trained deep learning model;

[0075] The images of the glass containers to be detected are used as input to the test set to train a converged deep learning model.

[0076] Step 6: Use a deep learning model to predict defects in the image data of the glass container to be inspected, and generate regression boxes and defect types;

[0077] The defect prediction task requires a computer to identify the category of a target image within a specific set of categories. During the learning process, data augmentation techniques, including translation, rotation, and cropping, are used when collecting and labeling defects. Mosaic augmentation is also employed during training to further enhance the data. Therefore, the algorithm does not need to concern itself with the location of defects within the image; it only needs to determine whether the image contains the defect type labeled in the specific set.

[0078] For example Figure 3 As shown, the input image is a 3*608*608 image.

[0079] After receiving image data, the algorithm model will provide probability values ​​for the four categories corresponding to the current input image. These four categories correspond to the classification categories labeled during training. For example, the bottom of a bottle is typically labeled as having stones, cracks, bubbles, and stains. If the input image is a bottle bottom with stains, the probability values ​​P obtained for predicting these four types of defects might be:

[0080] P = {0.1, 0.1, 0.8, 0.9}

[0081] Therefore, in practical defect detection, image classification algorithms are often used when the defect target is large and the individual defect features are relatively obvious. (See attached image.) Figure 7 , 8 Figure 9 shows a typical detection task in practical applications.

[0082] Step 7: Perform secondary feature extraction on the defects corresponding to the regression boxes to further determine the defect type;

[0083] Following AI defect prediction and bounding box prediction, this method adds a defect screening module layer. With this layer, the entire process runs very smoothly in the field application of glass containers. This method fully utilizes the bounding boxes predicted by deep learning for dual extraction of small sample features. Specific steps are as follows: Figure 6 .

[0084] Step 71: Segmentation of the target detection bounding box.

[0085] {CheckBox1...CheckBox n}

[0086] Filtering of regression boxes and detection boxes.

[0087] Suppose there are n bounding boxes, with corresponding confidence levels c_1, c_2, ..., c_n, and the required confidence level for a detection box is c_d. This can be expressed by the following formula:

[0088] max(c1,c2,...,cn)>c_{d}

[0089] Where max(c_{1},c_{2},...,c_{n}) represents the maximum value among n confidence scores. If the maximum value is greater than the confidence requirement c_{d} for the detection box, it means that the confidence scores of multiple regression boxes are greater than the set confidence score. If the confidence score is greater than the set confidence score, it indicates that the target in the regression box is a defect. Filtering multiple regression boxes means that the content of the regression boxes will be used as input for the next layer of recognition.

[0090] Step 72: Dual feature extraction based on regression boxes.

[0091] This method first crops the original image using the regression bounding box to obtain a new sub-image, also known as the ReduceImage. Then, a second round of feature extraction is performed on the ReduceImage to obtain richer and more refined feature information. This method can fully utilize the feature information of the target region, thereby improving the model's performance and robustness.

[0092] Let the coordinates of the bounding box in the image be (x1, y1) and (x2, y2), where (x1, y1) represents the coordinates of the top-left corner of the bounding box, and (x2, y2) represents the coordinates of the bottom-right corner. Then, the following formula can be used to extract the portion corresponding to the bounding box from the image:

[0093] Imagecropped=Image(y1:y2,x1:x2)

[0094] Where Image represents the original image, and Imagecropped represents the portion of the image cropped from the bounding box. The colon ":" indicates a slicing operation, y1:y2 means selecting the portion of the image with vertical coordinates from y1 to y2, and x1:x2 means selecting the portion of the image with horizontal coordinates from x1 to x2.

[0095] Step 73: Dynamic Clustering Small Sample Segmentation

[0096] Step 731, calculate the feature vector of each pixel within the regression bounding box:

[0097] f(x,y)=[I(x,y),G sigma I(x,y),G sigma I 2 (x,y)]

[0098] Where I(x,y) represents the gray value of pixel (x,y) in the image, and G sigma I and G sigma I 2 Representing images I and I respectively 2 The Gaussian blur result, where sigma represents the standard deviation of the Gaussian kernel.

[0099] Step 732: Cluster the feature vectors of all pixels within the regression bounding box to obtain the set of cluster centers:

[0100] C = {c1, c2, ..., c} k}

[0101] Where k is the number of cluster centers.

[0102] Step 733: For all pixels (x, y) within the bounding box, calculate the similarity between that pixel and the cluster center:

[0103] S k (x,y)=f(x,y)·c k

[0104] Step 734: Calculate c for each cluster center k Threshold:

[0105] T k =max{f(x,y)·c k}

[0106] f(x,y)·c k The feature vector f(x,y) and the cluster center c represent the relationship between the feature vector f(x,y) and the cluster center c. k The dot product.

[0107] Step 735: Set the threshold for pixel (x,y) to the threshold of the cluster center closest to it.

[0108] T(x,y)=T{hat{k}}

[0109] Where hat{k}=arg(max{S k (x,y)}).

[0110] Step 736: Extract valid defects from the regression box using a threshold.

[0111] By using a threshold, the background portion of the regression bounding box image is removed, the foreground is preserved, and the pattern of the effective defect is extracted.

[0112] In this method, the number of cluster centers C is predetermined. In practical applications, cross-validation is used to determine the number of cluster centers to obtain the optimal segmentation results. This method fully utilizes the local statistical information of pixels in the image, thereby improving the accuracy and robustness of segmentation. Furthermore, since this method uses only a small number of samples for training, it can effectively handle the problem of small sample defects in glass containers.

[0113] Step 74: Perform secondary feature extraction on the patterned area at the bottom of the bottle using a defect filter to determine the defect type and exact location;

[0114] Step 741, Construct the feature set:

[0115] F={area,width,height,anisometry,…m}

[0116] area represents the area of ​​the cluster region, width represents the length of the cluster region, height represents the width of the cluster region, anisometry represents the ratio of the major semi-axis to the minor semi-axis, and m represents the number of items in the constructed feature set.

[0117] Step 742, Construct the feature weight vector:

[0118] L = {a, b, c, d, ..., m′}

[0119] Where a, b, c, d, ..., m′ represent the weights of the corresponding constructed features in the constructed feature set F, and these weights are determined by machine learning and iteration through the features of effective defects in the regression box.

[0120] Step 743: The convolution operation between the feature set and the feature weights outputs the sample feature values.

[0121]

[0122] Where i∈(1,m) represents the i-th item in the constructed feature set.

[0123] Step 744: Compare the calculated sample feature values ​​with preset feature conditions. If the preset feature conditions are met, the sample is judged to have a defect; otherwise, it is not considered a defect.

[0124] Through the above steps, the current application of glass containers not only meets the needs of complex on-site environments and working conditions, but also satisfies customers' requirements for rapid production changeover and precise scrap removal, greatly improving customers' production efficiency and cost control.

[0125] Currently, using existing deep learning neural networks to detect defects in the patterned areas on the bottom of glass containers results in a defect miss rate of over 90%. However, the method proposed in this application can reduce the miss rate of defects in the patterned areas to 10%. Moreover, this data was obtained without increasing the misinterpretation rate.

[0126] Specifically, statistics were compiled based on the accuracy and testing results of this invention at the usage site, primarily focusing on testing results over an 11-day period. The statistical results and accuracy analysis are shown in the table below:

[0127]

[0128]

[0129] The table clearly shows that the reliability of this invention is high. The number of tests per day varies, ranging from several thousand to tens of thousands, and the detected defects also change constantly. However, the daily detection accuracy can be controlled at over 99%, and the number of false positives can be controlled to single digits, with an overall accuracy of approximately one in ten thousand.

[0130] The table below shows the statistics for key categories, including the number and percentage of defects in four key types: stones, stains, cracks, and bubbles.

[0131] Days Defects detected stone stain crack bubble Day 1 410 29 31 30 104 Day 2 737 45 59 46 198 Day 3 1045 54 83 436 269 Day 4 1687 72 121 751 425 Day 5 2381 94 176 1137 539 Day 6 2523 100 183 1211 569 Day 7 227 16 21 108 49 Day 8 1068 34 98 611 152 Day 9 683 13 51 391 100 Day 10 487 13 27 227 121 Day 11 351 19 9 118 117

[0132] As can be seen, cracks and bubbles account for a relatively high proportion, which is consistent with the manufacturing process of molded bottles. Defects that occur more frequently are more easily detected by neural networks. Additionally (refer to the pie chart), Figure 10 , Figure 11 Small blemishes, closed bubbles, and other less common defects can be correctly detected with the required accuracy. All data is statistically analyzed manually, using methods such as... Figure 12 The interface is manually verified.

[0133] This invention is not limited to the specific embodiments described above. Those skilled in the art can implement this invention using various other specific embodiments based on the disclosed content of the embodiments and accompanying drawings. Therefore, any design that adopts the design structure and concept of this invention and makes some simple changes or modifications falls within the protection scope of this invention.

Claims

1. A method for quality inspection of glass containers based on AI deep learning technology, characterized in that, The glass container quality inspection method based on AI deep learning technology includes the following steps: Step 1: Input the image data of glass containers with defect labels as the training set into the deep learning model for training; Step 2: Preprocess and normalize the training set; Step 3: Extract features from the training set; Step 4: Generate a loss function using features and the training set to bring the deep learning model to converge. Step 5: Input the image data of the glass container to be detected into the trained deep learning model; Step 6: Use a deep learning model to predict defects in the image data of the glass container to be inspected, and generate regression boxes and defect types; Step 7: Perform secondary feature extraction on the defects corresponding to the regression boxes to further determine the defect type; In step 7, the detection method further includes the following steps: Step 71: Filter the regression boxes corresponding to multiple defects based on confidence level to generate true defects; Step 72: Segment the regression boxes corresponding to the multiple defects predicted by the deep learning model. Step 73: Perform dynamic clustering of defects based on regression boxes; Step 74: Perform secondary feature extraction on the pattern area at the bottom of the bottle using a defect filter to determine the defect type and precise location.

2. The glass container quality inspection method based on AI deep learning technology according to claim 1, characterized in that, In step 1, the defect label includes common defects at the bottle mouth: bright scissor marks, bright stones, dark scissor marks, and dark stones; and common defects at the bottle bottom: stones, air bubbles, stains, and cracks.

3. The glass container quality inspection method based on AI deep learning technology according to claim 1, characterized in that, Step 73 also includes the following steps: Step 731, calculate the feature vector of each pixel within the regression bounding box: f(x,y)=[I(x,y),G sigma I(x,y),G sigma I 2 (x,y)] Where I(x,y) represents the gray value of pixel (x,y) in the image, and G sigma I and G sigma I 2 Representing images I and I respectively 2 The Gaussian blur result, where sigma represents the standard deviation of the Gaussian kernel; Step 732: Cluster the feature vectors of all pixels within the regression bounding box to obtain the set of cluster centers: C={c1,c2,…,c k } Among them, c k Let k be the number of cluster centers; Step 733: For all pixels (x, y) within the regression bounding box, calculate the similarity between that pixel and the cluster center. S k (x,y)=f(x,y)·c k Where f(x,y)·c k The feature vector f(x,y) and the cluster center c represent the relationship between the feature vector f(x,y) and the cluster center c. k The dot product; Step 734, calculate c for each cluster center k Threshold: T k =max{f(x,y)·c k } Step 735, and set the threshold of pixel (x,y) to the threshold of the cluster center closest to it; Step 736: Extract valid defects from the regression box using a threshold.

4. The glass container quality inspection method based on AI deep learning technology according to claim 1, characterized in that, Step 74 also includes the following steps: Step 741, Construct the feature set: F={area,width,height,anisometry,…m} area represents the area of ​​the cluster region, width represents the length of the cluster region, height represents the width of the cluster region, anisometry represents the ratio of the major semi-axis to the minor semi-axis, and m represents the number of items in the constructed feature set. Step 742: Construct the feature weight vector: L = {a, b, c, d, ..., m′} Where a, b, c, d, ..., m′ represent the weights of the corresponding constructed features in the constructed feature set F, respectively; Step 743: Convolution operation of feature set and feature weights outputs sample feature values. Where i∈(1,m) represents the i-th item in the constructed feature set; Step 744: Compare the calculated sample feature values ​​with preset feature conditions. If the preset feature conditions are met, the sample is judged to have a defect; otherwise, it is not considered a defect.

Images