A method and system for detecting surface defects of container
Through the combination of neural network model and compound eye imaging technology, noise superposition and multi-data mixing, combined with the strategy of reducing error detection rate, the problem of low detection accuracy of complex glass bottles is solved, and high-precision and low error detection rate detection is achieved Effect.
Patent Information
- Application Number
- CN202410581527.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-05-11
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2044-05-11
AI Technical Summary
The prior art has the problem of low detection accuracy when detecting defects in complex glass bottles, especially the detection of microcracks with a size below 1 mm and cracks on complex undulating glass surfaces is difficult to accurately complete.
Using neural network model combined with compound eye imaging technology, data-enhanced training sets are constructed through noise superposition and multi-data mixing, and introduced a strategy for reducing false detection rates in the model, including difficult sample mining, increasing negative sample and label smoothing, using Focal loss loss function to improve detection accuracy.
High-precision detection of surface defects of complex glass bottles is achieved, with a resolution of sub-mm, a low error detection rate, and a system is easy to deploy, which can effectively identify micro-cracks in the bottle body, bottle bottom, bottle mouth and other parts.
Smart Images

Figure CN118261908B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image detection, and in particular to a method and system for detecting surface defects of a container. Background Art
[0002] Glass bottles are important packaging containers for alcoholic beverages, cosmetics, medicines and chemicals. The production process of glass bottles may cause defects such as black spots, shiny threads, intermittent teeth, stones, etc., which not only affect the appearance of the product, but may also cause potential safety hazards.
[0003] The detection of various defects on the surface of glass bottles is currently mainly achieved through artificial vision, machine vision, etc. However, the detection of various defects on complex glass bottles is still a difficult problem in the industry. For example, the detection of micro cracks with a size of less than 1mm is a difficult problem in the industry, and the detection of cracks on complex undulating glass surfaces is a difficult problem in the industry.
[0004] The existing technology has problems such as low detection accuracy. Summary of the invention
[0005] In order to overcome the deficiencies of the prior art, the present invention provides a method and system for detecting surface defects of a container, which solve the problems of low detection accuracy and the like in the prior art.
[0006] The technical solution adopted by the present invention to solve the above problems is:
[0007] A method for detecting surface defects of a container comprises the following steps:
[0008] S1, model construction: establish a neural network model for container surface defect detection;
[0009] S2, training set construction: construct the training set of the neural network model;
[0010] S3, model training: train the neural network model based on the training set, and introduce a false positive rate reduction strategy into the neural network model;
[0011] S4, surface defect detection: Use the trained neural network model to detect surface defects of the container.
[0012] As a preferred technical solution, in step S1, the neural network model includes a CRC module L1, an Att module, a first composite module L2, a second composite module L3, a first Concatenate function layer, a third composite module L4, a second Concatenate function layer, a fourth composite module L5, a fifth composite module L6, and an FC module connected in sequence. The first composite module L2 is connected to the first Concatenate function layer, the Att module is connected to the second Concatenate function layer, and the first composite module L2, the second composite module L3, the third composite module L4, the fourth composite module L5, and the fifth composite module L6 all include interconnected CRC modules and self-attention mechanism modules.
[0013] As a preferred technical solution, in step S2, the compound eye camera image acquired by relying on the compound eye imaging technology is used to form a data-enhanced training set through noise superposition and multi-data mixing; the compound eye camera is a camera array, including at least 2 cameras.
[0014] As a preferred technical solution, multi-data mixing refers to mixing multiple compound eye images from the same glass bottle and compound eye camera. The specific method is: two images are randomly selected from the training set, and after being processed by data augmentation operations respectively, the two images are superimposed in the form of average pixel values to synthesize a new sample.
[0015] As a preferred technical solution, in step S2, the noise superposition process is combined with the network forward noise adjustment strategy to add personalized noise to each attribute of each layer of each neural network model, and the attributes, noise levels, and different folds are traversed for each layer of the neural network model to select the best noise addition method for each attribute; the specific method is: first, for an attribute x, take multiple different noise levels, train the neural network model respectively and obtain the final loss function value, take the noise level with the smallest loss function value as the best injection noise level for attribute x, and finally determine the best noise injection method for different attributes.
[0016] As a preferred technical solution, in step S3, the false positive rate reduction strategy includes one or two of the following methods: adopting a training strategy and adopting a loss function.
[0017] As a preferred technical solution, in step S2, the training strategy includes difficult sample mining, adding negative samples, and label smoothing.
[0018] As a preferred technical solution, the loss function is:
[0019]
[0020] Among them, FL(p t) represents the loss function of the target pt, pt represents the target, α t represents a weight factor, γ represents a modulation factor, Represents the actual label of the sample.
[0021] As a preferred technical solution, in step S4, the image captured by the restoration camera is input into the neural network model, and the neural network model outputs the confidence of each feature point in the image. When the confidence is greater than a set threshold, it is judged as a defect.
[0022] A container surface defect detection system, used to implement the container surface defect detection method, comprises the following modules connected in sequence:
[0023] Model building module: used to build a neural network model for container surface defect detection;
[0024] Training set construction module: used to construct the training set of the neural network model;
[0025] Model training module: used to train the neural network model based on the training set and introduce a false positive rate reduction strategy into the neural network model;
[0026] Surface defect detection module: used to detect surface defects of containers using the trained neural network model.
[0027] Compared with the prior art, the present invention has the following beneficial effects:
[0028] The present invention can obtain completely clear imaging of complex glass bottle surface defects, with a resolution of defects reaching sub-millimeter level, and has the advantages of high accuracy, low false detection rate and easy deployment. BRIEF DESCRIPTION OF THE DRAWINGS
[0029] Figure 1 A schematic diagram of the steps of a container surface defect detection method according to the present invention;
[0030] Figure 2 The flowchart of the adaptive noise addition algorithm of the present invention is as follows;
[0031] Figure 3 It is a schematic diagram of the multi-data fusion strategy of the present invention;
[0032] Figure 4 It is the detection effect diagram of the present invention;
[0033] Figure 5 It is a structural diagram of the neural network model;
[0034] Figure 6 It is a schematic diagram of the axial structure of the glass bottle inspection equipment;
[0035] Figure 7 It is a top view structural diagram of the glass bottle inspection equipment.
[0036] Marks and their corresponding names in the attached drawings: 1- conveyor belt motor; 2- bottle guide plate; 3- conveyor pulley; 4- conveyor belt; 5- glass bottle circular imaging camera; 6- glass bottle circular imaging camera depth of field adjustment slide; 7- glass bottle circular imaging camera pitch adjustment mechanism; 8- glass bottle circular imaging camera array angle adjustment right slide; 9- glass bottle conveying production line; 10- glass bottle circular imaging camera array angle adjustment left slide; 11- glass bottle conveying production line column; 12- glass bottle; 13- glass bottle bottom imaging camera. DETAILED DESCRIPTION
[0037] The present invention will be further described in detail below in conjunction with embodiments and drawings, but the embodiments of the present invention are not limited thereto.
[0038] Example 1
[0039] like Figures 1 to 7 As shown, the present invention provides a method for detecting surface defects of complex glass bottles. The technical problem to be solved by the present invention is: high-precision detection of surface defects of complex glass bottles.
[0040] A neural network model is constructed and trained based on the data-enhanced training set, and a false positive rate reduction strategy is introduced into the network. After training, complex surface defects of glass bottles can be detected. The compound eye camera images collected by compound eye imaging technology are used to form a data-enhanced training set through noise superposition and multi-data mixing.
[0041] The present invention can obtain completely clear imaging of complex glass bottle surface defects, with a resolution of defects reaching sub-millimeter level, and has the advantages of high accuracy, low false detection rate and easy deployment.
[0042] A method for detecting surface defects of complex glass bottles, such as Figure 1 As shown, the implementation steps are as follows:
[0043] Step S1: Establish a neural network model for complex glass bottle surface defect detection (this embodiment uses a convolutional neural network model, such as Figure 5 As shown, other existing technologies may also be used to implement it);
[0044] Step S2: Construct a data enhancement training set. The compound eye camera images collected by the compound eye imaging technology are used to form a data enhancement training set through noise superposition and multi-data mixing;
[0045] Step S3: Constructing a neural network model for a false positive rate reduction strategy. Then, a neural network model is constructed and trained based on the data-enhanced training set, while introducing a false positive rate reduction strategy into the network.
[0046] Step S4: Complete the neural network model training and perform sample testing.
[0047] Preferably, the compound eye camera described in step S2 is a camera array, including at least two cameras for imaging the complex glass bottle surface at different angles.
[0048] Preferably, the noise superposition process described in step S2 is combined with a network forward noise adjustment strategy to add personalized noise to each layer of attributes of each neural network model. For each layer of the neural network model, attributes, noise levels, and different folds (folds refer to the folds in the ten-fold cross validation method) are traversed separately, and the best noise addition method is selected for each attribute (which can play a regularization role, control model complexity, and prevent overfitting). The noise is generally selected as Gaussian noise.
[0049] Preferably, the multi-data mixing described in step S2 refers to mixing multiple compound eye images of the same glass bottle and compound eye camera. The specific method is: we randomly extract two images from the training set, and after processing them with basic data augmentation operations (such as random cropping, horizontal flipping, etc.), we superimpose the two images in the form of averaging pixel values to synthesize a new sample.
[0050] Preferably, the false positive rate reduction strategy described in step S2 is subdivided into a training strategy and a loss function.
[0051] The training strategies include difficult sample mining, adding negative samples, and label smoothing.
[0052] The loss function is:
[0053]
[0054] Among them, FL(p t ) represents the loss function of the target pt, pt represents the target, α t represents a weight factor, γ represents a modulation factor, Represents the actual label of the sample. The core idea of the Focal loss function is to assign different weights to the loss of samples with different confidence levels. The lower the confidence level, the greater the weight, implicitly increasing the weight of difficult samples. In this way, a smaller number of categories can be effectively learned.
[0055] Generally speaking, in the defect detection scenario, in order to ensure the accuracy of defect detection, it is usually necessary to put a large number of defect samples and a small number of normal samples in the data set for comparison. However, when the false positive rate is required, we also need to focus on learning these small number of normal samples. The loss function is a loss function that can guide the model to learn under the condition of sample imbalance. The core idea of the loss function is to assign different weights to the loss of samples with different confidence levels. The lower the confidence level, the greater the weight, implicitly increasing the weight of difficult samples.
[0056] Difficult sample mining means that after training the model once, we focus on selecting samples with high confidence (greater than 80%) that are misjudged, and adjust our loss function (adjust α t ) to let the model focus on learning these difficult samples again, realizing an iterative training process. Adding negative samples is to prevent the network model from focusing too much on defect detection.
[0057] About loss function: In the field of deep learning, the softmax function is usually used to calculate the scores of different labels and use them as the corresponding probability values. However, due to the characteristics of the exponential function in the softmax function, there are often extreme phenomena such as exponential explosion, where the predicted value of a certain label is particularly large and the values of other labels are almost 0. However, in actual situations, sometimes some defects do look like reflections or other similar normal parts. If label smoothing is not used at this time, it is likely that the confidence of the reflection is extremely high, while the confidence of the defect is very low, resulting in misjudgment. At this time, we can use label smoothing to avoid these extreme situations.
[0058] In the present invention, the restoration camera achieves both high resolution and a large field of view, and can detect microcracks up to 0.5 mm;
[0059] In the present invention, a new neural network model realizes the distinction between cracks and normal structures on the complex undulating glass surface.
[0060] Example 2
[0061] like Figures 1 to 7 As shown, as a further optimization of Example 1, based on Example 1, this embodiment also includes the following technical features:
[0062] like Figure 6 to Figure 7As shown, the following glass bottle detection equipment can be preferably used (it can also be implemented by other existing equipment), the conveyor belt motor 1 drives the conveyor belt pulley 3, the conveyor belt pulley 3 drives the conveyor belt 4 to run, clamps the glass bottle 12 and moves forward along the running direction of the production line, and the bottle guide plate 2 stabilizes the glass bottle 12 to prevent the bottle from falling. During the conveying process, the bottom imaging camera 13 of the glass bottle images the bottom of the glass bottle 12 to obtain the bottom image of the glass bottle 12. When the glass bottle 12 is transported by the glass bottle conveying production line 9 to the imaging position of the glass bottle circumferential imaging camera 5, the glass bottle circumferential imaging camera 5 obtains the circumferential image of the glass bottle 12. The circumferential imaging camera 5 is arranged in the axial array of the glass bottle 12, and each imaging camera can adjust the pitch angle through the pitch adjustment mechanism 7 of the glass bottle circumferential imaging camera to obtain a complete axial image of the glass bottle 12. The cameras 5 of the axial array of the glass bottle are evenly distributed at 45 degrees along the right slide 8 and the left slide 10 of the glass bottle circumferential imaging camera array angle adjustment to obtain a complete circumferential image of the glass bottle 12. The depth of field adjustment slide 6 of the glass bottle circular imaging camera can adjust the depth of field of the axial array imaging camera of the glass bottle 12. The right slide 8 and the left slide 10 of the glass bottle circular imaging camera array angle adjustment can adjust the circumferential distribution angle of the glass bottle axial array imaging camera. By adjusting the depth of field and the circumferential distribution angle of the glass bottle circular imaging camera 5, a clear circumferential image of the glass bottle can be obtained and adapt to different glass bottle sizes and bottle shapes.
[0063] like Figure 1 As shown, a method for detecting surface defects of complex glass bottles of the present invention is specifically implemented as follows:
[0064] Step S1 establishes a neural network model for complex glass bottle surface defect detection. In this embodiment, the neural network model used is a convolutional neural network model.
[0065] Step S2 constructs a data enhancement training set. The compound eye camera images acquired by the compound eye imaging technology are used to form a data enhancement training set through noise superposition and multi-data mixing;
[0066] The noise superposition process combines the network forward noise adjustment strategy to add personalized noise to each layer of each neural network model attribute. For each layer of the neural network model, the attributes, noise levels, and different folds (folds refer to the folds in the ten-fold cross-validation method) are traversed separately to select the best noise addition method for each attribute. The noise is generally selected as Gaussian noise. Figure 2 In the pseudocode, P represents the properties of each neural network hidden layer, N represents different noise intensity levels, and M represents the number of tests. Figure 2Take as an example, we traverse the attributes, noise levels, and different folds (folds refer to the folds in the ten-fold cross validation method) respectively, and select the best noise addition method for the p attributes x1 to xp. First, for an attribute x, we take multiple different noise levels from 1 to N, train the model separately and get the final loss function value, and take the noise level with the smallest loss function value as the best injected noise level for this attribute. Then repeat the above operations for different attributes, and finally determine the best noise injection method for different attributes.
[0067] Multi-data mixing refers to mixing multiple compound eye images from the same glass bottle and compound eye camera. Figure 3 As shown in FIG. 1 , two images are randomly selected from the training set, and after being processed by basic data augmentation operations (such as random cropping, horizontal flipping, etc.), the two images are superimposed in the form of taking the average pixel value to synthesize a new sample.
[0068] Step S3: Constructing a neural network model for a false positive rate reduction strategy. Then, a neural network model is constructed and trained based on the data-enhanced training set, while introducing a false positive rate reduction strategy into the network.
[0069] The false positive rate reduction strategy is subdivided into training strategy and loss function.
[0070] The training strategies include difficult sample mining, adding negative samples, and label smoothing.
[0071] Difficult sample mining means that after training the model once, we focus on selecting misjudged samples with high confidence, and adjust our loss function to let the model focus on learning these difficult samples again, realizing an iterative training process. Adding negative samples is to prevent the network model from focusing too much on defect detection.
[0072] Step S4: Complete the neural network model training and conduct sample testing. Input the image captured by the restoration camera into the neural network model, and the neural network model determines whether each feature point in the image is a defect and outputs the confidence level. When the confidence level is greater than 80%, it is considered to be a defect.
[0073] In the actual application process, there are often many interference factors, such as lighting, imaging quality, and the complex and various non-defective parts of the glass bottle, which can easily interfere with the model's learning of defects. Adding noise can improve the stability of the model and have better generalization and robustness. However, many current noise-adding methods are still based on subjective experience to add artificially set noise, which can easily miss or misselect the optimal noise-adding method, resulting in poor results. Therefore, we consider using an adaptive method to allow the model to automatically select the most appropriate noise injection method to a certain extent. In this system, we use a forward noise adjustment strategy to add personalized noise to each layer of attributes of each neural network model. Figure 2 For example, we traverse the attributes, noise levels, and different folds (folds refer to the folds in the ten-fold cross-validation method) to select the best noise addition method for the p attributes x1 to xp. First, for an attribute x, we take multiple different noise levels from 1 to N, train the model separately and get the final loss function value, and take the noise level with the smallest loss function value as the best injection noise level for this attribute. Then repeat the above operations for different attributes, and finally determine the best noise injection method for different attributes.
[0074] In addition to adding noise, we also proposed a set of strategies to reduce the false detection rate, which are divided into two parts: training strategy and loss function. The training strategy can be divided into difficult sample mining, adding negative samples, and label smoothing. Among them, difficult sample mining means that after training the model once, we will focus on selecting some samples that we misjudged and have a high confidence level, and adjust our loss function (for example, give these samples a larger α t ) to let the model focus on learning these difficult samples again, realizing an iterative training process. Adding negative samples is to prevent the network model from focusing too much on defect detection. If the proportion of defect samples in the model is too high, the model will be more inclined to detect defects, resulting in a large number of misjudgments. However, if too many negative samples are added, the defect detection ability will decrease. Therefore, we can continuously adjust the ratio of positive and negative samples according to the goal to be achieved, and finally get a more balanced result.
[0075] Figure 4 Given the results of glass bottle defect detection, this method can filter out various interferences such as light spots, shadows, and complex fluctuations on the bottle bottom, and achieve 100% recognition of various small targets such as micro-cracks of about 1 mm on various parts of the bottle body, bottle bottom, bottle mouth, etc.
[0076] As described above, the present invention can be preferably implemented.
[0077] All features disclosed in all embodiments in this specification, or steps in all methods or processes implicitly disclosed, except for mutually exclusive features and / or steps, can be combined and / or expanded or replaced in any manner.
[0078] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. According to the technical essence of the present invention, within the spirit and principles of the present invention, any simple modification, equivalent replacement and improvement made to the above embodiment still falls within the protection scope of the technical solution of the present invention.
Claims
1. A method for detecting surface defects of a container, characterized in that: The following steps are involved: S1, model construction: establishing a neural network model for container surface defect detection; S2, training set construction: constructing the training set of the neural network model; S3, model training: train the neural network model based on the training set and introduce a false positive rate reduction strategy into the neural network model; S4, surface defect detection: Surface defect detection of containers using the trained neural network model; In step S2, the compound eye camera images acquired by the compound eye imaging technology are used to form a data-enhanced training set by noise superposition and multi-data mixing; the compound eye camera is a camera array including at least two cameras; Multi-data blending refers to blending multiple compound eye images from the same glass bottle and compound eye camera. The specific method is: two images are randomly extracted from the training set, processed by data augmentation operations, and then the two images are superimposed by taking the average pixel value to synthesize a new sample; In step S2, the noise superposition process is combined with the network forward noise adjustment strategy to add personalized noise to each layer attribute of each neural network model, and traverse the attributes, noise levels, and different folds for each layer of the neural network model respectively, and select the best noise addition method for each attribute; the specific method is: first, for an attribute x, take multiple different noise levels, train the neural network model separately and obtain the final loss function value, take the noise level with the smallest loss function value as the best injection noise level for attribute x, and finally determine the best noise injection method for different attributes.
2. A container surface defect detection method according to claim 1, characterized in that: In step S3, the false positive rate reduction strategy includes one or both of the following methods: adopting a training strategy and adopting a loss function.
3. A container surface defect detection method according to claim 2, characterized in that: In step S2, the training strategy includes difficult sample mining, adding negative samples, and label smoothing.
4. A container surface defect detection method according to claim 3, characterized in that: The loss function is: Among them, FL(p t ) represents the loss function of the target pt, pt represents the target, α t represents a weight factor, γ represents a modulation factor, represents the actual label of the sample.
5. A container surface defect detection method according to any one of claims 1 to 4, characterized in that: In step S4, the image captured by the restoration camera is input into the neural network model, and the neural network model outputs the confidence level of each feature point in the image. When the confidence level is greater than a set threshold, it is judged as a defect.
6. A container surface defect detection system, characterized in that: A method for detecting surface defects of a container according to any one of claims 1 to 5, comprising the following modules connected in sequence: Model building module: used to build a neural network model for container surface defect detection; Training set construction module: used to construct the training set of the neural network model; Model training module: used to train the neural network model based on the training set and introduce false positive rate reduction strategy into the neural network model; Surface defect detection module: used to detect surface defects of containers using the trained neural network model.
Citation Information
Patent Citations
Surface defect detection method based on augmentation training of deep neural network
CN111105391A
Surface defect detection system and method suitable for steel plate, equipment and medium
CN115880209A
Chip surface defect identification method based on information entropy convolutional neural network
CN116452919A
Concrete structure surface defect classification method and system based on hybrid neural network
CN117218434A
Hybrid integrated circuit assembly defect detection method based on semi-supervised learning
CN117670889A
