Product defect detection method based on automatic training and deployment of standardized flow model

Abstract

The present invention discloses a product defect detection method based on automatic training and deployment of a standardized flow model. The training process of the standardized flow model adopts an early stopping strategy based on the statistical variance of the training data: when the mean square error of the output features of the standardized flow model is not greater than a preset threshold, it is judged that the detection model is about to crash, and the training of the detection model is stopped; and the detection model with the closest time point before the training is stopped is used as the final result. The present invention uses a lightweight standardized flow model for unsupervised defect detection, which occupies low computing resources, and can automatically complete the entire process from training to deployment at the edge device end, without the need for manual labeling, and without the need for algorithm personnel to participate in parameter adjustment.

Description

Technical Field

[0001] The present invention relates to the field of product defect detection, and in particular to a product defect detection method based on automatic training and deployment of a standardized flow model. Background Art

[0002] Product defect detection is a typical problem in industrial production quality management. With the development of artificial intelligence, edge computing products are widely used in product defect detection. The current edge defect detection algorithm is mainly trained and optimized by offline algorithm personnel or supervised algorithms or unsupervised methods, and then deployed to the online edge computing device. This non-automated method is difficult to adapt to products with short update cycles.

[0003] For example, the technical solution with publication number CN115496892A and patent application name "Industrial Defect Detection Method, Device and Electronic Equipment, Storage" uses a two-dimensional flow model for unsupervised defects, and describes in detail the training and testing process of the flow model and related formulas.

[0004] The disadvantages of this technical solution are:

[0005] (1) The training efficiency and stability of the stream model are related to the loss function and the distribution of training data. This solution does not solve the training crash problem of the stream model, which affects whether the stream model can achieve automatic training.

[0006] (2) This solution does not solve the problem of hyperparameter selection when deploying the streaming model, which affects whether the streaming model can be automatically deployed.

[0007] For another example, the technical solution with the publication number CN115423741A and the patent application name "A structural defect detection method based on deep learning" combines the advantages of the flow model based on the unsupervised training mode and the detection model based on the supervised training mode, and uses two models with different training paradigms to perform defect detection together to achieve a complementary effect. This solution uses DifferNet as the flow model. During the test process, DifferNet needs to input multiple transformed images and perform back propagation gradients to locate abnormal areas.

[0008] The disadvantages of this technical solution are:

[0009] (1) This solution does not solve the training crash problem of streaming models, which affects whether streaming models can be trained automatically.

[0010] (2) This solution does not solve the problem of hyperparameter selection when deploying the streaming model, which affects whether the streaming model can be automatically deployed.

[0011] (3) The segmentation results of DifferNet in this solution are relatively rough and require a lot of computation, so it cannot be deployed on many edge devices.

[0012] For another example, the technical solution with the publication number CN115496749A and the patent application name "Product defect detection method and system based on target detection training preprocessing" performs defect detection through the target detection model. In view of the problem that it is difficult to collect a large number of defective images in industrial scenarios, this solution enhances the diversity of samples by manually annotating them first and then amplifying the data preprocessing solution of defective samples.

[0013] The disadvantages of this technical solution are:

[0014] (1) This solution uses a supervised defect detection model, which requires a large number of defect samples. For some newly launched products or products with a short cycle, it is unrealistic to collect a large number of defect samples.

[0015] (2) This solution requires manual labeling, which results in high labeling costs.

[0016] For another example, the technical solution with the publication number CN115423807A and the patent application name "A cloth defect detection method based on outlier detection" uses LOF for cloth defect detection. LOF finds outliers in data based on density, which belongs to the unsupervised paradigm and does not require data labeling.

[0017] The disadvantages of this technical solution are:

[0018] (1) This solution requires specifying the neighborhood size, LOF threshold, distance measurement method, etc. according to a specific data set and cannot be automatically deployed.

[0019] (2) As the training data set expands, the computational efficiency of the algorithm will be greatly reduced.

[0020] Summarizing the above solutions, the shortcomings of the prior art are summarized as follows:

[0021] (1) The training efficiency and stability of the streaming model are related to the loss function and the distribution of the training data. The unsupervised method based on the streaming model does not solve the training collapse problem of the streaming model and the hyperparameter selection problem during model deployment. These problems affect whether the streaming model can achieve automatic training and deployment.

[0022] (2) Supervised object detection models require a large number of defect samples and manual labeling, and cannot be automatically trained and deployed.

[0023] (3) Some flow models such as DifferNet have large computational complexity and are difficult to deploy on most existing edge devices because they need to calculate reverse gradients during testing.

[0024] (4) Traditional unsupervised outlier detection methods require manual selection of hyperparameters, and their computational efficiency is affected by the training data, making them difficult to automatically train and deploy on edge devices.

[0025] Summarizing the above solutions, the shortcomings of the prior art are summarized as follows:

[0026] (1) The training efficiency and stability of the streaming model are related to the loss function and the distribution of the training data. The unsupervised method based on the streaming model does not solve the training collapse problem of the streaming model and the hyperparameter selection problem during model deployment. These problems affect whether the streaming model can achieve automatic training and deployment.

[0027] (2) Supervised object detection models require a large number of defect samples and manual labeling, and cannot be automatically trained and deployed.

[0028] (3) Some flow models such as DifferNet have large computational complexity and are difficult to deploy on most existing edge devices because they need to calculate reverse gradients during testing.

[0029] (4) Traditional unsupervised outlier detection methods require manual selection of hyperparameters, and their computational efficiency is affected by the training data, making them difficult to automatically train and deploy on edge devices. Summary of the invention

[0030] The purpose of the present invention is to overcome the shortcomings and deficiencies of the prior art and to provide a product defect detection method based on automatic training and deployment of a standardized flow model. The method uses a lightweight standardized flow model for unsupervised defect detection, occupies low computing resources, and can automatically complete the entire process from training to deployment on the edge device side without the need for manual labeling or algorithm personnel to participate in parameter adjustment.

[0031] The purpose of the present invention is achieved through the following technical solutions:

[0032] A product defect detection method based on automatic training and deployment of a standardized flow model, wherein the training process of the standardized flow model adopts an early stopping strategy based on the statistical variance of training data: when the standardized flow model outputs a feature When the mean square error is not greater than the preset threshold, it is determined that the detection model is about to crash and the training of the detection model is stopped; and the detection model with the time point closest to the time before the training is stopped is taken as the final result.

[0033] The overall training data of the standardized flow model is divided into The mean square error is calculated for each batch, and the mean square error of the output features of the overall training data after passing through the standardized flow model is calculated. The process is as follows:

[0034] ;

[0035] ;

[0036] ;

[0037] in, Before The overall variance of the batch data, Indicates the current The local variance of the batch, Indicates the current The local mean of the batches, Before The overall mean of the batch data; and Respectively represent and batches of images.

[0038] The product defect detection method of automatically training and deploying the standardized flow model outputs features through the standardized flow model The probability density value is used to determine whether a product defect occurs. If it is greater than the preset threshold, it is considered that a product defect occurs.

[0039] Output features through the standardized flow model Calculate defect probability map for all pixel locations The calculation method is as follows:

[0040] ;

[0041] in, Indicates that each downsampled feature map is upsampled to the original image The final defect probability value is the average of the results of the three scales. , , Represents output feature maps of different sizes;

[0042] Get the defect probability map After that, post-processing is required to obtain the segmentation results and detection results: by setting the threshold To segment the defect probability map, we get a segmentation map. In the segmentation map, the probability of The area with defective area is the defective area, otherwise it is the normal area; the minimum envelope of each segmentation image is used as the envelope result of defect detection; if there is a defective area, it is determined to be a defective product.

[0043] The set threshold To adaptively set the threshold, it is calculated as follows:

[0044] ;

[0045] Among them, setting the threshold is a defect probability value, indicating the dividing line between normal features and defect features. Features for training data The mean point of Features The mean square error, is the feature point used to calculate the segmentation threshold; and equivalence, The function is used to calculate the probability density of *.

[0046] The set threshold It is used to process the edge area of ​​the product image and use a high threshold in the non-edge area of ​​the product ( +0.1) for processing.

[0047] The standardized flow model adopts fastflow.

[0048] The product defect detection method for automatic training and deployment of the standardized flow model adopts fastflow as the standardized flow model, adopts the standard normal distribution as the posterior estimation of fastflow, and uses the ResNet18 network pre-trained on the ImageNet classification data set as the feature extractor.

[0049] At the same time, the present invention provides:

[0050] A server includes a processor and a memory, wherein the memory stores at least one program, and the program is loaded and executed by the processor to implement the above-mentioned product defect detection method based on automatic training and deployment of a standardized flow model.

[0051] A computer-readable storage medium stores at least one program, which is loaded and executed by a processor to implement the above-mentioned product defect detection method based on automatic training and deployment of a standardized flow model.

[0052] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0053] 1. Automated training. The present invention proposes an early stopping strategy based on the statistical variance of training data. The early stopping strategy based on the statistical results of training data can effectively prevent the feature collapse phenomenon in the model training process. No test data and test labels are required, and no algorithm personnel are required to participate in the training process of tuning the model. The automated training process of the algorithm can be quickly realized to adapt to the continuous updating of the product.

[0054] 2. Automated deployment. The present invention proposes a method for adaptively setting thresholds based on statistical results of training data and a dual-threshold post-processing solution. Algorithm personnel do not need to participate in the parameter adjustment process in model deployment, and all hyperparameters of the algorithm can be automatically calculated.

[0055] 3. Suitable for automated defect detection at the edge. The present invention can realize automated algorithm updates at the edge device end by selecting a lightweight model based on an unsupervised approach for defect detection and solving algorithm problems in the process of automatic training and automatic deployment. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 The flowchart of the product defect detection method based on automatic training and deployment of standardized flow models.

[0057] Figure 2 This is the workflow diagram of the FastFlow model.

[0058] Figure 3 Schematic diagram of adaptive threshold. DETAILED DESCRIPTION

[0059] The present invention is further described in detail below in conjunction with embodiments and drawings, but the embodiments of the present invention are not limited thereto.

[0060] like Figure 1 As shown in the figure, the product defect detection method based on automatic training and deployment of the standardized flow model mainly includes the description of four parts: product image preprocessing, early stopping strategy in model automatic training, calculation of segmentation threshold in model automatic deployment and post-processing of model output results.

[0061] The specific technical solution implementation process is as follows:

[0062] 1. Image enhancement pre-processing

[0063] like Figure 2 As shown, the present invention is a method for product defect detection using fastflow as a standardized flow model. In order to estimate the distribution of normal samples, the present invention uses the standard normal distribution As the posterior estimation of fastflow, the ResNet18 network pre-trained on the ImageNet classification dataset is used as the feature extractor. Specifically, the size of the product image I input into ResNet18 is 288*320, and the last layer features of the first three network blocks are output as the feature input of the subsequent flow model. These three layers of features They have different downsampling rates (4, 8, 16) and are mapped to probability distributions through 8 radiation transformation modules to obtain features of the same scale. The present invention uses the standardized flow model to output features The probability density value of is used to determine whether there are product defects, thereby realizing automatic detection of defects.

[0064] Compared with traditional defect detection methods, the present invention adopts the characteristics of being lightweight and convenient for edge training and deployment. This makes the present invention very suitable for automated training and deployment on edge devices. In addition, due to the high similarity between the collected defect-free product samples, the present invention also adopts some data enhancement methods to improve the diversity of training data. These methods include random rotation, cropping, changing brightness and contrast, etc.

[0065] This paper proposes a lightweight defect detection method based on FastFlow and ResNet18, which is fast, accurate and reliable. It can help manufacturers realize automated defect detection in the product production process, thereby improving product quality and production efficiency.

[0066] 2. Statistics-based early stopping of training

[0067] In edge computing environments, automating the training process is an important task, but problems that arise during training may cause the model to crash. These problems are often related to factors such as loss function, data distribution, and training strategy. In many cases, model training is affected by the loss function because the optimization objective of the loss function often does not conform to the actual data distribution.

[0068] The present invention has found through research that there is a problem in the automated training process of FastFlow, that is, after a certain period of training, the converted features tends to 0, which means The posterior distribution (Gaussian distribution) is not satisfied, that is, the model training collapses. The inventor creatively discovered that the main reason for the model collapse is the maximum likelihood loss function. The essential requirement of the algorithm is to learn distribution, but the maximum likelihood loss can only ensure that the input features The probability value is the largest under the probability distribution learned by the flow module, but this does not mean that the learned probability distribution is the same as The distribution of is consistent. In order to solve the problem of model collapse, the solution of the present invention is to adopt an early stopping strategy. For a data set with test data and test labels, you can continuously test on the test data during the training process and take the model result with the best test result. However, for edge automatic training, there is no test data, let alone labeled test data.

[0069] The early stopping strategy is a training strategy that can decide when to stop training based on some indicators during the training process. These indicators may include the rate of change of training loss, the rate of change of validation loss, the size of the gradient, and so on. When the test data and test labels are known, the test data can be used to evaluate the performance of the model and select the best model. However, in the process of automated training of edge devices, test data and test labels are usually not available, which makes it more difficult to select the best model.

[0070] To solve this problem, the present invention proposes an early stopping strategy based on the statistical variance of training data. The core idea of ​​this strategy is to use the feature The mean square error of the feature is used to determine whether the model is about to crash. When the mean square error of the feature drops below a certain threshold, the model training is stopped and the final model is used as the result. This method can avoid the problem of model crash and does not require the support of test data and test labels. Therefore, this method is particularly suitable for automated training scenarios on edge devices without the involvement of algorithm personnel.

[0071] In the actual training process, there are many training data, which cannot be loaded into the memory at one time to calculate the mean square error, especially on the edge device side, where computing resources are insufficient. The mean square error is calculated for batches, where Batches contain Frame image, calculate the mean square error of the overall training data The process is as follows:

[0072] ;

[0073] ;

[0074] ;

[0075] in, Before The overall variance of the batch data, Indicates the current The local variance of the batch, Indicates the current The local mean of the batches, Before The overall mean of the batch data; and Respectively represent and batches of images.

[0076] 3. Statistics-based adaptive threshold

[0077] After training, fastflow obtains the features The present invention calculates the defect probability map of each pixel position according to the following formula: :

[0078] ;

[0079] in, Indicates that each downsampled feature map is upsampled to the original image The final defect probability value is the average of the results of the three scales. , , Represents output feature maps of different sizes;

[0080] Get the defect probability map After that, post-processing is required to obtain the segmentation results and detection results: by setting the threshold To segment the defect probability map, we get a segmentation map. In the segmentation map, the probability of The area with defective area is the defective area, otherwise it is the normal area; the minimum envelope of each segmentation image is used as the envelope result of defect detection; if there is a defective area, it is determined to be a defective product.

[0081] After obtaining the defect probability map, post-processing is required to obtain the segmentation results and detection results. The common way is to set a threshold, but the setting of the threshold is related to the specific data set. If the threshold is too low, it will lead to false detection, and if the threshold is too high, it will lead to missed detection. Therefore, a suitable threshold has a great impact on the final result of the algorithm. In the process of automated deployment at the edge, there is no participation of algorithm personnel, which requires the algorithm to automatically select a threshold suitable for the data set.

[0082] To address this problem, the present invention proposes a method for adaptively setting thresholds based on the statistical results of training data. Specifically, the method is based on the trained feature , these features follow the characteristics of the standard normal distribution in the posterior distribution (such as Figure 3 The characteristic of the standard normal distribution is that most of the features are aggregated near the mean of 0 and are regarded as normal samples, while only a few features deviate from the mean and are regarded as defects.

[0083] Therefore, the present invention proposes a formula to adaptively set the threshold value, and the specific formula is as follows:

[0084] ;

[0085] Among them, setting the threshold is a defect probability value, indicating the dividing line between normal features and defect features. Features for training data The mean point of Features The mean square error, is the feature point used to calculate the segmentation threshold; and equivalence, The function is used to calculate the probability density of *.

[0086] 4. Double threshold post-processing

[0087] According to the actual test results, it is found that when two defects are close to each other, the product part between the defects will have a higher probability value, which will lead to a larger segmentation area. In order to solve this problem, the present invention proposes a new method, which uses a double threshold method for post-processing to obtain a more refined segmentation result.

[0088] Specifically, this method first converts the original RGB image into a grayscale image and uses OpenCV operators to obtain edge images. When processing edges, a low threshold is used. , while in the non-edge part, a high threshold is used ( +0.1) to obtain more accurate results. This is mainly based on the fact that the location of defects often presents irregular shapes, which easily produce dense edge lines. The product surface is relatively regular, the edge lines are relatively sparse, and the defect probability value at the center of the defect is higher than the threshold. The defect probability value of the product between two adjacent defects is often near the threshold. Using a high threshold will effectively suppress these product areas near the threshold. This dual threshold method can effectively deal with the mutual influence between defects and avoid misjudgment caused by adjacent defects.

[0089] In addition to the segmentation map, we also need to get the envelope position of the defect. We use the minimum envelope of each segmentation map as the envelope result of defect detection.

[0090] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. A product defect detection method based on automatic training and deployment of a standardized flow model, characterized in that: The training process of the standardized flow model adopts an early stopping strategy based on the statistical variance of the training data: when the standardized flow model outputs the feature When the mean square error is not greater than the preset threshold, it is determined that the detection model is about to crash and the training of the detection model is stopped; and the detection model with the closest time point to the time before the training is stopped is taken as the final result; The overall training data of the standardized flow model is divided into The mean square error is calculated for each batch, and the mean square error of the output features of the overall training data after passing through the standardized flow model is calculated. The process is as follows: ; ; ; in, Before The overall variance of the batch data, Indicates the current The local variance of the batch, Indicates the current The local mean of the batches, Before The overall mean of the batch data; and Respectively represent and batches of images; Output features through the standardized flow model Calculate defect probability map for all pixel locations The calculation method is as follows: ; in, Indicates that each downsampled feature map is upsampled to the original image The final defect probability value is the average of the results of the three scales. , , Represents output feature maps of different sizes; Get the defect probability map After that, post-processing is required to obtain the segmentation results and detection results: by setting the threshold To segment the defect probability map, we get a segmentation map. In the segmentation map, the probability of The area with defective area is the defective area, otherwise it is the normal area; the minimum envelope of each segmentation image is used as the envelope result of defect detection; if there is a defective area, it is determined to be a defective product.

2. The product defect detection method based on automatic training and deployment of a standardized flow model according to claim 1, characterized in that: The product defect detection method of automatically training and deploying the standardized flow model outputs features through the standardized flow model The probability density value is used to determine whether a product defect occurs. If it is greater than the preset threshold, it is considered that a product defect occurs.

3. The product defect detection method based on automatic training and deployment of a standardized flow model according to claim 1, characterized in that: The set threshold To adaptively set the threshold, it is calculated as follows: ; Among them, setting the threshold is a defect probability value, indicating the dividing line between normal features and defect features. Features for training data The mean point of Features The mean square error, is the feature point used to calculate the segmentation threshold; and equivalence, Function is calculated The probability density of .

4. The product defect detection method based on automatic training and deployment of a standardized flow model according to claim 1, characterized in that: The set threshold Used to process the edge area of ​​the product image and use thresholds in the non-edge area of ​​the product to be processed.

5. The product defect detection method based on automatic training and deployment of a standardized flow model according to claim 1, characterized in that: The standardized flow model adopts fastflow.

6. The product defect detection method based on automatic training and deployment of a standardized flow model according to claim 1, characterized in that: Its product defect detection model uses fastflow as the standardized flow model, standard normal distribution as the posterior estimate of fastflow, and uses the ResNet18 network pre-trained on the ImageNet classification dataset as the feature extractor.

7. A server, comprising a processor and a memory, characterized in that: At least one program is stored in the memory, and the program is loaded and executed by the processor to implement the product defect detection method based on automatic training and deployment of a standardized flow model as described in any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that: The storage medium stores at least one program, which is loaded and executed by a processor to implement the product defect detection method based on automatic training and deployment of a standardized flow model as described in any one of claims 1 to 6.

Images