A few-shot industrial anomaly detection method based on dual-guided contrast

Abstract

This invention discloses a few-shot industrial anomaly detection method based on dual-guided contrast, relating to the fields of computer vision and industrial defect detection. The method first constructs a few-shot reference set, extracts image features using a pre-trained visual encoder, and then performs dual-guided branch fusion after normal-anomaly guided branching. During the training phase, a global contrast aggregation module and pixel-level contrast loss are introduced, and during the testing phase, the anomaly detection results are output. Compared with existing technologies, this invention can fully utilize the guiding information from a small number of normal and anomaly samples, improving anomaly detection accuracy and cross-class generalization ability, and exhibiting faster inference speed.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and industrial defect detection, specifically to a method for detecting industrial anomalies with few samples based on dual-guided comparison. Background Technology

[0002] Surface defect detection in industrial products is a crucial aspect of quality control in intelligent manufacturing. Traditional manual visual inspection suffers from low efficiency, high subjectivity, and a high rate of missed detections, making it unable to meet the quality requirements of modern large-scale industrial production. Computer vision-based automated anomaly detection technology, with its advantages of non-contact operation, high precision, and high efficiency, has become the mainstream industrial quality inspection method.

[0003] With the development of deep learning, significant progress has been made in anomaly detection methods based on convolutional neural networks and visual Transformers. In industrial scenarios, anomaly samples are scarce and diverse, making it difficult to obtain sufficient labeled data. Therefore, few-shot anomaly detection techniques have attracted widespread attention. This technique requires only a small number of samples to complete model training, making it more suitable for practical industrial applications. However, existing few-shot anomaly detection methods suffer from problems such as difficulty in utilizing defective samples, insufficient ability to identify complex anomalies, blurred segmentation boundaries, and high computational costs. Summary of the Invention

[0004] This invention proposes a few-sample industrial anomaly detection method based on dual-guided comparison, aiming to fully utilize the guiding information from a small number of normal and defective samples to achieve efficient and accurate industrial anomaly detection. First, suitable normal and anomaly samples are selected as reference images. The query image and the few-sample reference images are input into a pre-trained visual encoder, then processed through a normal-anomaly guided branch, followed by dual-guided branch fusion, and finally the anomaly detection result is output. The method is implemented through the following technical solutions:

[0005] Step 1: Construct a small sample reference set, randomly select defect-free samples from normal samples, randomly select defective samples from abnormal samples, and extract the defect mask corresponding to the sample.

[0006] Step 2: Use a pre-trained visual encoder to extract features from the query image and the images in the few-sample reference set, respectively.

[0007] Step 3: In the normal boot branch, calculate the normal boot score.

[0008] Step 3.1: Flatten the query features and normal reference features respectively, and perform L2 normalization to calculate the cosine similarity. The specific expression is as follows:

[0009] ,

[0010] ,

[0011] ,

[0012] in, These are the query features after being flattened. This is the normal reference feature after flattening. This represents the normalized query characteristics. This represents the normalized reference feature.

[0013] Step 3.2: For the query image, the first... Local features The value most similar to all normal reference features is calculated as the normal guided score. :

[0014] ,

[0015] in, yes The higher the score of the normal guidance, the more similar the query region is to the normal reference set region, and the greater the possibility that the image is normal.

[0016] Step 4: In the abnormal boot branch, calculate the abnormal boot score.

[0017] Step 4.1: Downsample the defect mask of the abnormal reference image to match the features. Figure One At the same spatial scale, after binarization and flattening, the mean of the feature vectors in the anomaly regions is calculated respectively. The mean of the feature vectors in the normal region Define the residual prototype :

[0018] ,

[0019] in, Characteristic offset of defective regions relative to normal regions.

[0020] Step 4.2: After L2 normalization, the residual prototype is input into a two-layer perceptron mapping network to obtain the projection vector. The query features, after L2 normalization, are input into the same perceptron to obtain the projected query features. .

[0021] Step 4.3: Calculate the similarity between the projection vector and the projection query. :

[0022] ,

[0023] in, This is a learnable temperature parameter.

[0024] Step 4.4: Similarity is used to obtain anomaly guidance score through Sigmoid activation function. :

[0025] ,

[0026] in, The larger the value, the more consistent the query region is with the residual prototype, and the greater the possibility of image anomalies.

[0027] Step 5: Merge the scores of the two guiding branches.

[0028] Step 5.1: Concatenate the normal guidance score and the abnormal guidance score along the channel dimension to transform into a two-dimensional feature map. Upsample to the original query image size and obtain the abnormal segmentation probability map by applying a normalized exponential function. :

[0029] ,

[0030] in, Indicates the current position of the first Logical values ​​for each category, It represents the sum of logical values ​​for all categories at the same location.

[0031] Step 5.2: Calculate the final anomaly score :

[0032] .

[0033] Step 6: Introduce a global comparison aggregation module during the training phase.

[0034] Step 6.1: The final anomaly score is weighted and averaged based on the query features. A normalized exponential function with a temperature parameter is input, and the global features are calculated. The specific expression is as follows:

[0035] ,

[0036] in, For the query image of the first Local features, It is the query feature weight of the current position.

[0037] Step 6.2: Global features as Matrix, projection query features as Matrix and The matrix is ​​augmented using multi-head cross-attention, and the output is the augmented global feature.

[0038] Step 6.3: The enhanced global features are passed sequentially through a fully connected layer, an activation layer, and another fully connected layer to obtain the image-level binary classification result. The anomaly class is input into the Sigmoid activation function to obtain the anomaly probability. Calculate the global comparison aggregation loss :

[0039] ,

[0040] in, It refers to the batch size. For the first Image tags, Indicates the first The probability of anomalies in a single image.

[0041] Step 7: Introduce pixel-level contrast loss during the training phase. Use image labels to distinguish between defective and background regions in the query image, calculate the mean similarity between defective and background regions, and then calculate the pixel-level contrast loss. Total loss The specific expression is as follows:

[0042] ,

[0043] ,

[0044] in, To query the mean similarity of defect regions in an image, It queries the mean similarity of normal regions in an image. For image-level classification loss, It is a pixel-level segmentation loss. , These represent the pixel-level contrast loss coefficient and the global contrast aggregation loss coefficient, respectively.

[0045] Step 8: During the testing phase, only steps 1 to 5 need to be run to upsample the final anomaly score to the original image size to obtain the final anomaly segmentation result. The mean of the high scores in the final anomaly score is then binarized to obtain the final anomaly classification result.

[0046] This invention employs a dual-guided comparison mechanism of normal and abnormal samples, which can achieve high-precision anomaly detection with only a very small number of samples, significantly reducing the data annotation cost in industrial scenarios. It also has strong cross-class generalization ability and inference speed that meets industrial needs. Attached Figure Description

[0047] Figure 1 This is an overall flowchart of an embodiment of the present invention. Detailed Implementation

[0048] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0049] Figure 1 This is a flowchart of an embodiment. This embodiment provides a few-sample industrial anomaly detection method based on dual-guided contrast. The specific process includes: acquiring a small number of industrial images to construct a few-sample reference set; extracting features from the query image, normal reference image, and abnormal reference image using a pre-trained visual encoder; calculating the normal guidance score in the normal guidance branch; calculating the abnormal guidance score in the abnormal guidance branch; performing dual-guided branch fusion to obtain an anomaly segmentation probability map and the final anomaly score; during training, the final anomaly score is sent to the global contrast aggregation module, introducing pixel-level contrast loss, and calculating the total loss for multiple tasks; during testing, the anomaly detection result is directly output after dual-guided branch fusion.

[0050] The specific implementation steps of a small-sample industrial anomaly detection method based on dual-guided comparison are as follows:

[0051] Step 1: Use an industrial linear scan camera to acquire images of the surface of the workpiece to be inspected, and collect one normal image and one abnormal image to construct a small sample reference set.

[0052] Step 2: Load the DINOv2ViT-S / 14 pre-trained visual encoder, freeze all parameters, and extract features from the query image, normal reference image, and abnormal reference image.

[0053] Step 3: In the normal boot branch, calculate the normal boot score.

[0054] Step 3.1: Flatten the query features and normal reference features respectively, apply L2 normalization, and calculate the cosine similarity. The specific expression is as follows:

[0055] ,

[0056] ,

[0057] ,

[0058] in, These are the query features after being flattened. This is the normal reference feature after flattening. This represents the normalized query characteristics. This represents the normalized reference feature.

[0059] Step 3.2: For each local feature of the query image The value most similar to all normal reference features is calculated as the normal guide score. :

[0060] ,

[0061] in, yes The higher the score of the normal guidance, the more similar the query region is to the normal reference set region, and the greater the possibility that the image is normal.

[0062] Step 4: In the abnormal boot branch, calculate the abnormal boot score.

[0063] Step 4.1: Downsample the defect mask of the abnormal reference image to match the features. Figure One At the same spatial scale, after binarization and flattening, the mean of the feature vectors in the anomaly regions is calculated respectively. The mean of the feature vectors in the normal region Define the residual prototype :

[0064] ,

[0065] in, Characteristic offset of defective regions relative to normal regions.

[0066] Step 4.2: After L2 normalization, the residual prototype is input into a two-layer perceptron mapping network to obtain the projection vector. The query features, after L2 normalization, are input into the same perceptron to obtain the projected query features. .

[0067] Step 4.3: Calculate the similarity between the projected query features and the projected vectors. :

[0068] ,

[0069] in, This is a learnable temperature parameter.

[0070] Step 4.4: Similarity is used to obtain anomaly guidance score through Sigmoid activation function. :

[0071] ,

[0072] in, The larger the value, the more consistent the query region is with the residual prototype, and the greater the possibility of image anomalies.

[0073] Step 5: Merge the scores of the two guiding branches.

[0074] Step 5.1: Normal guidance score With abnormal guidance score The image is concatenated along the channel dimension and transformed into a two-dimensional feature map. This map is then upsampled to the original query image size using bilinear interpolation and normalized using an exponential function to obtain an anomaly segmentation probability map. :

[0075] ,

[0076] in, Indicates the current position of the first Logical values ​​for each category, This represents the sum of the logical values ​​of all categories at the same location.

[0077] Step 5.2: Calculate the final anomaly score :

[0078] .

[0079] Step 6: Introduce a global comparison aggregation module during the training phase.

[0080] Step 6.1: The final anomaly score is weighted and averaged based on the query features. A normalized exponential function with a temperature parameter is input, and the global features are calculated. The specific expression is as follows:

[0081] ,

[0082] in, For the query image of the first Local features, It is the query feature weight of the current position.

[0083] Step 6.2: Global features as Matrix, projection query features as Matrix and The matrix is ​​enhanced using a multi-head cross-attention layer, and the output is the enhanced global feature.

[0084] Step 6.3: The enhanced global features are passed sequentially through a fully connected layer, an activation layer, and another fully connected layer to obtain the image-level binary classification result. The anomaly class is input into the Sigmoid activation function to obtain the anomaly probability. Calculate the global comparison aggregation loss :

[0085] ,

[0086] in, It refers to the batch size. For the first Image tags, Indicates the first The probability of anomalies in a single image.

[0087] Step 7: Introduce pixel-level contrast loss during the training phase. Use image labels to distinguish between defective and background regions in the query image, calculate the mean similarity between defective and background regions, and then calculate the pixel-level contrast loss. Total loss The specific expression is as follows:

[0088] ,

[0089] ,

[0090] in, To query the mean similarity of defect regions in an image, It queries the mean similarity of normal regions in an image. For image-level classification loss, It is a pixel-level segmentation loss. , These represent the pixel-level contrast loss coefficient and the global contrast aggregation loss coefficient, respectively.

[0091] Step 8: During testing, only steps 1 to 5 are executed. The image to be detected is input into the trained model, and the final anomaly score is upsampled to the original image size to obtain the final pixel-level segmentation result. The mean of the top 20 high scores in the final anomaly score is binarized to confirm whether the image to be detected is an anomaly image. The total number of model parameters is 22.246M, and the number of trainable parameters is 0.19M, which meets the requirements of industrial online detection.

Claims

1. A method for detecting industrial anomalies in a small sample based on dual-guided comparison, characterized in that, Includes the following steps: Step 1: Construct a small-sample reference set; Step 2: Extract features using a pre-trained visual encoder; Step 3: In the normal boot branch, calculate the normal boot score; Step 4: In the abnormal boot branch, calculate the abnormal boot score; Step 5: Integrate normal guidance scores and abnormal guidance scores; Step 6: Introduce a global contrast aggregation module during the training phase and calculate the global contrast aggregation loss; Step 7: Introduce pixel-level contrast loss during the training phase and calculate the total loss; Step 8: Output the anomaly detection results during the testing phase.

2. The method for detecting industrial anomalies in a small sample based on dual-guided comparison according to claim 1, characterized in that, The abnormal boot branch mentioned in step 4 is implemented in the following steps: Step 4.1: Downsample the defect mask of the abnormal reference image to the same spatial scale as the feature map, binarize and flatten it, and calculate the mean of the feature vectors in the abnormal regions respectively. The mean of the feature vectors in the normal region Define the residual prototype : ， in, Characteristic offset of abnormal regions relative to normal regions; Step 4.2: After L2 normalization, the residual prototype is input into a two-layer perceptron mapping network to obtain the projection vector. The query features, after L2 normalization, are input into the same perceptron to obtain the projected query features. ; Step 4.3: Calculate the similarity between the projection vector and the projection query feature. : ， in, It is a learnable temperature parameter; Step 4.4: Similarity is used to obtain anomaly guidance score through Sigmoid activation function. : ， in, The larger the value, the more consistent the query region is with the residual prototype, and the greater the possibility of image anomalies.

3. The method for detecting industrial anomalies with few samples based on dual-guided comparison according to claim 1, characterized in that, The global comparison and aggregation module described in step 6 is implemented in the following steps: Step 6.1: The final anomaly score is weighted and averaged based on the query features. A normalized exponential function with a temperature parameter is input, and the global features are calculated. The specific expression is as follows: ， in, For the query image of the first Local features, It is the query feature weight of the current position; Step 6.2: Global features as Matrix, projection query features as Matrix and The matrix is ​​enhanced using multi-head cross-attention and outputs the enhanced global features. Step 6.3: The enhanced global features are passed sequentially through a fully connected layer, an activation layer, and another fully connected layer to obtain the image-level binary classification result. The anomaly class is then activated using the Sigmoid activation function to obtain the anomaly probability. Calculate the global comparison aggregation loss : ， Where B is the batch size. Let b be the label for the b-th image. Let be the probability of an anomaly in the b-th image.

Images