An industrial anomaly detection method based on normal feature guidance

Abstract

The application provides an industrial anomaly detection method based on normal feature guidance, aiming to improve the sensitivity and accuracy of anomaly detection in industrial images. The method is implemented by the following steps: first, based on the MVtec industrial image dataset, a training set containing normal images and simulated abnormal images is constructed; second, a normal feature guidance network is constructed, and the network is trained using the training set; then, an image reconstruction network is constructed and trained to achieve high-quality image reconstruction; finally, a multi-level feature anomaly scoring network is used to score the test images, thereby achieving accurate anomaly detection. The application can effectively improve the precision of industrial image anomaly detection and better adapt to multi-class detection requirements. By comprehensively using image reconstruction and anomaly scoring network, the application not only improves the detection efficiency, but also enhances the recognition ability of subtle defects.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of industrial image anomaly detection in computer vision, and in particular to an industrial anomaly detection method based on normal feature guidance. BACKGROUND

[0002] In traditional industrial anomaly detection, the identification and classification of anomalies often rely on manual inspection, rule-driven algorithms, or sensor-based data monitoring. These methods have significant limitations when faced with complex and diverse production environments. For example, manual detection is not only inefficient, but also susceptible to factors such as fatigue, leading to missed and false detections. Therefore, how to improve the accuracy, real-time performance, and automation level of industrial anomaly detection has become an important problem in current technical research.

[0003] With the development of computer vision technology, more and more image processing and deep learning-based anomaly detection methods have begun to be applied in the industrial field. These methods automatically identify anomalies in product surfaces or equipment states by analyzing image data on production lines. However, existing deep learning models mostly rely on single-class detection methods, i.e., training and detecting specific types of anomalies, which cannot handle multiple types of anomalies. Traditional single-class models usually only focus on a specific type of anomaly, lacking unified detection capabilities for multiple types of anomalies, which often leads to limitations in adapting to complex production environments, especially in diversified industrial products and equipment, where more powerful models are needed to handle different types of anomalies.

[0004] In recent years, diffusion models have been introduced to the field of anomaly detection due to their excellent performance in image generation and transformation tasks. Diffusion models generate or reconstruct high-quality images through a reverse diffusion process and have become a new powerful tool. In particular, ControlNet networks based on diffusion models, as a unified multi-class detection framework, can handle multiple types of anomalies simultaneously, exhibiting stronger adaptability and robustness in complex industrial scenarios. Compared with traditional single-class models, ControlNet networks have obvious advantages. They can solve the problem of multi-class anomaly recognition through a unified framework, avoiding the cumbersome process of training a model for each type of anomaly.

[0005] Although ControlNet network has significant advantages in multi-class detection tasks, existing anomaly detection methods still face the challenge of how to effectively identify minor abnormalities and complex environmental changes. Traditional anomaly detection methods usually use normal data for training and judge anomalies by comparing the differences between normal and abnormal. In some cases, there may be a lack of sufficient abnormal samples in the training data, which makes the model susceptible to noise interference, resulting in reduced detection accuracy. In order to further improve the accuracy and robustness of detection, the present application proposes an innovative method of "normal feature guidance", which learns the features of normal images to provide an effective reference framework for anomaly detection. Through normal feature guidance, the detection ability of the model in complex environments can be improved in the absence of a large number of abnormal samples, avoiding false positives and false negatives, thereby improving the overall effect of industrial anomaly detection. SUMMARY

[0006] In view of the deficiencies of the prior art, the purpose of the present application is to provide an industrial anomaly detection method based on normal feature guidance, which can improve the sensitivity and accuracy of industrial image anomaly detection, improve efficiency, and realize more reliable detection and positioning of product surface defects.

[0007] The technical solution for achieving the purpose of the present application is: an industrial anomaly detection method based on normal feature guidance, comprising the following steps:

[0008] Step 1: Based on the MVtec industrial image dataset, use its 15 categories totaling 3466 normal images as the normal image set, denoted as I; calculate the corresponding simulated abnormal image set of the normal image set I, denoted as I a ; and then pre-process the images in the normal image set and the simulated abnormal image set to obtain a training set consisting of 3466 image pairs, denoted as I train .

[0009] Step 2: Construct a normal feature guidance network and train the normal feature guidance network using the training set I train .

[0010] Step 3: Construct an image reconstruction network.

[0011] Step 4: Train the image reconstruction network using the training set I train .

[0012] Step 5: Construct a multi-level feature anomaly scoring network.

[0013] Step 6: Obtain the public MVtec industrial image dataset to obtain 1888 test images I test and the corresponding true value images I groundtruth .

[0014] Step 7: input the test image I test into the multi-level feature anomaly scoring network for testing to obtain the corresponding anomaly score S, and compare the obtained anomaly score S with the true value image I groundtruth , draw the corresponding AUROC curve to test the test effect.

[0015] The present application has the following beneficial effects:

[0016] 1) The present application proposes a new unified multi-class anomaly detection method, which can solve the recognition problem of multi-class anomaly through a unified model, avoiding the cumbersome process of training a model for each anomaly.

[0017] 2) The present application divides the anomaly detection network into two stages, and in the first stage, a feature extractor with normal feature perception ability is first trained, and is used to guide the reconstruction network in the second stage to reconstruct the picture to be detected, improving the detection effect. BRIEF DESCRIPTION OF DRAWINGS

[0018] The drawings accompanying the specification of the present application form a part thereof and serve to provide further understanding of the present application, the illustrative embodiments of the present application and its description serve to explain the present application, and do not constitute an improper limitation on the present application.

[0019] Figure 1 is the step flow chart of the anomaly detection model in the embodiment of the present application.

[0020] Figure 2 is the normal feature guidance network structure diagram in the embodiment of the present application.

[0021] Figure 3 is the image reconstruction network structure diagram in the embodiment of the present application.

[0022] Figure 4 is the ControlNet network structure diagram in the image reconstruction network in the embodiment of the present application.

[0023] Figure 5 is the multi-level feature anomaly scoring network structure diagram in the embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to more clearly understand the above-mentioned purposes, features and advantages of the present application, the present application will be further described in detail below in combination with the drawings and specific embodiments. It should be understood that the drawings in the present application example only serve the purpose of illustration and description, and do not limit the protection scope of the present application embodiment.

[0025] One specific embodiment of the present application is as follows: Figure 1 A normal feature guidance-based industrial anomaly detection method, the specific steps are as follows:

[0026] Step 1: Based on the MVtec industrial image dataset, use its 15 categories totaling 3466 normal images as the normal image set, denoted as I; calculate the corresponding simulated abnormal image set of the normal image set I, denoted as I a ; and then preprocess the images in the normal image set I and the simulated abnormal image set I a to obtain a training set consisting of 3466 image pairs, denoted as I train , as follows:

[0027] S1.1, based on the MVtec industrial image dataset, use its 15 categories totaling 3466 normal images as the normal image set, denoted as I; download the corresponding abnormal images of the normal image set I, denoted as the abnormal image set P (abnormal images from RealNet: A Feature Selection Network with Realistic Synthetic Anomaly for Anomaly Detection).

[0028] S1.2, use Perlin noise generator to capture the abnormal shape of each image in the abnormal image set P, and binarize the above abnormal shape into an abnormal mask M, as follows:

[0029]

[0030] where N(x, y) is the Perlin noise value at position (x, y), t is the threshold value, and M(x, y) is the binarized mask value at position (x, y).

[0031] S1.3, by weighted mixing the normal image set I and the abnormal image set P, obtain the corresponding simulated abnormal image set I a , as follows:

[0032] I a = |1-M|⊙I+(1-δ)·M⊙I+δ·M⊙P

[0033] where ⊙ is the Hadamard product, representing element-wise multiplication between two matrices, and δ is the opacity parameter during mixing, which is uniformly sampled from an interval, i.e. δ∈[0.1,1.0].

[0034] S1.4, preprocess the normal image set I and the simulated abnormal image set I a to obtain the training set I train :

[0035] I train = Tensor(Resize(I,I a ))

[0036] where Resize means adjusting all image resolutions to 256*256, and Tensor means linearly mapping the value range of the image from [0, 255] to [0.0, 1.0] so as to become a tensor that can be input into the neural network.

[0037] Go to step 2.

[0038] Step 2: Combine Figure 2 , build a normal feature guidance network, and use the training set I train Train the normal feature guidance network.

[0039] The normal feature guidance network includes two structurally identical feature extractors swin transformer, which are the first feature extractor and the second feature extractor. Among them, the first feature extractor is pre-trained to extract the features of normal images, and the second feature extractor needs to be trained to extract the features of simulated abnormal images.

[0040] The specific training process is as follows:

[0041] Input the normal images in the training set I train into the first feature extractor to obtain the corresponding normal features f, and input the abnormal images in the training set I train into the second feature extractor to obtain the corresponding simulated abnormal features f a .

[0042] The loss function for training the normal feature guidance network is the MSE mean square error loss function, which is as follows:

[0043]

[0044] where L f represents the loss, B is the feature batch size, L is the sequence length of the output feature, D is the embedding dimension of each feature, f b,l,d is the normal feature value of the b-th sample, the l-th position, and the d-th feature dimension, is the corresponding simulated abnormal feature value of the b-th sample, the l-th position, and the d-th feature dimension.

[0045] After training, the obtained second feature extractor can extract simulated abnormal features f a with normal feature information, which can be used to guide the image reconstruction network to perform image reconstruction. Go to step 3.

[0046] Step 3: Build an image reconstruction network.

[0047] As Figure 3As shown, the image reconstruction network includes a second feature extractor, a feature encoder, a ControlNet network, a feature decoder, and a cross-attention module.

[0048] The feature encoder adopts a lightweight neural network structure and is mainly used for converting the input image to a 64x64 feature space. The feature encoder includes an input convolutional layer, multiple intermediate convolutional blocks, and an output convolutional layer. Specifically, the input convolutional layer receives a 3-channel RGB image input and outputs a 16-channel feature map; each intermediate convolutional block adopts a cascaded structure, each intermediate convolutional block includes two convolutional layers, the first convolutional layer keeps the channel number and resolution unchanged, and the second convolutional layer realizes the downsampling of the feature map size by setting stride=2, while increasing the channel number, and the channel number changes to 16, 32, 96, and 256 in turn; and finally, the final feature representation is obtained through the output convolutional layer. The network structure of the feature decoder and the feature encoder is completely symmetrical, and the feature is restored to an image through step-by-step upsampling.

[0049] Further, the ControlNet network includes a backbone network and a conditional control network. The backbone network includes an encoding part and a decoding part. Figure 4 The encoding part in the backbone network and the conditional control network both adopt an encoder structure with step-by-step downsampling, and the feature map size is sequentially downsampled from 64x64 to 32x32, 16x16, and 8x8. Specifically, the encoding part in the backbone network includes four encoder modules and an intermediate layer module, which are used to extract a multi-scale representation of the input feature; the conditional control network also includes four encoder modules and an intermediate layer module, which are used to extract a feature representation of the control condition.

[0050] In the decoding stage, the decoding part of the backbone network gradually recovers the spatial resolution of the feature map from 8x8 to 64x64 through four decoder modules, wherein each decoder module receives the control feature from the corresponding scale and performs feature fusion through a skip connection. This design ensures that the control feature can accurately guide the generation process at different spatial scales.

[0051] To improve the image reconstruction effect, the present application introduces a cross-attention module in the ControlNet network. The cross-attention module fuses the features in the ControlNet network with the simulated abnormal feature f a through a cross-attention mechanism. Specifically, the cross-attention module takes the features extracted in the ControlNet network as the query (Query), and takes the simulated abnormal feature f a extracted by the second feature extractor as the key (Key) and value (Value). Through the cross-attention mechanism, the cross-attention module can dynamically capture the simulated abnormal feature f aThe correlation between the ControlNet network features is obtained, and the correlation is fed back to the feature map of the ControlNet network. Since the simulation anomaly features f a With the characteristics of normal images, therefore, this design enables the ControlNet network to combine the information of the normal images corresponding to the input images, thereby improving the image reconstruction effect of the network.

[0052] Go to step 4.

[0053] Step 4: Use the training set I train Train the image reconstruction network.

[0054] Use the training set I train Train the image reconstruction network to generate clear reconstructed images. The training loss of the image reconstruction network is the MSE mean square error loss function, which is used to measure the pixel-level error between the reconstructed image and the input image, and is as follows:

[0055]

[0056] Where L r represents the loss, B is the image batch size, H and W represent the height and width of the image respectively, and C represents the channel number of the image, represents the pixel value of the image in the bth batch in the simulation anomaly image set I a in the cth channel, at position (h, w), represents the pixel value of the reconstructed image in the bth batch output by the network in the cth channel, at position (h, w).

[0057] Go to step 5.

[0058] Step 5: Construct a multi-level feature anomaly scoring network.

[0059] The multi-level feature anomaly scoring network includes a pre-trained Resnet50 and a multi-level feature weighted fusion scoring module. Specifically, for the input image x input and the reconstructed image x output , the Resnet50 can extract the corresponding features f input of the input image x output and the reconstructed image x ​Wherein, 5 represents the number of feature layers, d represents the dimension of each layer of features, and h and w represent the height and width of the features, respectively. Considering that the first layer of features retains most of the information of the original image and the last layer of features is too abstract, the present application mainly uses the middle three layers of features, whose specific scales are [256, 64, 64], [512, 32, 32] and [1024, 16, 16] in turn.

[0060] As shown in Figure 5 , the multi-level feature weighted fusion scoring module calculates the abnormal image and the final score according to the extracted f input and f output .

[0061]

[0062]

[0063] Wherein, n represents the middle three layers of features in the pre-trained Resnet50, n = 2, 3, 4, represents the abnormal image of the nth layer, T represents the transpose of the matrix, represents the input feature of the nth feature layer, represents the output feature of the nth layer. S represents the final abnormal score, and a n is a coefficient, representing the weight of the nth layer of abnormal image.

[0064] Step 6.

[0065] Step 6: Reacquire the public MVtec industrial image dataset to obtain 15 categories of a total of 1888 test images I test and the corresponding true value image I groundtruth ; turn to step 7.

[0066] Step 7: input the test image I test into the multi-level feature anomaly scoring network for testing to obtain the corresponding abnormal score S, and compare the abnormal score S with the true value image I groundtruth , draw the corresponding AUROC curve to detect the model effect, as follows:

[0067]

[0068] TPR = TP / (TP+FN)

[0069] FPR = FP / (FP+TN)

[0070] where AUROC (Area Under the Receiver Operating Characteristic Curve) is an index to measure the performance of the model, TPR is the true positive rate, which indicates how many of the pixels predicted as abnormal are actually abnormal, FPR is the false positive rate, which indicates how many of the pixels predicted as abnormal are actually normal. TP indicates that the abnormal region is correctly predicted as abnormal, i.e. S(x, y) = 1 and I groundtruth (x, y) = 1; FP indicates that the normal region is incorrectly predicted as abnormal, i.e. S(x, y) = 1 and I groundtruth (x, y) = 0, TN indicates that the normal region is correctly predicted as normal, i.e. S(x, y) = 0 and I groundtruth (x, y) = 0, FN indicates that the abnormal region is incorrectly predicted as normal, i.e. S(x, y) = 0 and I groundtruth (x, y) = 1. S(x, y) and I groundtruth (x, y) represent the anomaly score S and the ground truth image I groundtruth the binarized value at pixel (x, y).

Claims

1. An industrial anomaly detection method based on normal feature guidance, characterized by, The method comprises the following steps: Step 1: Based on the MVtec industrial image dataset, use its 3466 normal images in total of 15 categories as a normal image set, denoted as ; calculate the normal image set ; the corresponding simulated abnormal image set, denoted as ; and then pre-process the images in the normal image set and the simulated abnormal image set to obtain a training set composed of 3466 image pairs, denoted as ; Step 2: Construct normal feature guided network and utilize training set Train normal feature guided network; The normal feature guidance network comprises two structural identical feature extractors swin transformers, which are a first feature extractor and a second feature extractor respectively; wherein the first feature extractor is pre-trained and used for extracting features of normal images, and the second feature extractor is trainable and used for extracting features of simulated abnormal images; After the training is completed, the obtained second feature extractor can extract the simulated abnormal features with normal feature information The second feature extractor is used for guiding the image reconstruction network to perform image reconstruction. Step 3: constructing an image reconstruction network; Step 4: Utilizing the training set The image reconstruction network is trained to generate clear reconstructed images; Step 5: constructing a multi-level feature abnormal scoring network, specifically as follows: The multi-level feature abnormal scoring network comprises a pre-trained Resnet50 and a multi-level feature weighted fusion scoring module; For the input image and reconstructed image set Reconstructed images ResNet50 can extract input images Corresponding features and reconstructed images Corresponding features ; Wherein, 5 represents the feature layer number, d represents the dimension of each layer feature, h and w represent the height and width of the feature respectively; The intermediate three-layer features are used, and the specific scales thereof are [256, 64, 64], [512, 32, 32] and [1024, 16, 16] in sequence; The multi-level feature weighting fusion scoring module calculates the final score according to the extracted features and The abnormal graph and the final score are calculated as follows: ， ， in, This represents the three middle feature layers in a pre-trained ResNet50. =2,3,4 Indicates the first Anomaly graph of the layer, To represent the transpose of a matrix, Indicates the first The input features of each feature layer Indicates the first The output features of the layer; This indicates the final outlier score. Indicates the first Weights of the layer anomaly graph; Step 6: Obtain the public MVtec industrial image dataset, resulting in 1888 test images in total for 15 classes and the corresponding ground truth images ; Step 7: input the test image to the multi-level feature anomaly scoring network for testing to obtain the corresponding anomaly score , and compare the obtained anomaly score with the true value image , draw the corresponding AUROC curve to test the test effect. 2.The normal feature guided based industrial anomaly detection method according to claim 1, wherein, In step 1, based on the MVtec industrial image dataset, 3466 normal images in total of its 15 categories are used as a normal image set, denoted as ; a normal image set corresponding to the simulated abnormal image set, denoted as ; and the images in the normal image set and the simulated abnormal image set are preprocessed to obtain a training set composed of 3466 image pairs, denoted as , as follows: S1.1, based on the MVtec industrial image dataset, with its 15 classes totaling 3466 normal images, as normal image set denoted by ; download normal image set corresponding abnormal images, denoted by abnormal image set ; S1.2, capturing a set of anomaly images with a Perlin noise generator abnormal shapes of each image in the middle, and binarizing the above abnormal shapes into an anomaly mask as follows: ， wherein is a Perlin noise value at position , is a threshold value, is a binarized mask value at position ; S1.3, by using the normal image set and abnormal image set By performing weighted mixing, the corresponding simulated anomaly image set is obtained. The details are as follows: ， wherein, is the Hadamard product, denoting element-wise multiplication between two matrices, is the opacity parameter at the time of mixing; S1.4, pre-process the normal image set and the simulated abnormal image set to obtain a training set : ， wherein, means to adjust all image resolutions to 256*256, means to linearly map the numerical range of the image from [0, 255] to [0.0, 1.0] to become a tensor that can be input to the neural network. 3.The normal feature guided based industrial anomaly detection method of claim 2, wherein, In step 2, the training set is used The normal feature guided network is trained as follows: training set Normal image set in The input is fed into the first feature extractor to obtain the normal features corresponding to each normal image. , training set Simulated anomaly image set The data is input into the second feature extractor to obtain the abnormal features corresponding to each simulated abnormal image. ; The loss function for training the normal feature guidance network is an MSE mean square error loss function, specifically as follows: ， in, Indicates loss, It is the feature batch size. It is the sequence length of the output features. It is the embedding dimension of each feature. It is the first The sample, the first The position, the Normal feature values ​​of each feature dimension It is the corresponding number The sample, the first The position, the Simulated abnormal feature values ​​of each feature dimension.

4. The normal feature guided based industrial anomaly detection method of claim 3, wherein, In step 3, the image reconstruction network is constructed, specifically as follows: The image reconstruction network comprises the second feature extractor, a feature encoder, a ControlNet network, a feature decoder and a cross attention module; The feature encoder adopts a lightweight neural network structure for converting an input image to a 64x64 feature space; The feature encoder comprises an input convolutional layer, a plurality of intermediate convolutional blocks and an output convolutional layer; the input convolutional layer receives a 3-channel RGB image input and outputs a 16-channel feature map; each intermediate convolutional block adopts a cascaded structure, each intermediate convolutional block comprises two convolutional layers, the first convolutional layer keeps the channel number and resolution unchanged, and the second convolutional layer realizes the downsampling of the feature map size by setting stride=2, while increasing the channel number, and the channel number changes in sequence as 16, 32, 96 and 256; finally, the final feature representation is obtained through the output convolutional layer; The network structure of the feature decoder is completely symmetrical to that of the feature encoder; The ControlNet network comprises a backbone network and a conditional control network; the backbone network comprises an encoding part and a decoding part, and the encoding part in the backbone network and the conditional control network both adopt an encoder structure with stepwise downsampling, and the feature map size is sequentially downsampled from 64x64 to 32x32, 16x16 and 8x8; The encoding part in the backbone network comprises four encoder modules and an intermediate layer module, and is used for extracting a multi-scale representation of an input feature; The conditional control network comprises four encoder modules and an intermediate layer module, and is used for extracting a feature representation of a control condition; In the decoding stage, the decoding part of the backbone network gradually recovers the spatial resolution of the feature map from 8x8 to 64x64 through four decoder modules, wherein each decoder module receives control features at a corresponding scale and performs feature fusion through a skip connection. The cross-attention module takes the features extracted in the ControlNet network as queries, and the analog abnormal features extracted by the second feature extractor As keys and values; through the cross-attention mechanism, the cross-attention module can dynamically capture the analog abnormal features The correlation between the ControlNet network features, and feed this correlation back into the feature map of the ControlNet network.

5. The normality feature guided industrial anomaly detection method of claim 4, wherein, In step 4, the training set is used The image reconstruction network is trained to generate clear reconstructed images, as follows: The training loss corresponding to the image reconstruction network is the MSE mean square error loss function, which is used to measure the pixel-level error between the reconstructed image and the input image as follows: ， in, Indicates loss, It is the image batch size. and These represent the height and width of the image, respectively. This represents the number of channels in an image. Represents a set of simulated anomaly images The first in Images in batch 1 One channel, located at Pixel value at that location, Indicates the network output of the first The reconstructed images in the batch are in the first One channel, located at The pixel value at that location.

6. The normality feature guided industrial anomaly detection method of claim 5, wherein, In step 7, the test image is input into the multi-level feature anomaly scoring network for testing to obtain a corresponding anomaly score The obtained anomaly score is compared with the true value image , and a corresponding AUROC curve is drawn to test the testing effect; specifically as follows: ， ， ， AUROC is an index to measure the performance of the model, is the true positive rate, indicating how many of the pixels predicted as abnormal are actually abnormal, is the false positive rate, indicating how many of the pixels predicted as abnormal are actually normal; represents that the abnormal region is correctly predicted as abnormal, i.e. ; represents that the normal region is incorrectly predicted as abnormal, i.e. , represents that the normal region is correctly predicted as normal, i.e. , represents that the abnormal region is incorrectly predicted as normal, i.e. ; represents the abnormal score , represents the true value image at the pixel binary value.

Images