A small sample anomaly detection method and device based on feature storage

Abstract

The present application relates to the technical field of industrial anomaly detection, in particular to a small sample anomaly detection method and device based on feature storage and a computer readable storage medium. The method comprises the following steps: dividing a data set into a training set and a test set; extracting features of the training set and storing them in a feature storage library; performing data enhancement on samples in the training set, constructing a feature transformation network to perform feature transformation on the features in the feature storage library, obtaining mapping features, and optimizing the feature transformation network by calculating the loss function of the features of the data enhanced image and the mapping features; extracting features of the test set, calculating the anomaly score of the sample image in the test set, determining the segmentation threshold of the normal image and the abnormal image; calculating the anomaly score of the image to be detected, and classifying the image to be detected according to the segmentation threshold. The present application extracts the information of the original stored features through the feature transformation network, expands the existing information quantity in the feature storage library, and improves the accuracy of small sample anomaly detection.

Description

Technical Field

[0001] This invention relates to the field of industrial anomaly detection technology, and in particular to a small-sample anomaly detection method, device, and computer-readable storage medium based on feature storage. Background Technology

[0002] Unsupervised anomaly detection is a crucial research topic in computer vision. The unsupervised anomaly detection task trains a model on a training set containing only normal samples, requiring the model to distinguish between normal and abnormal samples during testing. This technology can be widely applied in the manufacturing industry to detect industrial products with defects such as dirt and scratches, reducing the defect rate.

[0003] Existing research has explored a particularly practical unsupervised anomaly detection method: few-shot anomaly detection. During training, only a limited number of normal samples are provided for each category. Current methods for few-shot anomaly detection mainly include those based on normalized flow, Siamese networks, and feature storage. Normalized flow-based methods use a convolutional neural network to extract image features during training and use normalized flow to estimate the distribution of normal samples. During testing, the distance between the features of the test image and this distribution is calculated as the anomaly score. Siamese network-based methods use paired normal image samples for training. The goal of training the Siamese network is to minimize the representational space distance between positive sample pairs. After training, the Siamese network is used for testing to determine the similarity between input samples and normal samples. Feature storage-based methods use a pre-trained CNN (Convolutional Neural Network) to extract feature maps of the image during training. All feature maps are stored to form a multidimensional Gaussian distribution. During testing, the distance between the features of the test image and this Gaussian distribution is calculated as the anomaly score.

[0004] Compared to the other two methods, feature storage-based methods are simple and can achieve good results without training. However, existing feature storage-based methods are mainly geared towards anomaly detection under normal conditions. With small sample sizes, the detection accuracy drops significantly as the number of stored samples decreases. Summary of the Invention

[0005] Therefore, the technical problem to be solved by the present invention is to overcome the problem that the detection accuracy of unsupervised anomaly detection methods based on feature storage in the prior art is seriously reduced in the case of small samples.

[0006] To address the aforementioned technical problems, this invention provides a small-sample anomaly detection method based on feature storage, comprising:

[0007] S1. Obtain a small sample unsupervised anomaly detection dataset and divide the dataset into a training set and a test set;

[0008] S2. Input each patch of all sample images in the training set into the feature extractor to obtain the output feature set of each patch of all sample images in the training set, and store it in the feature storage store.

[0009] S3. Perform data augmentation on all sample images in the training set, input each patch of the data-augmented image into the feature extractor to obtain the output feature set of each patch of the data-augmented image, calculate the distance between each output feature in the output feature set of each patch and all features in the feature repository, and select the minimum value from the distances to construct the nearest distance set of the patch.

[0010] S4. Create a feature transformation network. Input the features in the feature repository corresponding to the minimum value in the nearest distance set of each patch into the feature transformation network to obtain the mapping features of each patch. Calculate the loss function value between the mapping features of each patch and the output features of the data-enhanced image corresponding to the minimum value in the nearest distance set of that patch. Optimize the parameters of the feature transformation network based on the loss function value to minimize the loss function value.

[0011] S5. Use a feature extractor to extract features from each patch of the sample images in the test set. Find the feature in the feature repository that is closest to the output feature of each patch of the sample images in the test set, and input it into the optimized feature transformation network to obtain the mapping feature of each patch of the sample image. Calculate the loss function value between the mapping feature of each patch and the output feature of the patch to form an anomaly hotspot map, and select the largest loss function value in the anomaly hotspot map as the anomaly score of the sample image. Compare the anomaly scores of normal images and anomaly images in the test set, and set the segmentation thresholds for normal images and anomaly images.

[0012] S6. Calculate the anomaly score of the image to be detected; if the anomaly score is greater than the segmentation threshold, the image to be detected is an abnormal image; if the anomaly score is less than or equal to the segmentation threshold, the image to be detected is a normal image.

[0013] In one embodiment of the present invention, in S1, the training set contains only normal images, and the test set contains both normal images and abnormal images.

[0014] In one embodiment of the present invention, step S2, which involves inputting each patch of all sample images in the training set into a feature extractor to obtain the output feature set of each patch of all sample images in the training set, includes:

[0015] Use a feature extractor to extract a patch from sample image b. Output characteristics The formula is:

[0016]

[0017] Where B is the total number of sample images in the training set, h is the height of the sample image, w is the width of the sample image; φ represents the feature extractor;

[0018] Get patch p j Output feature set

[0019] In one embodiment of the present invention, in step S3, the distance between each output feature in the output feature set of each patch and all features in the feature repository is calculated, and the minimum value of the distances is selected to construct the nearest distance set for the patch, including:

[0020] Calculate patch p j Output feature set Each output feature The minimum distance to all features in the feature repository M is used to construct the nearest distance set for the patch. The formula is:

[0021]

[0022] Where M is the feature repository. represents the features in the feature repository, and N is the total number of data-augmented images.

[0023] In one embodiment of the present invention, in S4, the feature transformation network includes two linear layers and one activation layer.

[0024] In one embodiment of the present invention, the loss function of the feature transformation network is:

[0025]

[0026] Where F represents the feature transformation network, The nearest distance set d j The feature in the feature repository corresponding to the minimum value in the middle. The nearest distance set d j Output the sample image corresponding to the minimum value.

[0027] In one embodiment of the present invention, the expression for the abnormal hotspot map P is:

[0028] P = |Q i -F(m i )| 2 ,i∈R h*w

[0029] Where F represents the feature transformation network, Q i Patch p for R of the test set sample images i The output features, mi For the feature repository with Q i The feature that is closest to you.

[0030] In one embodiment of the present invention, in step S5, after constructing the abnormal heat map, the abnormal heat map is smoothed by Gaussian filtering, and the largest loss function value in the smoothed abnormal heat map is selected as the abnormal score of the sample image.

[0031] The present invention also provides a small sample anomaly detection device based on feature storage, comprising:

[0032] The sampling module is used to acquire a small sample unsupervised anomaly detection dataset, which is divided into a training set and a test set.

[0033] The feature repository building module is used to input each patch of all sample images in the training set into the feature extractor to obtain the output feature set of each patch of all sample images in the training set and store it in the feature repository.

[0034] The feature network construction module is used to perform data augmentation on all sample images in the training set. Each patch of the augmented image is input into a feature extractor to obtain the output feature set of each patch. The distance between each output feature in the output feature set of each patch and all features in the feature repository is calculated, and the minimum value is selected to construct the nearest distance set for that patch. A feature transformation network is created, and the features in the feature repository corresponding to the minimum value in the nearest distance set of each patch are input into the feature transformation network to obtain the mapped features of each patch. The loss function value between the mapped features of each patch and the output features of the augmented image corresponding to the minimum value in the nearest distance set of that patch is calculated. The parameters of the feature transformation network are optimized based on the loss function value to minimize the loss function value.

[0035] The threshold setting module is used to extract features for each patch of the sample images in the test set using a feature extractor. It searches the feature repository for the feature input optimized feature transformation network that has the closest feature distance to each patch of the sample images in the test set, and obtains the mapping features of each patch of the sample image. It calculates the loss function value of the mapping features of each patch and the output features of the patch to form an anomaly hotspot map, and selects the largest loss function value in the anomaly hotspot map as the anomaly score of the sample image. It compares the anomaly scores of normal images and anomaly images in the test set and sets the segmentation thresholds for normal images and anomaly images.

[0036] The detection module is used to calculate the anomaly score of the image to be detected. If the anomaly score is greater than the segmentation threshold, the image to be detected is an abnormal image. If the anomaly score is less than or equal to the segmentation threshold, the image to be detected is a normal image.

[0037] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described feature-stored small sample anomaly detection method.

[0038] The technical solution of the present invention has the following advantages compared with the prior art:

[0039] This invention discloses a few-sample anomaly detection method based on feature storage. It constructs a feature repository using a small number of samples and creates and optimizes a feature transformation network. The features in the feature repository are further processed by the feature transformation network to extract feature information, resulting in mapped features that calculate the anomaly score of the image. Segmentation thresholds are set for normal and abnormal images. During detection, the feature with the smallest difference from the output features of the image to be detected is found in the feature repository. This feature is then transformed to extract information from the original stored features, expanding the information content in the feature repository and improving the accuracy of few-sample anomaly detection. This invention is better suited to situations where training samples are limited in industrial production while maintaining high anomaly detection accuracy. Attached Figure Description

[0040] To make the content of this invention easier to understand, the invention will be further described in detail below with reference to specific embodiments and accompanying drawings, wherein...

[0041] Figure 1 This is a flowchart of a small sample anomaly detection method based on feature storage according to the present invention;

[0042] Figure 2 This is a training structure diagram of a small sample anomaly detection method based on feature storage according to the present invention. Detailed Implementation

[0043] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand and implement the present invention. However, the embodiments described are not intended to limit the present invention.

[0044] Example 1

[0045] Reference Figure 1 As shown, this invention provides a small-sample anomaly detection method based on feature storage, comprising:

[0046] S1. Obtain a small sample unsupervised anomaly detection dataset and divide the dataset into a training set and a test set.

[0047] This embodiment uses the MVTec dataset, which contains 15 categories of objects or textures, such as bottles, screws, and textile surface textures. Each category includes a training set and a test set. The training set is a small sample containing only normal images, while the test set contains both normal and abnormal images.

[0048] S2, Reference Figure 2 As shown, each patch of all sample images in the training set is input into the feature extractor, i.e., a pre-trained convolutional neural network. In this embodiment, the output features of the second and third layers of the feature extractor are used, and upsampling is performed to make the output feature sizes of the second and third layers equal. Then, the two are concatenated as the final output feature to obtain the patch of sample image b in the training set. Output characteristics The formula is:

[0049]

[0050] Where B is the total number of sample images in the training set, h is the height of the sample image, v is the width of the sample image, and φ represents the feature extractor.

[0051] This leads to the acquisition of patch p j Output feature set Store in the feature repository M.

[0052] S3. Perform data augmentation on all sample images in the training set, and input each patch of the augmented image into the feature extractor to obtain patch p in image n. j Output characteristics Where n = 1, 2, ..., N, and N is the total number of data-augmented images.

[0053] Obtain the data-enhanced image patch p j Output feature set Calculate the output feature set Each output feature The minimum distance to all features in the feature repository M is used to construct the nearest distance set for the patch. The calculation formula is:

[0054]

[0055] Where M is the feature repository. represents the features in the feature repository, and N is the total number of data-augmented images.

[0056] Selecting the minimum value from the nearest distance set yields the feature in the feature repository corresponding to that minimum value. The output feature of the data-enhanced image corresponding to the minimum value is:

[0057] S4. Since the feature repository is small, directly use the feature in the feature repository corresponding to the minimum value in the nearest distance set. Output features of data-enhanced images Calculating anomaly scores resulted in low detection accuracy; therefore, features from the feature repository corresponding to the minimum value in the nearest distance set were used instead. Feature transformation is performed using the feature transformation network F.

[0058] Create a feature transformation network, which includes two linear layers and one activation layer. Apply patch p. j The feature in the feature repository corresponding to the minimum value in the nearest distance set. Input the feature transformation network F to obtain patch p j Mapping features Calculate patch p j Mapping features Output features of the data-augmented image corresponding to the minimum value in the nearest distance set of the patch. Loss function value:

[0059]

[0060] The parameters of the feature transformation network are optimized based on the loss function value to minimize the loss function value.

[0061] S5. Use a feature extractor to extract patches p from the sample images R in the test set. i The features are used to find patches p in the feature repository that match the sample images in the test set. j Feature Q i The nearest feature m i , will feature m i Input the optimized feature transformation network to obtain the mapped features F(m) i ). Calculate patch p i The mapping feature F(m) i ) and the output feature Q of this patch i The loss function values ​​are used to construct an abnormal hotspot map P. The expression for the abnormal hotspot map P is:

[0062] P = |Q i -F(m i )| 2 ,i∈R h*w

[0063] Where F represents the feature transformation network, Q iPatch p for R of the test set sample images i The output features, m i For the feature repository with Q i The feature that is closest to you.

[0064] Gaussian filtering is applied to smooth the abnormal heatmap, and the maximum loss function value in the smoothed abnormal heatmap is selected as the abnormal score of the sample image.

[0065] Compare the abnormality scores of normal and abnormal images in the test set, and set the segmentation thresholds for normal and abnormal images.

[0066] S6. Calculate the anomaly score of the image to be detected; if the anomaly score is greater than the segmentation threshold, the image to be detected is an abnormal image; if the anomaly score is less than or equal to the segmentation threshold, the image to be detected is a normal image.

[0067] The present invention discloses a small sample anomaly detection method based on feature storage. It finds the feature with the smallest difference from the output feature of the image to be detected from the feature storage repository, performs feature transformation on the feature with the smallest difference, extracts the information of the original stored feature, expands the amount of information in the feature storage repository, and improves the accuracy of small sample anomaly detection.

[0068] Example 2

[0069] This embodiment provides a small-sample anomaly detection device based on feature storage, including:

[0070] The sampling module is used to acquire a small sample unsupervised anomaly detection dataset, which is divided into a training set and a test set.

[0071] The feature repository building module is used to input each patch of all sample images in the training set into the feature extractor to obtain the output feature set of each patch of all sample images in the training set and store it in the feature repository.

[0072] The feature network construction module is used to perform data augmentation on all sample images in the training set. Each patch of the augmented image is input into a feature extractor to obtain the output feature set of each patch. The distance between each output feature in the output feature set of each patch and all features in the feature repository is calculated, and the minimum value is selected to construct the nearest distance set for that patch. A feature transformation network is created, and the features in the feature repository corresponding to the minimum value in the nearest distance set of each patch are input into the feature transformation network to obtain the mapped features of each patch. The loss function value between the mapped features of each patch and the output features of the augmented image corresponding to the minimum value in the nearest distance set of that patch is calculated. The parameters of the feature transformation network are optimized based on the loss function value to minimize the loss function value.

[0073] The threshold setting module is used to extract features for each patch of the sample images in the test set using a feature extractor. It searches the feature repository for the feature input optimized feature transformation network that has the closest feature distance to each patch of the sample images in the test set, and obtains the mapping features of each patch of the sample image. It calculates the loss function value of the mapping features of each patch and the output features of the patch to form an anomaly hotspot map, and selects the largest loss function value in the anomaly hotspot map as the anomaly score of the sample image. It compares the anomaly scores of normal images and anomaly images in the test set and sets the segmentation thresholds for normal images and anomaly images.

[0074] The detection module is used to calculate the anomaly score of the image to be detected. If the anomaly score is greater than the segmentation threshold, the image to be detected is an abnormal image. If the anomaly score is less than or equal to the segmentation threshold, the image to be detected is a normal image.

[0075] Example 3

[0076] This embodiment provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the aforementioned feature-based small sample anomaly detection method.

[0077] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0078] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0079] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0080] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0081] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A small-sample anomaly detection method based on feature storage, characterized in that, include: S1. Obtain a small sample unsupervised anomaly detection dataset and divide the dataset into a training set and a test set; S2. Input each patch of all sample images in the training set into the feature extractor to obtain the output feature set of each patch of all sample images in the training set, and store it in the feature storage store. S3. Perform data augmentation on all sample images in the training set, input each patch of the data-augmented image into the feature extractor to obtain the output feature set of each patch of the data-augmented image, calculate the distance between each output feature in the output feature set of each patch and all features in the feature repository, and select the minimum value from the distances to construct the nearest distance set of the patch. S4. Create a feature transformation network. Input the features in the feature repository corresponding to the minimum value in the nearest distance set of each patch into the feature transformation network to obtain the mapping features of each patch. Calculate the loss function value between the mapping features of each patch and the output features of the data-enhanced image corresponding to the minimum value in the nearest distance set of that patch. Optimize the parameters of the feature transformation network based on the loss function value to minimize the loss function value. S5. Use a feature extractor to extract features from each patch of the sample images in the test set. Find the feature in the feature repository that is closest to the output feature of each patch of the sample images in the test set. Input the feature transformation network into the optimized feature transformation network to obtain the mapping feature of each patch of the sample image. Calculate the loss function value between the mapping features of each patch and the output features of that patch to form an anomaly hotspot map, and select the largest loss function value in the anomaly hotspot map as the anomaly score of the sample image; compare the anomaly scores of normal images and anomaly images in the test set, and set the segmentation thresholds for normal images and anomaly images. S6. Calculate the anomaly score of the image to be detected; if the anomaly score is greater than the segmentation threshold, the image to be detected is an abnormal image; if the anomaly score is less than or equal to the segmentation threshold, the image to be detected is a normal image.

2. The small sample anomaly detection method based on feature storage according to claim 1, characterized in that, In S1, the training set contains only normal images, and the test set contains both normal and abnormal images.

3. The small sample anomaly detection method based on feature storage according to claim 1, characterized in that, In S2, the step of inputting each patch of all sample images in the training set into the feature extractor to obtain the output feature set of each patch of all sample images in the training set includes: Use a feature extractor to extract a patch from sample image b. Output characteristics The formula is: Where B is the total number of sample images in the training set, h is the height of the sample image, w is the width of the sample image; φ represents the feature extractor; Get patch p j Output feature set 4. The small sample anomaly detection method based on feature storage according to claim 3, characterized in that, In S3, the distance between each output feature in the output feature set of each patch and all features in the feature repository is calculated. The minimum value among these distances is selected to construct the nearest distance set for that patch, including: Calculate patch p j Output feature set Each output feature The minimum distance to all features in the feature repository M is used to construct the nearest distance set for the patch. The formula is: Where M is the feature repository. represents the features in the feature repository, and N is the total number of data-augmented images.

5. The small sample anomaly detection method based on feature storage according to claim 4, characterized in that, In S4, the feature transformation network includes two linear layers and one activation layer.

6. The small sample anomaly detection method based on feature storage according to claim 5, characterized in that, The loss function of the feature transformation network is: Where F represents the feature transformation network, The nearest distance set d j The feature in the feature repository corresponding to the minimum value in the middle. The nearest distance set d j Output the sample image corresponding to the minimum value.

7. The small sample anomaly detection method based on feature storage according to claim 6, characterized in that, In S5, the expression for the abnormal hotspot graph P is: P=|Q i -F(m i )| 2 ,i∈R h*w Where F represents the feature transformation network, Q i Patch p for R of the test set sample images i The output features, m i For the feature repository with Q i The feature that is closest to you.

8. The small sample anomaly detection method based on feature storage according to claim 7, characterized in that, In S5, after constructing the abnormal heat map, Gaussian filtering is applied to smooth the abnormal heat map, and the largest loss function value in the smoothed abnormal heat map is selected as the abnormal score of the sample image.

9. A small-sample anomaly detection device based on feature storage, characterized in that, include: The sampling module is used to acquire a small sample unsupervised anomaly detection dataset, which is divided into a training set and a test set. The feature repository building module is used to input each patch of all sample images in the training set into the feature extractor to obtain the output feature set of each patch of all sample images in the training set and store it in the feature repository. The feature network construction module is used to perform data augmentation on all sample images in the training set. Each patch of the data-augmented image is input into the feature extractor to obtain the output feature set of each patch of the data-augmented image. The distance between each output feature in the output feature set of each patch and all features in the feature repository is calculated. The minimum value of the distance is selected to construct the nearest distance set of the patch. A feature transformation network is created. The features in the feature repository corresponding to the minimum value in the nearest distance set of each patch are input into the feature transformation network to obtain the mapped features of each patch. The loss function value of the mapped features of each patch and the output features of the data-augmented image corresponding to the minimum value in the nearest distance set of that patch is calculated. The parameters of the feature transformation network are optimized based on the loss function value to minimize the loss function value. The threshold setting module is used to extract the features of each patch of the sample image in the test set using the feature extractor, and to find the feature closest to the feature of each patch of the sample image in the feature repository. The optimized feature transformation network is then used to obtain the mapped features of each patch of the sample image. Calculate the loss function value between the mapping features of each patch and the output features of that patch to form an anomaly hotspot map, and select the largest loss function value in the anomaly hotspot map as the anomaly score of the sample image; compare the anomaly scores of normal images and anomaly images in the test set, and set the segmentation thresholds for normal images and anomaly images. The detection module is used to calculate the anomaly score of the image to be detected. If the anomaly score is greater than the segmentation threshold, the image to be detected is an abnormal image. If the anomaly score is less than or equal to the segmentation threshold, the image to be detected is a normal image.

10. A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the feature-stored small sample anomaly detection method as described in any one of claims 1 to 8.

Images