Abnormity detection method based on synthetic texture fusion anomaly and self-attention mechanism
By synthesizing the texture fusion abnormality and self-attention mechanism, the problem of random sources of abnormalities and inaccurate positions in image abnormality detection is solved, efficient and accurate context abnormality detection is achieved, and high-precision detection is achieved.
Patent Information
- Application Number
- CN202510563424.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-30
- Publication Date
- 2025-08-15
AI Technical Summary
In the existing image anomaly detection methods, the abnormality source is random, it is very different from the real industrial anomaly, lacks authenticity, and the abnormal target location may be inaccurate, making it difficult to effectively detect context abnormalities.
The synthetic texture fusion exception method and self-attention mechanism are adopted to construct the synthetic texture fusion exception method TA, and the self-attention module SSA is introduced to the middle layer feature map of the convolutional neural network, and anomaly detection is performed in combination with the coreset downsampling algorithm.
It realizes efficient detection of context abnormalities in industrial images, reduces the computational complexity, improves detection accuracy, and reaches 99.13% image level AUC and 97.02% pixel level AUC.
Smart Images
Figure CN120495657A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to an anomaly detection method, and in particular to an anomaly detection method based on synthetic texture fusion anomaly and self-attention mechanism. Background Art
[0002] Image anomaly detection (IAD) identifies abnormal patterns that differ from normal instances. It has applications across multiple industries, but faces challenges such as the difficulty in acquiring anomaly data and the subtlety of anomaly features. Anomalies can be categorized as point anomalies, contextual anomalies, and group anomalies. Contextual anomalies are common in IAD, which is related to the self-attention mechanism. While self-attention is effective for image anomaly detection, its computational complexity limits its applications.
[0003] The MVTec Anomaly Detection (MVTec-AD) dataset is a benchmark for deep learning anomaly detection techniques. It contains 15 categories of normal and abnormal images. It is used to test different anomaly detection methods, but currently these methods are rarely optimized for contextual anomalies.
[0004] We focus on algorithms that process normal images and simulate abnormalities in industrial production, similar to the CutPaste method. However, CutPaste may not work well for certain categories because it may simulate abnormalities in the background area rather than the object itself.
[0005] Due to the limited amount of anomaly data and the high cost of collecting it, research on industrial visual anomaly detection has primarily focused on unsupervised methods, where the training set contains only normal samples and the test set includes both normal and abnormal samples. Before deep learning, difference detection and filtering were used to detect anomalies in industrial images. In recent years, deep learning-based anomaly detection methods have been divided into two categories: reconstruction-based methods and representation-based methods. Summary of the Invention
[0006] The purpose of the present invention is to provide an anomaly detection method based on synthetic texture fusion anomalies and self-attention mechanism to solve the problems in existing image anomaly detection methods, such as the random sources of anomalies, large differences from real industrial anomalies, and lack of authenticity; and the problem that the abnormal target position may be inaccurate, which violates the original intention of simulating the detailed anomalies of objects.
[0007] To achieve the above objectives, the specific plan is as follows:
[0008] An anomaly detection method based on synthetic texture fusion anomaly and self-attention mechanism, comprising the steps of:
[0009] S1: Constructing the synthetic texture fusion anomaly method TA and the self-attention module SSA;
[0010] S2: Use TA to train a convolutional neural network for binary classification;
[0011] S3: Introduce SSA into the intermediate layer feature map of the binary classification convolutional neural network pre-trained in step S2;
[0012] S4: After TA synthesis is completed, the pre-trained Resnet-18 is used to extract the features of image A;
[0013] S5: Use coreset downsampling algorithm to reduce the feature map size;
[0014] S6: After the feature blocks described in the test phase are searched by the KNN function, the point in the feature database with the closest distance to each feature block is obtained. This distance is used as the outlier value of each feature block. The maximum value of these feature block outliers is the image-level outlier.
[0015] Furthermore, in step S1, constructing a synthetic texture fusion anomaly method TA includes the following steps:
[0016] S1a: Input texture image T and normal image N in the dataset; use GrabCut segmentation method to extract
[0017] The object area in image N is obtained as image O containing only the object;
[0018] S2a: Randomly crop the image O to obtain the area to be replaced dst;
[0019] S3a: Randomly crop the texture image T to obtain the replacement area src;
[0020] S4a: Synthetic texture fusion anomaly, the synthesis formula is: dst = αdst + (1-α)src;
[0021] S5a: Replace the synthesized texture fusion anomaly into the corresponding area in the normal image N to obtain image A.
[0022] Furthermore, the objective loss function of the self-supervised learning method is:
[0023] L TA =CE x∈X {CE(f(x),0)+CE(f(TA(x)),1)}.
[0024] Furthermore, in step S1, the self-attention module SSA is composed of K, Q, and V.
[0025] Furthermore, in step S4, three layers of feature maps are included.
[0026] Furthermore, the characteristics of each patch that aggregates local features are:
[0027] F2=Ψ agg [i(w2,h2,c2)].
[0028] Furthermore, in step S4, the expression for obtaining the third layer features as contextual anomaly detection is:
[0029] F3=φ SSA [i * (w3,h3,c3)].
[0030] Furthermore, the self-attention module SSA is applied to the feature map obtained from the third layer of ResNet18 and used as a conversion module to derive the feature map I SSA , I SSA The expression is:
[0031]
[0032] Furthermore, in step S6, the formula used is:
[0033] p * =argmin p∈M ||p test -p||2
[0034] S A =||p test -p||2.
[0035] Furthermore, if the outlier value is greater than 5 times the average outlier value of the feature block in S6, a segmentation operation is performed based on the feature block outlier value to perform bilinear interpolation to obtain an outlier distribution image with the same size as the original image. The outlier distribution image is subjected to domain value processing to obtain a mask image of the segmented outlier.
[0036] In summary, the present invention has the following beneficial effects compared to the prior art:
[0037] (1) This paper proposes a method for synthesizing texture-fused anomalies (TA) by performing a weighted fusion of normal target regions and textures to simulate anomaly samples that are consistent with the target environment. A convolutional neural network for binary classification is trained using the synthesized TA to qualitatively perform anomaly detection.
[0038] (2) This paper designs a simple self-attention (SSA) module and introduces it into the output of the intermediate layer feature map of a pre-trained binary classification convolutional neural network. SSA can calculate the relationship between features without using the linear transformation in traditional self-attention models. This not only takes advantage of the unique advantages of the self-attention module to better detect contextual anomalies in industrial images, but also avoids the high computational complexity of the conventional self-attention mechanism to a certain extent. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:
[0040] Figure 1 Flowchart for synthetic texture fusion anomaly;
[0041] Figure 2 Describe the graph for the testing phase;
[0042] Figure 3 is the self-attention module;
[0043] Figure 4 Visualization results of the model on the MVTec-AD dataset. DETAILED DESCRIPTION
[0044] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments of the present invention can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.
[0045] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form can also include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.
[0046] Unless otherwise specifically stated, the relative arrangement of the parts and steps, numerical expressions and numerical values set forth in these embodiments do not limit the scope of the present invention. At the same time, it should be understood that, for ease of description, the sizes of the various parts shown in the drawings are not drawn according to actual proportional relationships. The techniques, methods and equipment known to those of ordinary skill in the relevant art may not be discussed in detail, but where appropriate, the techniques, methods and equipment should be considered as part of the authorized specification. In all examples shown and discussed herein, any specific values should be interpreted as being merely exemplary and not as limitations. Therefore, other examples of the exemplary embodiments may have different values. It should be noted that similar numbers and letters represent similar items in the following figures, and therefore, once an item is defined in one figure, it does not need to be further discussed in subsequent figures.
[0047] See also Figures 1 to 3 As shown, the present invention provides an anomaly detection method based on synthetic texture fusion anomaly and self-attention mechanism, comprising the steps of:
[0048] S1: Constructing the synthetic texture fusion anomaly method TA and the self-attention module SSA;
[0049] As a preferred embodiment, in step S1, constructing a synthetic texture fusion anomaly method TA includes the following steps:
[0050] S1a: Input texture image T and normal image N in the dataset; extract object area in image N, and get
[0051] To an image O containing only objects;
[0052] S2a: Randomly crop the image O to obtain the area to be replaced dst;
[0053] S3a: Randomly crop the texture image T to obtain the replacement area src;
[0054] S4a: Synthetic texture fusion anomaly, the synthesis formula is: dst = αdst + (1-α)src;
[0055] S5a: Replace the synthesized texture fusion anomaly into the corresponding area in the normal image N to obtain image A.
[0056] The objective loss function of the self-supervised learning method proposed in this invention (1)
[0057] L TA =CE x∈X {CE(f(x),0)+CE(f(TA(x)),1)} (1)
[0058] CE stands for cross entropy loss, which includes two parts: normal and abnormal. X is the normal dataset, TA is the abnormal synthesis method, and f is the trained binary classifier.
[0059] As a preferred embodiment, in step S1, the self-attention module SSA is composed of K (key vector), Q (query vector), and V (value vector). The initial K, Q, and V are formed by performing a maximum pooling operation on the feature map of the third layer in the pre-trained convolutional neural network. Then, the attention score is obtained by the dot product operation of Q and K, and then normalized by softmax. The result is matrix multiplied with V and finally reconstructed into a feature map of the same size as the input to the module SSA.
[0060] S2: Use TA to train a convolutional neural network for binary classification.
[0061] S3: Introduce SSA into the intermediate layer feature map of the binary classification convolutional neural network pre-trained in step S2.
[0062] SSA computes the relationships between features without using the linear transformations found in traditional self-attention models. This leverages the unique advantages of the self-attention module to better detect contextual anomalies in industrial images while also avoiding the high computational complexity of conventional self-attention mechanisms.
[0063] S4: After TA is synthesized, the pre-trained convolutional neural network is used to extract the features of image A.
[0064] The present invention uses a convolutional neural network that has been pre-trained after synthesizing TA to extract the features of the input image. Generally, the output of each convolutional network structure can obtain a feature map. The deeper the feature map, the more global the feature information can be captured, while the shallower the feature map, the more local the feature information can be captured. They are both very meaningful for the task of anomaly detection. In order to take both into account, the present invention selects the feature map of the middle layer of the above-mentioned pre-trained convolutional neural network, namely the second and third layers. The second layer has more local feature information than the third layer, which is very helpful for detecting local anomalies. Each feature block that aggregates local features can be represented by the following expression:
[0065] F2=Ψ agg [i(w2,h2,c2)] (2)
[0066] Ψ agg is an aggregation function used to integrate local features. The size of the second-layer feature map i is w2, h2, c2, and TASA is processed by 3x3 average pooling. Since the third-layer feature map contains more global information, we use it as the input of the simple self-attention mechanism module to effectively detect contextual anomalies. The expression (3) used to obtain the third-layer features as contextual anomaly detection is
[0067] F3=φ SSA [i * (w3,h3,c3)] (3)
[0068] φ SSA is a simple self-attention module for transforming features into features that contain the information needed for contextual anomalies. * It is the third layer feature map with the size of width w3, length h3 and number of channels c3. The detected local features F2 and global features F3 will be merged and stored in the feature database D.
[0069] Specifically, the present invention introduces a simple self-attention module (such as Figure 3 This module is used to obtain the feature map from the third layer of the pre-trained convolutional network and serves as a conversion module to derive the feature map I SAA, which contains the information needed to detect contextual anomalies. Once the feature map i*(w3,h3,c3) of the third layer is obtained, maximum pooling with a kernel size of 3, a stride of 1, and a padding of 1 is applied to strengthen nearby features. These features are then converted to a vector X∈Rh3×w3×c3. In Transfomer, the queries, keys, and values used to calculate self-attention are obtained by performing different linear transformations on the input matrix. For example, I is multiplied by three trainable parameter matrices W_Q, W_K, and W_V to generate Q, K, and V, respectively. However, in order to reduce computational complexity, I in SSA is copied into three copies after a maximum pooling, and directly used as the so-called query, key, and value to calculate self-attention. I SAA It is expressed as follows:
[0070]
[0071] d I Represents the dimension of I. The relationship between pixels is calculated by multiplying the query and key as weights, and combining these weights with the product of the SoftMax and value to form a vector I SAA , reflecting the association of different distance features. Finally, adjust X SAA to i * size, and obtain the feature map F3 containing contextual anomaly detection.
[0072] S5: In order to reduce the prediction time of the algorithm caused by the longer feature map size, the present invention adopts the coreset downsampling algorithm to reduce the feature map size. In general, the selection of the core set aims to find a subset This allows the solution to the problem on A to be obtained through the solution calculated on S, and in particular, to be approximated faster. In the hierarchical feature space of the feature block, we filter out the core set Dc from the feature database D so that its coverage is roughly the same as that of the original feature database D.
[0073] S6: As Figure 2 As shown in the figure, after the feature blocks described in the test phase are searched by the KNN function, the point in the feature database closest to each feature block is obtained. The distance is used as the outlier value of each feature block, and the maximum value of these feature block outliers is the image-level outlier.
[0074] p * =argmin p∈M ||p test -p||2 (5)
[0075] S A =||p test -p||2 (6)
[0076] If the outlier value exceeds a certain range (the outlier value is greater than 5 times the average outlier value of the feature block in S6), it means that there is indeed an anomaly in the image. Then, through the segmentation operation, bilinear interpolation is performed based on the outlier value of the feature block to obtain an anomaly distribution image with the same size as the original image. The domain value processing is performed on the anomaly distribution image to obtain a mask image of the segmented anomaly.
[0077] Figure 4 Visualizations of the proposed model on the MVTec-AD dataset are presented. As shown, the detection algorithm exhibits a significant, concentrated response (yellow-green) around true anomaly regions, while the response in normal regions is very weak. These visualizations demonstrate the proposed TASA's ability to segment anomalous regions across various object categories on the MVTec-AD dataset, providing supporting evidence for the model's effectiveness in anomaly detection tasks.
[0078] The method of the present invention achieved an AUC of 99.13% at the image level and an AUC of 97.02% at the pixel level on the MVTec-AD dataset.
[0079] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.
Claims
1. An anomaly detection method based on synthetic texture fusion anomaly and self-attention mechanism, characterized in that: Including steps: S1: Constructing the synthetic texture fusion anomaly method TA and the self-attention module SSA; S2: Use TA to train a convolutional neural network for binary classification; S3: Introduce SSA into the intermediate layer feature map of the binary classification convolutional neural network pre-trained in step S2; S4: After TA synthesis is completed, the pre-trained Resnet-18 is used to extract the features of image A; S5: Use coreset downsampling algorithm to reduce the feature map size; S6: After the feature blocks described in the test phase are searched by the KNN function, the point in the feature database with the closest distance to each feature block is obtained. This distance is used as the outlier value of each feature block. The maximum value of these feature block outliers is the image-level outlier.
2. The anomaly detection method based on synthetic texture fusion anomaly and self-attention mechanism according to claim 1 is characterized in that: In step S1, constructing a synthetic texture fusion anomaly method TA includes the following steps: S1a: Input texture image T and normal image N in the dataset; use GrabCut segmentation method to extract the object area in image N and obtain image O containing only the object; S2a: Randomly crop the image O to obtain the area to be replaced dst; S3a: Randomly crop the texture image T to obtain the replacement area src; S4a: Synthetic texture fusion anomaly, the synthesis formula is: dst = αdst + (1-α)src; S5a: Replace the synthesized texture fusion anomaly into the corresponding area in the normal image N to obtain image A.
3. The anomaly detection method based on synthetic texture fusion anomaly and self-attention mechanism according to claim 2 is characterized in that: The objective loss function of the self-supervised learning method is: IT TA =EC x∈X {CE(f(x),0)+CE(f(TA(x)),1)}.
4. The anomaly detection method based on synthetic texture fusion anomaly and self-attention mechanism according to claim 1 is characterized in that: In step S1, the self-attention module SSA is composed of K, Q, and V.
5. The anomaly detection method based on synthetic texture fusion anomaly and self-attention mechanism according to claim 1 is characterized in that: In step S4, three layers of feature maps are included.
6. The anomaly detection method based on synthetic texture fusion anomaly and self-attention mechanism according to claim 5 is characterized in that: The characteristics of each patch that aggregates local features are: F2=Ψ agg [i(w2,h2,c2)]。 7. The anomaly detection method based on synthetic texture fusion anomaly and self-attention mechanism according to claim 5 is characterized in that: In step S4, the expression used to obtain the third layer features as contextual anomaly detection is: F3=φ SSA [i * (w3,h3,c3)]。 8. The anomaly detection method based on synthetic texture fusion anomaly and self-attention mechanism according to claim 5 is characterized in that: The self-attention module SSA is applied to the feature map obtained from the third layer of ResNet18 and used as a conversion module to derive the feature map I SSA , I SSA The expression is:
9. The anomaly detection method based on synthetic texture fusion anomaly and self-attention mechanism according to claim 1 is characterized in that: In step S6, the formula used is: p * =argmin p∈M ||p test -p||2 S A =||p test -p||2。 10. The anomaly detection method based on synthetic texture fusion anomaly and self-attention mechanism according to claim 9 is characterized in that: If the outlier value is greater than 5 times the average outlier value of the feature block in S6, a segmentation operation is performed based on the feature block outlier value, and bilinear interpolation is performed to obtain an outlier distribution image with the same size as the original image. The outlier distribution image is subjected to domain value processing to obtain a mask image of the segmented outlier.