Unsupervised Industrial Anomaly Detection and Localization Method Based on Self-Organizing Maps

Abstract

This invention provides an unsupervised industrial anomaly detection and localization method based on self-organizing map networks. During feature extraction, features from different layers are treated as different views to reduce mutual interference between information at different semantic levels. Each image is treated as multiple image blocks, and the distribution information of these image blocks is learned using self-organizing map networks. Then, an anomaly score is given for each image block based on the distribution information. Finally, a threshold for determining anomalies is set to achieve anomaly detection.

Description

Technical Field

[0001] This invention belongs to the field of machine learning, specifically relating to an unsupervised method for industrial anomaly detection and localization based on self-organizing map networks. Background Technology

[0002] Anomaly detection refers to the detection of samples that do not conform to expected patterns or normal distributions. It plays a crucial role in quality control in industrial production, but supervised anomaly detection is difficult to perform in real-world production environments. On the one hand, defective samples are far fewer than normal samples; on the other hand, certain categories of image defects are unavailable, meaning it cannot be guaranteed that all defect types encountered during testing will be encountered during training. Therefore, anomaly detection models are typically implemented using single-class learning, meaning the model can only access normal samples during training and not defective samples, while identifying anomalies during inference.

[0003] In recent years, feature embedding similarity-based methods have achieved great success in industrial anomaly detection scenarios. These methods require deep neural networks to extract meaningful information from images and generate feature embeddings. For anomaly detection tasks, this method describes the entire image as a feature vector; for anomaly localization tasks, it often treats the entire image as multiple small image patches, and then describes each small image patch as a feature vector. In this case, anomaly detection is often achieved by comparing the distance between the feature vector of the test image and the reference vector of the corresponding normal image. The reference vector can be composed of all normal embedding vectors or selected from a subset of representative vectors through certain methods. The former approach is very effective, but because it requires storing all normal embedding vectors, it requires a huge amount of storage space, and due to the large amount of information stored, the testing time is also long, thus greatly limiting the industrial deployment of this method. The latter approach, while significantly reducing the size of the reference vector, has two drawbacks: firstly, the method for selecting representative vectors is too complex, forcing the use of a greedy strategy that cannot guarantee the quality of the reference vectors; secondly, a lot of information is lost in the selection process. Furthermore, since shallow feature maps in deep networks focus more on detailed information while deep feature maps focus more on contextual information—both of which are crucial for anomaly detection—these methods often combine feature maps from multiple layers to improve detection accuracy. However, these methods use multi-level features in an overly simplistic way, merely upsampling deep feature maps to the same size as shallow ones and then concatenating multiple feature maps along the channel direction. This leads to interference between feature maps from different layers. Especially since deep feature maps are already high-dimensional, upsampling results in many identical values, which severely negatively impacts the diversity of shallow features.

[0004] Self-Organizing Maps (SOMs) are a classic unsupervised learning method proposed by Kohonen. The main objective of SOMs is to represent the distribution of input data items in a low-dimensional space (typically two-dimensional) using a finite number of neurons, while preserving the topological structure in the process. It involves organizing data in a two-dimensional grid. There are M*N neuron nodes in the array, where c i Let represent the weight vector of neuron i. During training, all data in the training set are input into the SOM, and the best-matching unit is found through competition. The weight matrix then needs to be updated. To maintain the neighborhood relationships of input vectors in the topological space, not only the weights of the best-matching unit are updated, but also the weights of its neighboring nodes. When the model converges, each input vector is assigned to a neuron node, exhibiting topological preservation—similar vectors are grouped closer together on the grid. This neighborhood learning property makes it extremely suitable for processing image patch data.

[0005] There are two problems in industrial anomaly detection scenarios. First, in order to comprehensively consider information at different semantic levels, various algorithms combine feature maps from different layers, but the operation is too rudimentary and ignores the mutual interference between information at different levels. Second, in order to improve the efficiency of the testing phase, attempts are made to reduce the size of the reference vector, but it is difficult to retain as much information as possible while reducing the size of the reference vector. Summary of the Invention

[0006] Purpose of the invention: The technical problem to be solved by the present invention is to provide an unsupervised industrial anomaly detection and localization method based on self-organizing mapping networks, which addresses the shortcomings of the existing technology.

[0007] This invention treats each image as multiple image patches and employs a Wide-Resnet50x2 model pre-trained on the ImageNet dataset as a feature extractor. During feature extraction, the feature maps extracted by the first three layers of the model are selected as the image's feature vectors, and each layer's feature map is treated as an independent view to reduce mutual interference. Furthermore, to reduce the size of the reference vector during testing, a Self-Organizing Map (SOM) network is used to organize the image patch feature embeddings. Since SOM is a simple and effective online learning method, its neighborhood learning mechanism can utilize the location information of image patches, making it well-suited for processing image patches. After SOM training, each training image patch has a corresponding neuron node, thus storing a large number of training image patch feature vectors with a limited number of neurons, significantly reducing the algorithm's cost. Finally, the distribution of image patches is modeled by calculating the mean and variance of all image patch feature vectors projected onto each node in the SOM, thereby better preserving the information contained in the training image patches. The proposed method has been validated on 15 datasets of MVTec AD, achieving state-of-the-art performance in unsupervised anomaly detection and localization tasks.

[0008] The specific implementation process of this invention is as follows:

[0009] (1) Divide the training image into image patches and use a pre-trained model as a feature extractor to extract features, treating the feature maps extracted by the first three layers of the model as different views. Thus, for a training image x, each image patch and three feature vectors are represented. Related, where i, j represent the positions in the image, and t represents the view to which the vector belongs;

[0010] (2) Use three self-organizing map networks to process the feature vectors of the three views respectively, project all the feature vectors of the training images onto the nodes, and calculate the mean μ and variance Σ of all feature vectors on each node to construct the distribution corresponding to the normal image patch;

[0011] (3) For the three feature vectors corresponding to each image block in the training image, find the corresponding winning neuron node in the corresponding SOM and calculate the Mahalanobis distance. Finally, combine the Mahalanobis distances of different views to obtain the final anomaly score.

[0012] (4) Use the obtained abnormal scores to set the abnormal score threshold θ;

[0013] (5) Divide the test object x into blocks and extract the feature vector of each block in different views. Calculate the Mahalanobis distance of each block in different views and obtain the corresponding anomaly score. Compare it with the anomaly score threshold. If it is less than the threshold, it is considered normal; otherwise, it is considered abnormal.

[0014] Beneficial effects: This invention reduces interference between views by treating feature maps extracted from different layers of the model as different views and processing them separately; it significantly reduces the size of feature vectors and lowers algorithm costs by projecting the extracted image patch feature vectors onto a self-organizing map network; and it enables efficient image alignment by processing images in blocks, meaning similar image patches are projected onto the same neurons, resulting in more accurate calculated distributions. All three measures improve performance. Attached Figure Description

[0015] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments, and the advantages of the present invention in the above and / or other aspects will become clearer.

[0016] Figure 1 This is a flowchart of an industrial anomaly detection and localization algorithm based on self-organizing mapping networks.

[0017] Figure 2 This is a schematic diagram illustrating the specific training process of an industrial anomaly detection and localization algorithm based on self-organizing mapping networks.

[0018] Figure 3 This is a schematic diagram of some anomaly detection and localization results. Detailed Implementation

[0019] In this specific implementation, the experimental data all come from the MVTec AD dataset obtained from real industrial production scenarios. There are five texture class datasets, including carpet, grid, leather, tile, and wood; and ten object class datasets, including bottle, cable, capsule, hazelnut, metal nut, pill, screw, toothbrush, transistor, and zipper.

[0020] Figure 1 The flowchart of the industrial anomaly detection and localization algorithm based on self-organizing map networks mentioned in this invention is shown, and it specifically includes the following steps:

[0021] Step 1: Divide the image into blocks and extract features to obtain feature vectors for multiple views.

[0022] Since anomalous regions in images often appear in small areas, dividing the image into smaller blocks makes the extracted features more representative. Furthermore, image segmentation is well-suited for anomaly localization tasks, as identifying the presence of anomalies in each block naturally yields the anomaly localization result. Finally, image segmentation also aligns similar regions in the image, projecting them onto the same nodes, resulting in more reasonable anomaly scores. In summary, the first step is to consider the image as composed of multiple image blocks.

[0023] Deep learning models, once trained, possess extremely powerful feature extraction capabilities. The larger the training dataset and the more categories the model covers, the more representative and robust the extracted features become. In industrial anomaly detection scenarios, target data are often of the same type with extremely high similarity, making it difficult to train high-performance models as feature extractors. However, the ImageNet dataset, with its large volume of data and diverse categories, has proven effective in various scenarios using models trained on it. Therefore, a Wide-ResNet50x2 model pre-trained on the ImageNet dataset is used as the feature extractor.

[0024] Specific operations are as follows: Figure 2 As shown, input is the training image example, ResNet box represents the residual network model, where layer1, layer2, layer3 and layer4 are the four main network layers of the residual network model, and Feature Map is the feature map.

[0025] The feature maps extracted from the first three layers of the ResNet model are selected as the feature vectors of the original image under different views. Since the size of the feature maps decreases as the network deepens, upsampling is used to enlarge the deep feature maps to a size comparable to the shallower ones. Figure 1 The above process can be formally expressed as follows: For an image x∈X in a dataset X, consider it as a W×H image patch. Then, for each image patch x containing position i... i The corresponding eigenvector is f i t , where t∈[1,3], i∈[1,W×H].

[0026] Step 2: Use three SOMs to process the data from different views, and then construct the distribution corresponding to the image patches.

[0027] Three self-organizing map networks are used to process feature embeddings for different views separately, in order to view Figure 1 For example, SOM 1 The training process includes four steps: initialization, random sampling, determining the winning neuron, and parameter updating. First, SOM...1 The grid size is set to match the number of image patches, i.e., length W and width H. The weight vector is initialized by calculating the mean of the feature vectors of all image patches corresponding to each position. Then, a feature vector f is randomly selected from the feature set each time. 1 Next, the winning neuron is determined according to the formula below.

[0028]

[0029] Where i is the subscript of the neuron, Let i be the weight vector of the neuron with index i.

[0030] Finally, update the weight vectors of the winning neuron and its neighboring neurons according to the formula below.

[0031]

[0032] Where α(n) is the learning rate at the nth update, h j,i(f) (n) is the neighborhood function of the winning neuron i(f).

[0033] After SOM training, each neuron node has a corresponding weight vector. The goal is for the SOM network to characterize the distribution of normal feature vectors as accurately as possible. This invention employs a method that calculates the mean and variance of all data projected onto each node. In this way, all feature vectors corresponding to each node can be characterized by a multivariate Gaussian distribution, thereby enabling the construction of the distribution corresponding to normal image patches using the SOM network. The variance calculation formula is as follows:

[0034]

[0035] Where N i f is the number of input vectors projected onto node i. i j Let represent the j-th input vector of node i. The regularization term ∈ I makes the covariance matrix Σ i Fully rank and reversible.

[0036] Step 3: Calculate the anomaly score for the image patch.

[0037] At this point, we can obtain three trained self-organizing map networks (SOMs), denoted as SOM. 1 SOM 2 SOM 3 For the i-th image patch x i There are three corresponding eigenvectors f i t t∈[1,3]. Then for image patch x i Feature embedding f it It needs to be in SOM t Find the k nearest nodes and calculate the Mahalanobis distance between the feature embedding and these nodes. Finally, select the minimum distance as the final Mahalanobis distance score of the image patch in this view, as shown in the formula below:

[0038]

[0039] Among them, f i t This represents the feature vector corresponding to the image patch t at position i. and Let represent the mean and covariance matrix of the distribution corresponding to the winning neuron node in the SOM view t, respectively.

[0040] After calculating the Mahalanobis distance maps corresponding to the three views, image patch x i abnormal score M i The sum of the Mahalanobis distances of the three views is given:

[0041]

[0042] Finally, bilinear interpolation is used to enlarge the anomaly score map to the same size as the original image, thereby obtaining the anomaly score for each pixel and achieving anomaly localization. If anomaly pixels are present in the image, it is considered an anomaly image, achieving anomaly detection. It is worth noting that the final score can be obtained by weighting the scores of the three views. Adjusting the weights of the scores in different views under different scenes can yield better results, but in the experiment, all weights were set to 1.

[0043] Step 4: Set the anomaly score threshold

[0044] An anomaly score threshold θ is set according to a certain strategy, such as introducing a normal sample rejection ratio r to adjust the anomaly score threshold. The anomaly scores of the training samples are sorted from smallest to largest to obtain u. r Then select u r The Num×r+1th element is used as the anomaly score threshold θ.

[0045] Step 5: Testing Phase

[0046] The test object x is divided into blocks, and the feature vector of each block is extracted in different views. The Mahalanobis distance of each block in different views is calculated, and the corresponding anomaly score s is obtained. i Finally, bilinear interpolation was used to enlarge the anomaly score map to the same size as the original image, obtaining the x-position of each pixel. pixel abnormal scores s pixel And compare it with the anomaly score threshold, for each pixel x pixel The discriminant function is:

[0047]

[0048] In this way, we can determine whether each pixel location is abnormal. If there is an abnormal pixel, the sample is an abnormal sample and the area where the abnormal pixel is located has achieved abnormal positioning; if there is no abnormal pixel, the sample is a normal sample.

[0049] To verify the effectiveness of this invention, experimental analysis was conducted in conjunction with its implementation scheme. The analysis primarily compared it with two of the most commonly used methods: one based on reconstruction principles, including AE+SSIM and VAE; and the other based on feature embedding similarity, including SPADE (2020), PaDiM (2021), and PatchCore (2022).

[0050] To ensure a fairer comparison with the aforementioned feature similarity-based methods, the data preprocessing in this embodiment remains consistent with theirs. First, all methods use a Wide-Resnet50x2 model pre-trained on the ImageNet dataset as their feature extractor. Then, all images are resized to 256×256 pixels and cropped to 224×224 pixels. Finally, after obtaining pixel-level anomaly scores using bilinear interpolation, a Gaussian filter with parameter σ=4 is applied to smooth the results. The experimental results of this invention and several comparative methods are shown in Table 1. The visualization of some detection results from this invention is shown below. Figure 3 As shown.

[0051] Table 1 compares the proposed method with several other state-of-the-art industrial anomaly detection and localization methods on the MVTec AD dataset. Results are presented as tuples (image-level AUROC%, pixel-level AUROC%).

[0052] Table 1

[0053]

[0054] The accuracy results of the comparative experiments show that the method proposed in this invention can effectively handle the problem of anomaly detection and localization in industrial scenarios.

[0055] In its specific implementation, this application provides a computer storage medium and a corresponding data processing unit. The computer storage medium is capable of storing a computer program, which, when executed by the data processing unit, can run the invention's content regarding the unsupervised industrial anomaly detection and localization method based on self-organizing mapping networks, as well as some or all of the steps in various embodiments. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0056] Those skilled in the art will clearly understand that the technical solutions in the embodiments of the present invention can be implemented using computer programs and their corresponding general-purpose hardware platforms. Based on this understanding, the technical solutions in the embodiments of the present invention, or the parts that contribute to the prior art, can be embodied in the form of computer programs, i.e., software products. These computer program software products can be stored in a storage medium and include several instructions to cause a device containing a data processing unit (which may be a personal computer, server, microcontroller, MUU, or network device, etc.) to execute the methods described in various embodiments or certain parts of the embodiments of the present invention.

[0057] This invention provides an unsupervised industrial anomaly detection and localization method based on self-organizing mapping networks. Many methods and approaches exist for implementing this technical solution; the above description is merely a preferred embodiment of the invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications should also be considered within the scope of protection of this invention. All components not explicitly stated in this embodiment can be implemented using existing technologies.

Claims

1. An unsupervised industrial anomaly detection and localization method based on self-organizing map networks, characterized in that, Includes the following steps: Step 1: Divide the training image into n*n image blocks, n∈[2,size], where size is the pixel-level size of the training image, and perform feature extraction. Each image block yields feature vectors for three views. Step 2: Use three self-organizing map networks (SOMs) to process the feature vectors of the three views respectively, project the feature vectors of all training image patches onto the nodes, and calculate the mean μ and variance Σ of all feature vectors on each node to construct the distribution corresponding to the normal image patch. Step 3: Calculate the anomaly score for the image patch; Step 4, set the abnormal score threshold θ; Step 5: Detect and locate industrial anomalies; Step 2 includes: Step 2-1: Three self-organizing map networks are used to process the image patch feature vectors of the three views respectively. The image patch feature vectors obtained in Step 1 are used as the training set, and one feature vector f is extracted from the training set each time. t Then, the winning neuron is determined using the following formula: Among them, i t f is the subscript of the neuron. t The input feature vector, Let be the weight vector of the i-th neuron; Then update the weight vectors of the neuron and its neighboring neurons, as shown in the following formula: in Let α(n) be the weight vector of the j-th neuron during the (n+1)-th update, and h be the learning rate during the n-th update. j,i(f) (n) is the neighborhood function of the winning neuron i(f), f t (n) is the feature vector input during the nth update; Step 2-2: Repeat step 2-1 until the weight vectors of the three self-organizing map networks no longer change. Training is complete, and three trained self-organizing map networks (SOMs) are obtained, denoted as SOMs respectively. 1 SOM 2 SOM 3 ; Steps 2-3: Calculate the mean and variance of all data projected to each node, thus constructing the distribution corresponding to normal image patches using a Self-Organizing Map (SOM) network. The variance calculation formula is as follows: Where N i c is the number of input vectors projected onto node i. i Let f represent the weight vector of the i-th neuron. i j Let I represent the j-th input vector of neuron i; the regularization term ∈ I makes the covariance matrix Σ i Full rank and reversible; In step 3, for the loc-th image patch x i For loc∈[1,n*n], there are three corresponding eigenvectors. If t∈[1,3], then for image patch x i Feature embedding Need to be in SOM t Find the k nearest nodes and calculate the Mahalanobis distance between the feature embedding and the k nearest nodes. Finally, select the minimum distance as the final Mahalanobis distance score of the image patch in the current view, as shown in the formula below: in f represents the Mahalanobis distance corresponding to the loc-th image patch t in view . i t This represents the feature vector corresponding to the loc-th image patch t in view. and Let represent the mean and covariance matrices of the distribution corresponding to the winning neurons in the t-view self-organizing map network (SOM), respectively.

2. The method according to claim 1, characterized in that, Step 1 includes: acquiring the MVTec AD dataset from a real industrial production scenario. The MVTec AD dataset includes a texture dataset and an object dataset. Images from the texture dataset and the object dataset are used as training images. The training images are divided into n*n image patches. A Wide-Resnet50x2 model pre-trained on the ImageNet dataset is used as a feature extractor to extract features. The feature maps extracted by the first three layers of the model are regarded as different views. The size of the extracted feature maps is upsampled to be consistent with the number of image patches, so that each image patch has a corresponding feature vector.

3. The method according to claim 2, characterized in that, In step 1, for a training image x, each image patch at its location is associated with three feature vectors. Related, where i,j∈[1,n], t∈[1,3], Let represent the feature vector at position i (horizontal coordinate i) and position j (vertical coordinate j) in the image under the t-th view.

4. The method according to claim 3, characterized in that, In step 3, after calculating the Mahalanobis distance diagrams corresponding to the three views, the image patch x... i abnormal score M i The sum of the Mahalanobis distances of the three views is given:

5. The method according to claim 4, characterized in that, In step 4, the anomaly scores of the image patches are sorted from smallest to largest to obtain sequence u. r Then select sequence u r The Num×r+1th element is used as the anomaly score threshold θ, where Num is the total number of all feature vectors and r is a decimal between 0 and 1.

6. The method according to claim 5, characterized in that, In step 5, the test image x is divided into blocks, and the feature vector of each block is extracted in different views. The Mahalanobis distance of each block in different views is calculated, and the corresponding anomaly score s is obtained. i Finally, bilinear interpolation was used to enlarge the anomaly score map to the same size as the test image x, obtaining the position x of each pixel. pixel abnormal scores s pixel And compare it with the anomaly score threshold.

7. The method according to claim 6, characterized in that, In step 5, each pixel x pixel The discriminant function f(s) pixel )for:

8. A storage medium, characterized in that, It stores a computer program or instructions that, when executed, implement the method as described in any one of claims 1 to 7.

Images