Anomaly detection method based on attention U-shaped convolutional network

Abstract

The present invention discloses an anomaly detection method based on an attention U-shaped convolutional network, which relates to an anomaly detection method, comprising the following steps: S1: performing global and local sampling, wherein the global sampling uses the patch of the entire image as the input of the model, and the local sampling uses the patch on an image as the input of the model; S2: constructing a model: the model includes a basic U‑Net and three embedding modules; S3: model embedding: embedding the model into the EfficientAD method, replacing the original autoencoder with the attention U‑Net, and forming an overall network architecture; S4: model training: adopting different rotation training strategies for different logical anomalies, inputting the trained images into the network for training, generating an anomaly detection model and saving it; S5: detection results: inputting a batch of test images, using the trained anomaly detection model for reasoning, and returning the detection and positioning results. The anomaly map obtained after the present invention uses U‑Net reconstruction has a clearer outline, and the positioning results obtained are more refined.

Description

Technical Field

[0001] The present invention relates to the field of industrial anomaly detection, and in particular to an anomaly detection method based on an attention U-shaped convolutional network. Background Art

[0002] Anomaly detection is a basic problem in tasks based on industrial images. In recent years, it has been widely used in industrial production, intelligent driving, medical image analysis, etc., and has been deeply studied by the academic community. In recent years, with the widespread application of deep neural networks in target detection and the good results achieved, people have also explored the use of neural networks for anomaly detection. The most important features of anomaly detection are structural anomalies and logical anomalies. The structural anomaly areas that may appear in the former are very small and difficult to detect, which poses a great challenge to the precise positioning of the abnormal area. The latter is more abstract. It is not that a specific part of the image has an abnormality, but the logical relationship represented in the image, which is related to information such as position, direction, color, and size.

[0003] However, anomaly detection is still a challenging task in computer vision. First, it is difficult to reconstruct only normal images for anomaly detection; second, for logical anomalies, the current anomaly detection methods are still in the exploratory stage.

[0004] In order to solve the above difficulties, Kilian Batzner et al. proposed an accurate visual anomaly detection method with millisecond-level delay. The model combines a student-teacher network and a U-Net network. The student-teacher network realizes local reconstruction, and the combination of U-Net and teacher network realizes global reconstruction.

[0005] However, the above method still has much room for improvement in the field of logical anomalies, and the U-Net network used is not suitable for anomaly detection.

[0006] The existing technology still has problems such as low accuracy of logical anomaly detection and weak effect of U-Net in reconstructing semantic information.

[0007] Therefore, it is necessary to provide an anomaly detection method based on an attention U-shaped convolutional network to solve the above problems. Summary of the invention

[0008] The purpose of the present invention is to provide an anomaly detection method based on an attention U-shaped convolutional network. A channel attention module is inserted into the encoder so that the model can learn features better and enhance the feature expression ability of the network. A spatial attention module is inserted into the decoder so that the model can focus on the features at the effective position in each picture, ignore the background features, and improve the discrimination ability of the model. A multi-scale feature fusion module is embedded so that the model can learn refined semantic information. Different rotation strategies are adopted for different logical anomalies to further improve the performance of logical anomaly detection.

[0009] To achieve the above object, the present invention provides an anomaly detection method based on an attention U-shaped convolutional network, comprising the following steps:

[0010] S1: Perform global and local sampling. Global sampling uses the patch of the entire image as the input of the model, and local sampling uses the patch on an image as the input of the model.

[0011] S2: Build the model: The model includes the basic U-Net and three embedding modules;

[0012] S3: Model embedding: The model is embedded into the EfficientAD method, and the original autoencoder is replaced with the attention U-Net to form the overall network architecture;

[0013] S4: Model training: Different rotation training strategies are used for different logical anomalies. The training images are input into the network for training, and the anomaly detection model is generated and saved.

[0014] S5: Detection results: Input a batch of test images, use the trained anomaly detection model for inference, and return the detection and positioning results.

[0015] Preferably, in step S2, the embedding module includes a channel attention module, a spatial attention module and a multi-scale feature fusion module, which specifically includes the following steps:

[0016] S21: A channel attention module is inserted into the encoder in the basic U-Net to enhance the representation ability of the encoder. A maximum pooling branch and a long connection branch are added to the channel attention module. The maximum pooling branch performs texture anomaly detection, and the long connection branch reduces the negative impact of feature reconstruction. After each layer of the encoder is embedded with the channel attention module, the last layer of the encoder will extract important features.

[0017] S22: Insert a spatial attention module into the decoder of U-Net. In the feature reconstruction process, upsampling operation is used to restore the feature vector to the scale of the image. The spatial attention module focuses on important features at the spatial scale.

[0018] S23: After step S21 and step S22, a multi-scale feature fusion module is inserted into the encoder in the basic U-Net to refine the semantic information, the important features in the encoder are divided into four features of different scales and fused, and the fused features are added to the corresponding decoder features.

[0019] Preferably, in step S4, different rotation training strategies are adopted for different logical anomalies, specifically including:

[0020] For the breakfast box images in the dataset, add images rotated 180 degrees for training;

[0021] For the screw images in the dataset, add images rotated by 90 degrees, 180 degrees, and 45 degrees for training;

[0022] Input the images in the training set into the network structure in step S4, specify the number of training iterations, perform training in batches, and save the optimal model results.

[0023] Preferably, in step S5, 10 different equal images are randomly sampled from the test set, and the final prediction is obtained by the average of all scores, using the metrics: AUROC on the MAD dataset and MAD LOCO dataset for anomaly detection.

[0024] Therefore, the present invention adopts the above-mentioned anomaly detection method based on the attention U-shaped convolutional network, which has the following beneficial effects:

[0025] (1) The present invention inserts a channel attention module into the encoder, so that the model can better learn features and enhance the feature expression ability of the network. The spatial attention module is inserted into the decoder, so that the model can focus on the features at the effective position in each image, ignore the background features, and improve the model's discrimination ability. The multi-scale feature fusion module is embedded, so that the model can learn refined semantic information. Different rotation strategies are adopted for different logical anomalies, which further improves the performance of logical anomaly detection.

[0026] (2) The present invention solves the problems existing in the prior art such as low accuracy of logical anomaly detection and weak effect of U-Net in reconstructing semantic information.

[0027] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a flow chart of an anomaly detection method based on an attention U-shaped convolutional network of the present invention;

[0029] Figure 2It is the overall network structure diagram of the anomaly detection method based on the attention U-shaped convolutional network of the present invention;

[0030] Figure 3 It is a CSI-UNet structure diagram of the anomaly detection method based on the attention U-shaped convolutional network of the present invention;

[0031] Figure 4 It is a structural diagram of a channel attention module of an anomaly detection method based on an attention U-shaped convolutional network of the present invention;

[0032] Figure 5 It is a structural diagram of a multi-scale feature fusion module of an anomaly detection method based on an attention U-shaped convolutional network of the present invention;

[0033] Figure 6 This is a positioning result diagram of the anomaly detection method based on the attention U-shaped convolutional network of the present invention. DETAILED DESCRIPTION

[0034] The technical solution of the present invention is further described below through the accompanying drawings and embodiments.

[0035] Unless otherwise defined, technical or scientific terms used in the present invention shall have the common meanings understood by one having ordinary skills in the field to which the present invention belongs.

[0036] The words "include" or "comprises" and the like used in the present invention mean that the elements before the word include the elements listed after the word, and do not exclude the possibility of also including other elements. The orientation or position relationship indicated by the terms "inside", "outside", "upper", "lower", etc. is based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it cannot be understood as a limitation of the present invention. When the absolute position of the described object changes, the relative position relationship may also change accordingly. In the present invention, unless otherwise clearly specified and limited, the terms "attachment" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral body; it can be directly connected, or indirectly connected through an intermediate medium, and it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0037] Example

[0038] like Figure 1 As shown, the present invention provides an anomaly detection method based on an attention U-shaped convolutional network, comprising the following steps:

[0039] S1: Perform global and local sampling. Global sampling uses the patch of the entire image as the input of the model, and local sampling uses the patch on an image as the input of the model.

[0040] S2: Construct model: Figure 3 As shown in , the model includes a basic U-Net and three embedding modules; in step S2, the embedding module includes a channel attention module, a spatial attention module and a multi-scale feature fusion module, as shown in Figure 4-Figure 5 As shown, the specific steps include:

[0041] S21: Insert a channel attention module into the encoder in the basic U-Net to enhance the encoder's representation ability. A maximum pooling branch and a long connection branch are added to the channel attention module. The maximum pooling branch performs texture anomaly detection with significant detection results. The long connection branch reduces the negative impact of feature reconstruction. After each layer of the encoder is embedded with the channel attention module, the last layer of the encoder will extract important features; this is conducive to the reconstruction of subsequent features.

[0042] S22: Insert a spatial attention module into the decoder of U-Net to enhance the representation ability of the encoder. In the feature reconstruction process, upsampling operation is used to restore the feature vector to the image scale. The spatial attention module focuses on important features on the spatial scale; therefore, inserting a spatial attention module into the decoder is conducive to the reconstruction of important spatial features.

[0043] S23: After step S21 and step S22, a multi-scale feature fusion module is inserted into the encoder in the basic U-Net to refine the semantic information, divide the important features in the encoder into four different scales of features and fuse them, and add the fused features to the corresponding decoder features. The multi-scale feature fusion module can refine the reconstructed semantic information, so the anomaly map obtained after reconstruction using U-Net has a clearer outline and the obtained positioning result is more refined.

[0044] S3: Model embedding: Figure 2 As shown, the model is embedded into the EfficientAD method, and the original autoencoder is replaced with the attention U-Net to form the overall network architecture;

[0045] S4: Model training: Different rotation training strategies are adopted for different logical anomalies, and the training images are input into the network for training, and an anomaly detection model is generated and saved; in step S4, different rotation training strategies are adopted for different logical anomalies, specifically including:

[0046] For the breakfast box images in the dataset, add images rotated 180 degrees for training;

[0047] For the screw images in the dataset, add images rotated by 90 degrees, 180 degrees, and 45 degrees for training;

[0048] Input the images in the training set into the network structure in step S4, specify the number of training iterations, perform training in batches, and save the optimal model results.

[0049] S5: Detection results: Input a batch of test images, use the trained anomaly detection model for reasoning, and return the detection and positioning results. The positioning results are as follows: Figure 6 shown.

[0050] In step S5, 10 different equal images are randomly sampled from the test set and the final prediction is obtained by averaging all the scores using the metrics: AUROC on the MAD dataset and MAD LOCO dataset for anomaly detection.

[0051] Comparative Example

[0052] This scheme is compared with the anomaly detection methods: GCAD, FastFlow, DSR, PatchCore, AST and EfficientAD-S. The detection results are shown in Table 1.

[0053] Table 1

[0054]

[0055]

[0056] As can be seen from Table 1, our scheme achieves higher scores on both datasets compared with the other six methods, proving that our method is superior to the compared methods.

[0057] Therefore, the present invention adopts the above-mentioned anomaly detection method based on the attention U-shaped convolutional network, inserts a channel attention module in the encoder, so that the model can better learn the features and enhance the feature expression ability of the network. Inserting a spatial attention module in the decoder enables the model to focus on the features at the effective position in each picture, ignore the background features, and improve the model's discrimination ability. Embedding a multi-scale feature fusion module enables the model to learn refined semantic information. And different rotation strategies are adopted for different logical anomalies, which further improves the performance of logical anomaly detection.

[0058] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. An anomaly detection method based on an attention U-shaped convolutional network, characterized in that: The following steps are involved: S1: Perform global and local sampling. Global sampling uses the patch of the entire image as the input of the model, and local sampling uses the patch on an image as the input of the model. S2: Build the model: The model includes the basic U-Net and three embedding modules; In step S2, the embedding module includes a channel attention module, a spatial attention module and a multi-scale feature fusion module, which specifically includes the following steps: S21: A channel attention module is inserted into the encoder in the basic U-Net to enhance the representation ability of the encoder. A maximum pooling branch and a long connection branch are added to the channel attention module. The maximum pooling branch performs texture anomaly detection, and the long connection branch reduces the negative impact of feature reconstruction. After each layer of the encoder is embedded with the channel attention module, the last layer of the encoder will extract important features. S22: Insert a spatial attention module into the decoder of U-Net. In the feature reconstruction process, upsampling operation is used to restore the feature vector to the scale of the image. The spatial attention module focuses on important features at the spatial scale. S23: After step S21 and step S22, a multi-scale feature fusion module is inserted into the encoder in the basic U-Net to refine the semantic information, the important features in the encoder are divided into four features of different scales and fused, and the fused features are added to the corresponding decoder features; S3: Model embedding: The model is embedded into the EfficientAD method, and the original autoencoder is replaced with the attention U-Net to form the overall network architecture; S4: Model training: Different rotation training strategies are used for different logical anomalies. The training images are input into the network for training, and the anomaly detection model is generated and saved. S5: Detection results: Input a batch of test images, use the trained anomaly detection model for inference, and return the detection and positioning results.

2. The anomaly detection method based on an attention U-shaped convolutional network according to claim 1, characterized in that: In step S4, different rotation training strategies are adopted for different logical anomalies, specifically including: For the breakfast box images in the dataset, add images rotated 180 degrees for training; For the screw images in the dataset, add images rotated by 90 degrees, 180 degrees, and 45 degrees for training; Input the images in the training set into the network structure in step S4, specify the number of training iterations, perform training in batches, and save the optimal model results.

3. The anomaly detection method based on an attention U-shaped convolutional network according to claim 1 is characterized in that: In step S5, 10 different equal images are randomly sampled from the test set and the final prediction is obtained by averaging all the scores using the metrics: AUROC on the MAD dataset and MAD LOCO dataset for anomaly detection.

Images