An image anomaly detection method integrating AVAE and SE modules

Abstract

The present invention discloses an image anomaly detection method that integrates AVAE and SE modules: Step 1: Acquire image data to be detected for anomaly, perform preprocessing on it, and divide the preprocessed data into a training set and a test set; Step 2: Use the training set to train an image anomaly detection model to obtain a trained image anomaly detection model; The image anomaly detection model includes an attention-based variational autoencoder, an SE module, and a discriminator, the discriminator is connected to the output end of the attention-based variational autoencoder, and the attention-based variational autoencoder includes an encoder and a decoder, with the SE module connected between the encoder and the decoder; Step 3: After preprocessing, the image to be detected is imported into the trained image anomaly detection model to obtain an anomaly score. The method of the present invention is very effective for anomaly detection of image data in actual industrial environments.

Description

Technical Field

[0001] The present invention belongs to the technical field of industrial image anomaly detection, and specifically relates to an image anomaly detection method that integrates AVAE and SE modules. Background Art

[0002] Industrial image anomaly detection is a key technology widely used in the manufacturing sector. With the rapid development of digitalization and informatization in the manufacturing industry, vast amounts of image data are being collected and recorded. This data contains a wide range of details and characteristics of the industrial production process. By performing anomaly detection on this image data, anomalies in the production process can be promptly identified, helping companies improve product quality and reduce production risks and losses. With the continuous advancement of technology, industrial image anomaly detection will usher in broader application prospects, bringing greater value and innovation to industrial production. Given the complexity and diversity of industrial image data, developing effective anomaly detection methods is crucial to ensuring the stability and reliability of the production process.

[0003] Modern computer vision technology and machine learning algorithms provide powerful tools and methods for detecting anomalies in industrial images. First, computer vision techniques can be used to preprocess and extract features from industrial images. This includes operations such as image denoising, image enhancement, edge detection, and texture analysis to extract key features from the image. These features can reflect normal conditions in the industrial production process and be compared with abnormal conditions.

[0004] Secondly, machine learning algorithms play a vital role in industrial image anomaly detection. Common machine learning algorithms include support vector machines (SVMs), random forests, and convolutional neural networks (CNNs). These algorithms can learn the characteristics and patterns of normal images by training on labeled normal and abnormal images, and then classify and identify anomalies based on new, unknown images.

[0005] Currently, the VAE model is commonly used for image anomaly detection. It is a generative model that learns latent variables from input data and generates new samples. For normal data, this model can reconstruct the input data well. However, for anomalous data, due to its different distribution from normal data, the model often has difficulty reconstructing the input data, resulting in large reconstruction errors. Therefore, by setting a reconstruction error threshold, data with reconstruction errors exceeding this threshold can be identified as anomalous data. However, VAE models tend to generate fuzzy or ambiguous samples. This is because the generation process is random and cannot guarantee the high quality of each sample, resulting in low detection accuracy. Summary of the Invention

[0006] The purpose of the present invention is to propose an image anomaly detection method that integrates AVAE and SE modules to solve the technical problem of low detection accuracy in existing industrial image anomaly detection technology.

[0007] In order to achieve the above object, the present invention adopts the following technical solutions:

[0008] An image anomaly detection method integrating AVAE and SE modules specifically comprises the following steps:

[0009] Step 1: Obtain the image data to be detected for anomaly, preprocess it, and divide the preprocessed data into a training set and a test set;

[0010] Step 2: Using the training set to train the image anomaly detection model to obtain a trained image anomaly detection model; the image anomaly detection model includes a variational autoencoder based on the attention mechanism, an SE module and a discriminator, the discriminator is connected to the output end of the variational autoencoder based on the attention mechanism, the variational autoencoder based on the attention mechanism includes an encoder and a decoder, and the SE module is connected between the encoder and the decoder;

[0011] In step 3, the image to be detected is preprocessed and then imported into the image anomaly detection model trained in step 2 to obtain an anomaly score as the anomaly detection result.

[0012] Furthermore, the encoder is obtained by cross-superposition of multiple convolutional layers and multiple self-attention mechanism modules.

[0013] Furthermore, the encoder includes 3 convolutional layers and 3 self-attention mechanism modules.

[0014] Furthermore, the SE module includes a compression module, an excitation module and a scale operation connected in sequence; the compression module includes a global average pooling layer, and the excitation module includes an FC layer, a ReLU activation function, an FC layer and a Sigmoid activation function connected in sequence.

[0015] Furthermore, the decoder is obtained by cross-superimposing multiple deconvolution layers and multiple self-attention mechanism modules and then connecting them with a Sigmoid activation function.

[0016] Furthermore, the decoder includes 3 deconvolution layers and 3 self-attention mechanism modules.

[0017] Furthermore, in step 2, the reconstruction loss function of the image anomaly detection model is:

[0018] L total =L BCE +L KLD

[0019] Among them, L BCE is the BCE loss function, L KLD is the KLD loss function.

[0020] Furthermore, a regularization term is added between the convolution layer and the attention mechanism module in the encoder, and a regularization term is added between the deconvolution layer and the attention mechanism module in the decoder.

[0021] Compared with the prior art, the present invention has the following beneficial effects:

[0022] 1. The encoder uses multi-layer convolution to enhance the model's ability to learn features of two-dimensional data such as images and improve its generalization performance. Convolution operations can efficiently process the spatial structure of image data, reduce the number of parameters, provide translation invariance, and increase nonlinear expression capabilities.

[0023] 2. A self-attention mechanism is incorporated into both the encoder and decoder to enhance hidden layer features, allowing the model to focus more on important features, capture global or long-range dependencies, and improve feature representation capabilities. This improves model accuracy and the quality of generated samples, enhances the flexibility and adaptability of feature interactions, improves the consistency of generated samples, and enhances the model's robustness against attacks. It also helps the model adaptively learn the importance of features.

[0024] 3. The encoder part contains multiple convolutional layers for extracting hidden features of the input image, and adds a self-attention mechanism after the convolutional layer to improve the encoding and extraction of global information, improve the latent variable sampling process, and enhance the interpretability of the model.

[0025] 4. The SE module is added between the encoder and decoder in AVAE, further enhancing the model's attention to each channel in the input data and its ability to express features, improving the distinguishability and generalization of features, better reconstructing images, and helping to improve the accuracy of anomaly detection. The SE module's excitation module uses ReLU as the activation function and adds a Sigmoid activation function at the end. It has the advantages of nonlinear mapping, output probability interpretation, and smooth gradient propagation, enabling the neural network to learn and represent complex nonlinear relationships and output probability values ​​in binary classification tasks, while reducing the problems of gradient vanishing and gradient exploding. At the same time, the SE module is embeddable, easy to implement and interpret, and can improve the quality and diversity of generated samples.

[0026] 5. To prevent model overfitting and constrain model parameters, a regularization term (Dropout) is added after the convolutional layer of the encoder and the deconvolution layer of the decoder in the variational autoencoder based on the attention mechanism. This term randomly inactivates neurons to reduce overfitting of the neural network and improve the network's generalization and robustness. This is used to prevent model overfitting and constrain model parameters. Furthermore, the self-attention mechanism is integrated to enable the model to better reconstruct images.

[0027] 6. The model's minimization loss function consists of two parts: BCE loss and KLD loss. By minimizing the reconstruction loss, the decoder is attempted to restore the original data, thereby achieving the reconstruction goal. At the same time, the latent space representation is learned so that the latent variables generated by the encoder are close to the prior distribution; and the latent space is regularized through KL divergence loss to improve the generalization ability and interpretability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The accompanying drawings are used to provide further understanding of the present disclosure and constitute a part of the specification. Together with the following detailed description, they are used to explain the present disclosure, but do not constitute a limitation of the present disclosure.

[0029] Figure 1 This is a diagram of the AVAE encoder architecture in the present invention;

[0030] Figure 2 This is the structural diagram of the SE model integrated in the present invention;

[0031] Figure 3 This is a diagram of the AVAE decoder architecture in the present invention;

[0032] Figure 4 The order of activation functions and regularization terms between Conv2d and Attention layers in the AVAE encoder architecture diagram of the present invention;

[0033] Figure 5 It is the order of activation functions and regularization terms between ConvTranspose2d and Attention in each layer of the AVAE decoder architecture diagram in the present invention.

[0034] The technical solution of the present invention is further explained below with reference to the accompanying drawings and specific embodiments. DETAILED DESCRIPTION

[0035] It should be noted that the following detailed description is illustrative and is intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used in the present invention have the same meanings as commonly understood by those of ordinary skill in the art to which the present application belongs. It should be noted that in the present invention:

[0036] VAE, or VAE (Variational Autoencoder), refers to variational autoencoder.

[0037] AVAE, namely Attention-based Variational Autoencoder, refers to the variational autoencoder based on the attention mechanism in this invention.

[0038] SE, or Squeeze-and-Excitation, refers to an attention mechanism used to enhance the representation capabilities of neural networks.

[0039] Loss refers to the loss function of the deep learning model.

[0040] AUC, or Area Under the Curve, is a performance indicator for measuring the quality of a learner.

[0041] BCE, or Binary Cross-Entropy, is a commonly used binary classification loss function.

[0042] KLD, or Kullback-Leibler Divergence, is the KL divergence loss.

[0043] The overall technical concept of the present invention is: to propose an image anomaly detection method that integrates AVAE and SE modules: first, obtain the image data to be detected and perform preprocessing operations on it. Secondly, the above-mentioned data to be detected is imported into the trained anomaly detection model; the anomaly detection model first uses the encoder part of the Attention-based Variational Autoencoder to extract the hidden layer characteristics of the image to be tested, and then inputs the extracted features into the SE model, performs enhancement operations on the extracted features, and then inputs the features into the decoder for reconstruction operations, and scores the reconstructed image to obtain the score of anomaly detection. Finally, the anomaly score of the test image data is used as its anomaly detection result.

[0044] In accordance with the above technical solution, specific embodiments of the present invention are given below. It should be noted that the present invention is not limited to the following specific embodiments, and all equivalent changes made on the basis of the technical solution of this application fall within the protection scope of the present invention.

[0045] The present invention provides an image anomaly detection method that integrates AVAE and SE modules, which specifically includes the following steps:

[0046] Step 1: Obtain the image data for anomaly detection (either a public dataset or a private dataset) and preprocess it. The preprocessed data is divided into a training set and a test set, where the training set is known normal data.

[0047] Step 2: Use the training set to train the image anomaly detection model to obtain a trained image anomaly detection model;

[0048] The model of the present invention uses only normal data for training during the training phase. During the testing phase, since the model is trained only on normal data, it can reconstruct the input data well for normal data. However, for abnormal data, due to its different distribution from normal data, the model often has difficulty reconstructing the input data, resulting in large reconstruction errors. Therefore, by setting a reconstruction error threshold, we can identify data with reconstruction errors exceeding this threshold as abnormal data.

[0049] Among them, the image anomaly detection model includes a variational autoencoder (AVAE) based on the attention mechanism, an SE module and a discriminator. The discriminator is connected to the output end of the variational autoencoder (AVAE) based on the attention mechanism. The variational autoencoder (AVAE) based on the attention mechanism includes an encoder and a decoder, and the SE module is connected between the encoder and the decoder.

[0050] like Figure 1 As shown in the figure, the encoder is used to extract the hidden features of the input image and then input the extracted features into the SE module. Preferably, the encoder is obtained by cross-superimposing multiple convolutional layers (Conv2d) (preferably 3) and multiple self-attention mechanism modules (Attention) (preferably 3). In this design, the multi-layer convolution superposition is to enhance the model's ability to learn the features of two-dimensional data such as images and improve the model's generalization performance. Adding a self-attention mechanism module after the convolution layer is used to improve the model's ability to model long-range dependencies, improve the accuracy of the model and the quality of the generated samples.

[0051] The self-attention mechanism module is used to implement the following process: First, the input sequence is mapped into the query (Query), key (Key), and value (Value) space through a linear transformation; then, the attention score matrix is ​​calculated, that is, the query vector is multiplied by the transpose of the key vector and scaled; next, the Softmax function is applied to convert the attention score matrix into an attention weight matrix, where each element represents the importance of the value at the corresponding position; finally, the weighted sum result is obtained by multiplying the attention weight matrix by the value vector, which is the output of the self-attention mechanism. This calculation process can operate on each position in the sequence in parallel, thereby improving efficiency. This process can also be expressed by a mathematical formula:

[0052]

[0053] Among them, Q, K and V represent the matrix forms of queries, keys and values ​​respectively, d k Is the dimension of keys. Represents the similarity score between queries and keys, divided by d k To reduce the variance of the gradient. Finally, the scores are converted into weights through the softmax function and applied to the values ​​matrix to obtain the output matrix.

[0054] like Figure 2 As shown in Figure 1, the SE module is used to adaptively adjust the importance of channels by learning the relationship between channels to improve the model's performance in feature representation. This allows the model to focus more on important features and enhance the model's discriminative and representational capabilities. The SE module consists of a compression module, an excitation module, and a scale operation, which are connected in sequence.

[0055] The compression module is used to compress the original input features in the spatial dimension to obtain a channel-dimensional feature description, which is then sent to the excitation module. The original input features refer to the features of the input SE. Preferably, the compression module includes a global average pooling layer.

[0056] The excitation module is used to obtain the weight vector of the feature channel according to the features input by the compression module. Preferably, the excitation module includes an FC layer (fully connected layer), a ReLU activation function, an FC layer and a Sigmoid activation function connected in sequence.

[0057] In the above technical solution, the input data is compressed into a scalar value through global average pooling, and then it is excited by the transformation of two fully connected layers (FC layers). Finally, the learned channel weight vector is scaled (multiplied) with the input feature map in the scaling stage to enhance important features and suppress unimportant features, thereby improving the discriminability and representation ability of the model. In general, the SE module achieves adaptive adjustment of the input feature channel weights by compressing the global average pooling operation and the fully connected layer of the excitation module. This adaptive adjustment can increase the model's attention to different channels, enhance the representation of useful features and weaken the influence of unimportant features, thereby improving the performance and representation ability of the model.

[0058] like Figure 3As shown, the decoder is constructed by stacking multiple deconvolutional layers (ConvTranspose2d) (preferably three) and multiple self-attention modules (Attention) (preferably three), followed by a Sigmoid activation function. In this design, the self-attention module improves the model's ability to model long-range dependencies, improving model accuracy and the quality of generated samples, thereby enabling better image reconstruction. The Sigmoid activation function maps the output to a probability distribution to generate samples that conform to the data distribution. The Sigmoid activation function converts real values ​​into probabilities between 0 and 1, thereby enabling the training and sampling of the generative model.

[0059] The discriminator determines the difference between input and output data and identifies data with a reconstruction error exceeding a preset threshold as anomaly. The decoder's reconstructed image and the input image are fed into the discriminator, which then scores the reconstructed image to determine anomaly detection scores.

[0060] During the training process of step 2, the reconstruction loss function of the image anomaly detection model is:

[0061] L total =L BCE +L KLD

[0062] Among them, L BCE The BCE loss function is a loss function for binary classification tasks that measures the model's prediction error by calculating the cross entropy. Specifically, for each sample, the BCE loss function adds the model's predicted probability to the binary cross entropy of the corresponding true label and takes the average to obtain the overall loss, which is used to measure the reconstruction error between the decoder's output and the input data.

[0063] L KLD The KLD loss function is a loss function used to measure the difference between probability distributions and is commonly used in variational autoencoders (VAEs). It measures the difference between the latent distribution generated by the model and the true latent distribution, and measures the model's learning effect by calculating the KL divergence between the two distributions. Specifically, the KLD loss function calculates the KL divergence of the latent distribution generated by the model and the true latent distribution, and adds them together to obtain the overall KLD loss.

[0064] In step 3, the image to be detected is preprocessed and then imported into the image anomaly detection model trained in step 2 to obtain an anomaly score as the anomaly detection result.

[0065] Preferably, in order to prevent the model from overfitting and constrain the model parameters, regularization terms are added after the convolutional layer in the encoder and the deconvolutional layer in the decoder, specifically, Dropout regularization is added before the corresponding attention mechanism module (such as Figure 4 、 Figure 5 By randomly inactivating neurons, the overfitting of the neural network is reduced and the generalization ability and robustness of the network are improved.

[0066] Example 1

[0067] This example uses the AUC and F1 scores to evaluate the model's anomaly detection performance. The AUC is a metric used to evaluate the performance of a binary classification model and is typically applied to a receiver operating characteristic (ROC) curve. The ROC curve plots the relationship between the true positive rate (TruePositiveRate) and the false positive rate (FalsePositiveRate) at different thresholds.

[0068] AUC represents the area under the receiver operating characteristic (ROC) curve and ranges from 0 to 1. A larger AUC value indicates better model performance. The advantage of AUC is that it is not affected by class imbalance. Even when the number of positive and negative examples in a dataset is unbalanced, AUC still provides a reliable evaluation metric. It comprehensively considers both the true positive rate and the false positive rate and is suitable for evaluating various classifier models.

[0069] The AUC calculation steps are as follows: Calculate the True Positive Rate (TPR) and the False Positive Rate (FPR). The True Positive Rate (TPR) refers to the proportion of samples correctly predicted as positive among actual positive examples. It is calculated as follows: TPR = TP / (TP + FN), where TP represents the number of true positive examples (positive examples correctly classified as positive by the classifier) ​​and FN represents the number of false negative examples (positive examples incorrectly classified as negative by the classifier). The False Positive Rate (FPR) refers to the proportion of samples incorrectly predicted as positive among actual negative examples. It is calculated as follows: FPR = FP / (FP + TN), where FP represents the number of false positive examples (negative examples incorrectly classified as positive by the classifier) ​​and TN represents the number of true negative examples (negative examples correctly classified as negative by the classifier). Based on different thresholds, a receiver operating characteristic (ROC) curve is plotted. The horizontal axis of the ROC curve is FPR, and the vertical axis is TPR. Calculate the AUC, the area under the ROC curve. The value range of AUC is between 0 and 1. The closer the AUC is to 1, the better the model performance.

[0070] The F1 score is a commonly used comprehensive evaluation metric for measuring the performance of binary classification models. It comprehensively considers the trade-off between precision and recall.

[0071] Precision refers to the proportion of samples predicted by the model as positive that are actually positive, and can be understood as the model's prediction accuracy. Recall refers to the proportion of samples correctly predicted by the model as positive out of all positive samples, and can be understood as the model's recall rate.

[0072] The F1 score is the harmonic mean of precision and recall, and is used to comprehensively evaluate the performance of the model on positive and negative samples. Its calculation formula is:

[0073] F1=2*(Precision*Recall) / (Precision+Recall)

[0074] The F1 score ranges from 0 to 1, with values ​​closer to 1 indicating better model performance. It takes into account both the model's accuracy and recall. A model with high precision and recall will have a high F1 score; a large discrepancy between precision and recall will result in a low F1 score.

[0075] This paper uses the public CIFIA-10 dataset for anomaly detection. The model is implemented in PyTorch. To train the model, the Adam optimizer with an initial learning rate of 0.003 is introduced. BCE and KLD are used as loss functions for the original and reconstructed data. The training set consists of 50,000 images, the test set consists of 10,000 images, the batch size is 128, and the training rounds are 100. The entire experiment was conducted on a computer running the Windows 11 operating system, using an AMD Ryzen 7 5800H with Radeon Graphics, and a GeForce GTX3050 with 4GB of memory.

[0076] The experimental results are as follows:

[0077]

[0078] The above experiments have verified the effectiveness of the method of the present invention in detecting anomalies in image data under actual industrial environments.

[0079] The preferred embodiments of the present disclosure are described in detail above in conjunction with the accompanying drawings. However, the present disclosure is not limited to the specific details of the above embodiments. Within the technical concept of the present disclosure, various simple modifications can be made to the technical solutions of the present disclosure, and these simple modifications all fall within the scope of protection of the present disclosure.

[0080] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any appropriate manner without contradiction. In order to avoid unnecessary repetition, the present disclosure will not further describe various possible combinations.

[0081] In addition, the various embodiments of the present disclosure may be arbitrarily combined, and as long as they do not violate the concept of the present disclosure, they should also be regarded as the contents disclosed by the present disclosure.

Claims

1. An image anomaly detection method integrating AVAE and SE modules, characterized in that: The specific steps include: Step 1: Obtain the image data to be detected for anomaly, preprocess it, and divide the preprocessed data into a training set and a test set; Step 2: Using the training set to train the image anomaly detection model to obtain a trained image anomaly detection model; the image anomaly detection model includes a variational autoencoder based on the attention mechanism, an SE module and a discriminator, the discriminator is connected to the output end of the variational autoencoder based on the attention mechanism, the variational autoencoder based on the attention mechanism includes an encoder and a decoder, and the SE module is connected between the encoder and the decoder; The encoder is obtained by cross-superposition of multiple convolutional layers and multiple self-attention mechanism modules; A regularization term is added between the convolution layer and the attention mechanism module in the encoder, and a regularization term is added between the deconvolution layer and the attention mechanism module in the decoder; the decoder is obtained by cross-stacking multiple deconvolution layers and multiple self-attention mechanism modules and then connecting them with a Sigmoid activation function; The reconstruction loss function of the image anomaly detection model is: in, is the BCE loss function, is the KLD loss function; In step 3, the image to be detected is preprocessed and then imported into the image anomaly detection model trained in step 2 to obtain an anomaly score as the anomaly detection result.

2. The image anomaly detection method integrating AVAE and SE modules according to claim 1, characterized in that: The encoder includes 3 convolutional layers and 3 self-attention mechanism modules.

3. The image anomaly detection method integrating AVAE and SE modules according to claim 1, characterized in that: The SE module includes a compression module, an excitation module and a scale operation connected in sequence; the compression module includes a global average pooling layer, and the excitation module includes an FC layer, a ReLU activation function, an FC layer and a Sigmoid activation function connected in sequence.

4. The image anomaly detection method integrating AVAE and SE modules according to claim 1, wherein: The decoder includes 3 deconvolution layers and 3 self-attention mechanism modules.

Images