Anomaly detection method based on image augmentation and image reconstruction

Abstract

The application provides an abnormality detection method based on image augmentation and image reconstruction, comprising the following steps: S1, acquiring a test image; S2, inputting the test image into a generation network to realize abnormality-free reconstruction of an input sample; S3, cascading the test sample and the abnormality-free reconstructed sample obtained by the generation network at a channel to input into a discrimination network to generate an abnormality score, thereby realizing instance-level abnormality detection; and S4, based on a gradient class weighting activation mapping method, using gradient information flowing to the last convolutional layer in the discrimination network to generate a positioning map of an abnormal region, thereby realizing abnormality positioning. On the basis of reconstructing an abnormality-free sample based on a generative model, the application uses a data augmentation strategy to construct a pseudo abnormality sample on the basis of a normal sample and add the pseudo abnormality sample, thereby converting unsupervised learning into supervised learning, converting an original image instance-level abnormality detection problem into an image classification problem, and further improving the accuracy of abnormality detection, and relatively accurate abnormality positioning can be realized.

Description

Technical Field

[0001] This invention relates to the field of image anomaly detection technology, and in particular to an anomaly detection method based on image augmentation and image reconstruction. Background Technology

[0002] In practical applications and engineering, due to the sudden, diverse, and unpredictable nature of faults, it is often difficult to obtain a large number of labeled or unlabeled real anomaly samples for direct network training. In such cases, traditional object detection and image segmentation methods are no longer applicable. Anomaly detection, on the other hand, is a detection model built using unlabeled or normal samples to detect anomalous samples that differ from normal patterns, matching the characteristics of faults. Generally, collecting normal images is far less difficult than collecting anomalous images. Therefore, in practical applications, anomaly detection methods require less time and manpower. Furthermore, since the anomaly detection model detects anomalies by analyzing the differences between the model and normal images, this gives anomaly detection algorithms a strong ability to detect various types and even entirely new anomaly patterns.

[0003] Most image anomaly detection algorithms based on generative models rely on image reconstruction for anomaly detection and localization. This is because generative models can directly generate similar samples, and image reconstruction methods can efficiently locate anomaly regions using the reconstructed image without resorting to sliding window or region-by-region analysis, offering better real-time performance and advantages unmatched by distance-based and classification-based methods. However, research on unsupervised reconstruction methods has revealed a trade-off between instance-level anomaly detection and pixel-level anomaly localization, making it impossible to achieve optimal performance simultaneously. Furthermore, the quality of normal samples in real-world industrial datasets places higher demands on the generative capabilities of generative models, further impacting the quality of the reconstructed image and consequently affecting the final anomaly detection and localization performance.

[0004] Recent advances in deep learning have enabled algorithms to achieve state-of-the-art performance in various applications such as image classification, image segmentation, and object detection. Due to its advantages, including not requiring manual feature design, greater algorithm versatility, and the ability to effectively learn the inherent patterns and representational levels within samples in an end-to-end manner, it has been widely adopted in image anomaly detection tasks, achieving performance superior to traditional anomaly detection methods. Generative models, a highly active research branch in deep learning, encompass basic structures such as autoencoders, variational autoencoders, and generative adversarial networks. With their powerful semantic information representation capabilities, they have become the most promising machine learning methods in semi-supervised or self-supervised learning and are widely applied in image anomaly detection.

[0005] In the field of image anomaly detection, image reconstruction-based methods are mainly based on generative models. The core idea is to encode and decode the input normal image and train a neural network with the reconstruction input as the target, thereby learning the distribution patterns of normal images. Image reconstruction methods, using the reconstructed image, can efficiently locate anomaly regions without employing sliding window or region-by-region analysis methods, possessing advantages unmatched by distance-based and classification surface-based methods.

[0006] In the research of unsupervised reconstruction methods, it was found that there is a certain contradiction between instance-level anomaly detection and pixel-level anomaly localization tasks, making it impossible to achieve optimal performance simultaneously. This is because in generative models, whether autoencoders or generative adversarial networks, the feature space corresponding to their intermediate layers contains regions that can decode other types of images. Therefore, when there are many training samples, the entire structure exhibits a strong learning ability and an overly strong adaptability to potential anomaly samples, leading to model generalization problems and affecting anomaly detection performance. In instance-level anomaly detection tasks based on reconstruction error metrics, the key is to amplify the reconstruction error between normal and anomaly samples, that is, to further address the model's generalization problem and eliminate the reconstruction of possible anomaly regions as much as possible. However, when most reconstruction algorithms address the generalization problem by optimizing latent variables, selecting only a portion of effective features for reconstruction inevitably loses a lot of detail, thus affecting reconstruction quality. If the image reconstruction is blurry, even normal regions will show significant differences during the comparison of reconstructed images. This makes it even more difficult to achieve pixel-level anomaly localization, especially for anomalies that already occupy a small proportion in the original image.

[0007] Furthermore, the quality issues of normal samples in real-world industrial datasets place higher demands on the generative capabilities of generative models. Because real-world industrial images often contain numerous white patches, black dots, and other irregularities in normal samples, aside from parts, these are often environmental imperfections like rust or dirt, not the anomalies we need to detect. These inclusions significantly interfere with the reconstruction process, affecting the generative adversarial network's learning of normal distribution features during training and causing numerous instances of excessively large pixel errors in normal images during testing. This results in normal and anomaly images being virtually indistinguishable. In short, when only normal samples are used, and these samples also contain significant and unclear interference, the detection performance will inevitably be limited. Summary of the Invention

[0008] To address the shortcomings of existing technologies, this invention provides an instance-level anomaly detection method based on image augmentation and image reconstruction. Building upon the reconstruction of anomaly-free samples using a generative model, this invention employs a data augmentation strategy to construct pseudo-anomaly samples from normal samples and incorporate them into the training process. This transforms unsupervised learning into supervised learning, converting the original instance-level anomaly detection problem into an image classification problem. Pixel-level anomaly localization then becomes an image segmentation or object detection task, thus fundamentally resolving the contradiction between the performance constraints of instance-level anomaly detection and pixel-level anomaly localization. Furthermore, existing image classification and segmentation networks are relatively mature, and their performance improvement is less limited. Compared to reconstructed anomaly metrics, they are no longer entirely constrained by the quality of positive samples or the generative model. Therefore, in principle, a certain performance improvement can be achieved for real-world industrial datasets where low sample resolution and significant interference negatively impact detection performance.

[0009] To address the aforementioned technical problems, this invention provides the following technical solution: an instance-level anomaly detection method based on image augmentation and image reconstruction, comprising the following steps:

[0010] S1 acquires the test image;

[0011] S2 inputs the test image into the generator network to achieve anomaly-free reconstruction of the input sample;

[0012] S3 concatenates the test sample and the anomaly-free reconstructed sample obtained by the generator network at the channel and sends them into the discriminator network to generate an anomaly score map corresponding to a single image, thereby achieving instance-level anomaly detection.

[0013] S4 uses the gradient-weighted activation mapping method to generate a localization map of the abnormal region using the gradient information of the last convolutional layer in the flow direction discriminant network, thereby achieving anomaly localization.

[0014] Furthermore, in step S2, the method for constructing the sub-network includes the following steps:

[0015] S21 builds and generates a sub-network;

[0016] S22 inputs normal samples, using structural similarity loss L... SSIM Reconstruction error loss L con Feature matching loss L lat and countermeasures against losses L adv The weighted sum is used as the objective function to train and generate subnetworks.

[0017] Furthermore, in step S21, the generating subnetwork adopts a bow-tie symmetrical structure of encoder-decoder to realize the mutual mapping from the input image to the latent space, thereby reconstructing the input image. Then, an encoder is added as a discriminator to form adversarial training, ensuring the quality of the reconstructed image.

[0018] Furthermore, in step S22, normal samples are input, with structural similarity loss L. SSIM Context loss L con Feature matching loss L lat and countermeasures against losses L adv The weighted sum is used as the objective function to train and generate a sub-network. The specific process includes the following steps:

[0019] S221 Structural Similarity Loss L SSIM Measuring the input image I and the reconstructed image I r The degree of distortion and similarity are expressed as:

[0020]

[0021] Where H and W are the height and width of the input image I, respectively, and N... p SSIM(I,I) represents the number of pixels in the input image I. r ) (i,j) Using image coordinates (i,j) as the center value, input image I and reconstructed image I r Structural similarity values ​​corresponding to the same patch;

[0022] S222 Context Loss L con It is the input image I and the reconstructed image I r The reconstruction error loss between the original and reconstructed images is reduced at the pixel level, and its expression is:

[0023] L con =‖II r ||2

[0024] S223 Feature Matching Loss L lat It is a reconstructed image I r The mean square error between the input image I and the output of the discriminator's hidden layer is used to obtain the input image I and the reconstructed image I. r The most similar potential representation is expressed as:

[0025]

[0026] Where z is the latent representation of the input image I obtained by passing it through the hidden layer f of the discriminator in the generator network. It is a reconstructed image I rThe latent representation obtained through the discriminator hidden layer f in the generative network;

[0027] S224 Countermeasures Against Losses L adv It is the input image I and the reconstructed image I r The cross-entropy obtained after discrimination by the discriminator aims to maximize the reconstruction capability of the normal image I during training, and is expressed as:

[0028]

[0029] in, Let I represent the distribution of real data p. I The samples obtained from sampling, D(I), D(I) r ) represent the output obtained after the input passes through the discriminator in the generator network.

[0030] Furthermore, in step S3, the test sample and the anomaly-free reconstructed sample obtained by the generator network are concatenated at the channel and fed into the discriminator network to generate the anomaly score corresponding to a single image. The specific process includes the following steps:

[0031] S31 constructs a discriminative subnetwork;

[0032] S32 trains the discriminant subnetwork by concatenating normal samples and constructed pseudo-abnormal samples with the abnormal reconstruction samples generated by the subnetwork, and then feeding them into the discriminant subnetwork for training as positive and negative samples.

[0033] In the S33 testing phase, thresholds are set based on the probability values ​​of all test images output by the sigmoid activation layer of the discrimination network to achieve binary classification of normal and abnormal images.

[0034] Furthermore, in step S31, the discriminant sub-network model consists of 7 blocks, one convolutional layer, and one sigmoid activation layer. Each block contains two sets of convolutional layers, a BN layer, a ReLU activation layer, and a max pooling layer.

[0035] Furthermore, in step S32, the normal samples and the constructed pseudo-abnormal samples are concatenated with the abnormal-free reconstructed samples generated by the sub-network, and then used as positive and negative samples to be fed into the discriminative sub-network for training. The specific process includes the following steps:

[0036] S321 generates two types of pseudo-abnormal samples, CutpasteNormal and CutpasteScar, from normal samples based on the cutpaste method.

[0037] S322 constructs perlin-noise-based pseudo-anomaly samples by sampling irregular texture fragments on anomaly source image datasets that are independent of the normal input sample distribution, based on the use of Perlin noise.

[0038] S323 randomly selects pseudo-abnormal samples as negative sample inputs.

[0039] The S324 sample and the non-anomaly sample reconstructed by the generator subnetwork are concatenated at the channel and fed into the discriminant subnetwork. This ensures that the input pseudo-anomalies carry more features of normal samples, making them as close to normal samples as possible. This increases the training difficulty of the discriminant subnetwork and prevents overfitting due to easy differentiation.

[0040] S325 uses the focal loss function as the objective function for the training phase of the discriminant subnetwork, expressed as:

[0041] FL=-α i (1-p i ) γ y i log(p i )-α i (p i ) γ (1-y i log(1-p) i )

[0042] Among them, y i p represents the label of sample i, with 1 for positive class and 0 for negative class; i α represents the probability that sample i is predicted as positive; γ is the adjustment of this loss function. When γ = 0, this expression degenerates into binary classification cross-entropy loss. i It is a balancing factor.

[0043] By employing the above technical solution, the present invention provides an instance-level anomaly detection method based on image augmentation and image reconstruction, which has at least the following beneficial effects:

[0044] 1. To address the problem of insufficient abnormal samples in anomaly detection, a method based on cutpaste and Berlin noise is proposed to introduce local irregularities into normal samples, thereby constructing multiple classes of pseudo-abnormal samples to augment the data and supplement the training set with abnormal samples. This transforms general unsupervised anomaly detection into a supervised classification problem and pixel-level anomaly localization into an image segmentation problem.

[0045] 2. A generative subnetwork is introduced by combining an autoencoder and a generative adversarial network, so that all input pseudo-anomaly samples can obtain corresponding non-anomaly reconstructed samples. These samples are fed into the discriminator network through channel cascading, so that the constructed pseudo-anomaly samples retain more normal features when participating in training. This increases the training difficulty of the discriminator network and prevents overfitting due to the ease of distinguishing between normal and abnormal.

[0046] 3. The loss functions used for reconstructing network images are mostly mean squared error, peak signal-to-noise ratio, etc. Considering that the way these loss functions calculate the difference only treats the image as an isolated pixel, ignoring some visual features contained in the image, especially the local structural information of the image, the image reconstruction is blurry and the model is prone to generalization. In order to solve this problem, we propose a structural similarity measure and a weighted sum of reconstruction pixel error loss, feature matching loss and adversarial loss as the total loss function, so that the input sample can achieve clearer, more detailed and more similar anomaly-free reconstruction.

[0047] 4. Considering that positive and negative samples are usually highly imbalanced in anomaly detection, the cross-loss entropy function commonly used in binary classification problems is prone to overfitting and lacks the ability to generalize to detect real anomalies during the testing phase. To solve this problem, the focal loss function is used, which allows the model to focus on the sparse and difficult-to-distinguish sample categories, thereby improving the robustness of the discrimination network.

[0048] 5. The main function of the generative subnetwork proposed in this invention is to realize the reconstruction of samples without anomalies. It can be replaced by any similar generative model. The discriminative subnetwork mainly realizes the differentiation or localization of normal and abnormal samples. It can also be replaced by any mature classification or segmentation network at present, such as ResNet18, UNet, etc. This shows the universality and practicality of the anomaly detection algorithm based on image augmentation and image reconstruction proposed in this invention. Attached Figure Description

[0049] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0050] Figure 1 This is a flowchart of the image anomaly detection method of the present invention;

[0051] Figure 2 This invention generates subnetwork test results;

[0052] Figure 3 A sub-network structure block diagram is generated for this invention;

[0053] Figure 4This is a block diagram of the discrimination subnetwork structure of the present invention;

[0054] Figure 5 This is a schematic diagram of some pseudo-anomaly samples constructed based on the cutpaste augmentation method of this invention;

[0055] Figure 6 This is a schematic diagram of some pseudo-anomaly samples constructed based on Berlin noise augmentation according to the present invention;

[0056] Figure 7 This is a detailed framework diagram of the implementation process of the present invention;

[0057] Figure 8 This diagram illustrates the comparison of instance-level anomaly detection AUC between the present invention and existing typical reconstruction algorithms based on generative models on the mvtec_ad dataset and a private dataset.

[0058] Figure 9 The image shows the anomaly localization results of this invention on the mvtec_ad dataset and a private dataset. Detailed Implementation

[0059] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. This will allow for a full understanding of how the present application uses technical means to solve technical problems and achieve technical effects, and to facilitate its implementation.

[0060] Given the limitations of the generative model-based reconstruction anomaly algorithms described in the background section, this invention, in its exploration of generative model-based image anomaly detection algorithms, simulates the addition of pseudo-anomaly samples to positive samples, thereby achieving data augmentation. This effectively alleviates the problem of difficulty in obtaining sufficient anomaly data for model training. Compared with reconstruction anomaly metrics, it is no longer entirely constrained by the quality of positive samples or generative models, transforming unsupervised to supervised learning and turning the image instance-level anomaly detection task into a mature image classification task. Furthermore, by using the gradient-weighted activation mapping method, the region of interest for the anomaly class of the classification network can be directly output, thus achieving anomaly localization.

[0061] Please refer to Figures 1-9 This illustrates a specific implementation of the present embodiment. Based on image augmentation and reconstruction, the constructed pseudo-abnormal samples and the reconstructed samples obtained by anomaly-free reconstruction through the generator subnetwork are concatenated and input into the discriminant subnetwork for training. When detecting real defects, this can generalize well and effectively help detect actual industrial faults.

[0062] Please refer to Figure 1 This embodiment proposes an anomaly detection method based on image augmentation and image reconstruction, including the following steps:

[0063] S1 acquires test images, which are derived from the public dataset mvtec_ad and the private railway component failure dataset.

[0064] The Mvtec_ad dataset simulates real-world industrial production scenarios and is primarily used for unsupervised anomaly detection. It includes 15 object categories, each with 4-6 fault behaviors, covering 73 fault types. During the experiment, the dataset was divided into 15 tasks based on object categories for separate training and testing.

[0065] The private dataset is mainly based on real-world railway component failures and contains two different types of failure detection tasks. During the experiment, the two tasks were trained and tested separately according to the object category.

[0066] S2 inputs the test image into the generator network to achieve anomaly-free reconstruction of the input sample;

[0067] S3 concatenates the test sample and the anomaly-free reconstructed sample obtained by the generator network at the channel and sends them into the discriminator network to generate the anomaly score map corresponding to the input image, thereby achieving instance-level anomaly detection.

[0068] For images whose instance-level test results are classified as anomalous, S4 uses the gradient class weighted activation mapping method and the gradient information of the last convolutional layer in the flow discriminant network to generate a localization map of the anomalous region in the image, thereby achieving anomaly localization.

[0069] Furthermore, in step S2, the method for constructing the sub-network includes the following steps:

[0070] S21 builds and generates a sub-network;

[0071] S22 inputs normal samples, using structural similarity loss L... SSIM Reconstruction error loss L con Feature matching loss L lat and countermeasures against losses L adv The weighted sum is used as the objective function to train and generate subnetworks;

[0072] The S23 test generates a subnetwork. Input normal and abnormal samples, and observe whether any abnormal regions are reconstructed in the output samples. Some test results can be found in [the relevant documentation]. Figure 2 .

[0073] For further details, please refer to Figure 3 In step S21, the generating subnetwork adopts a bow-tie symmetrical structure of encoder-decoder to realize the mutual mapping from the input image to the latent space, thereby reconstructing the input image. Then, an encoder is added as a discriminator to form adversarial training and ensure the quality of the reconstructed image.

[0074] The autoencoder reduces dimensionality and extracts features through a series of convolutional operations, mapping the input image to latent variables in the latent space. The generator structure is similar to the decoder, remapping the latent variables in the latent space back to the image through deconvolution, increasing generalization through batch normalization, and activating with the LeakyReLU activation function. The discriminator structure is similar to the encoder, also extracting features through convolutional operations, and finally mapping the features to a score through a sigmoid activation layer to determine whether the input image is a real image or an image reconstructed by the generator.

[0075] Since the generative subnetwork is trained only on normal samples, it learns only the normal features of the samples. When abnormal samples are input, the generative subnetwork can only reconstruct normal features. Therefore, the local abnormal regions in the abnormal samples will not be reconstructed. In principle, the generative subnetwork can achieve abnormal reconstruction of input samples.

[0076] Furthermore, in step S22, normal samples are input, with structural similarity loss L. SSIM Context loss L con Feature matching loss L lat and countermeasures against losses L adv The weighted sum is used as the objective function to train and generate a sub-network. The specific process includes the following steps:

[0077] S221 Structural similarity measurement between input image I and reconstructed image I r Unlike MSE and PSNR, which measure absolute error, SSIM is a perceptual model that primarily examines three key features of an image: brightness, contrast, and structure. Its calculation method is as follows:

[0078]

[0079] Where μ is the average value, σ 2 For variance, The covariance is c, which is a constant used to maintain stability.

[0080] In this algorithm, the structural similarity loss function is expressed as follows:

[0081]

[0082] Where H and W are the height and width of the input image I, respectively, and N... p SSIM(I,I) represents the number of pixels in the input image I. r ) (i,j) Using image coordinates (i,j) as the center value, input image I and reconstructed image I r The structural similarity value corresponding to the same patch.

[0083] S222 utilizes the adversarial loss L between the generator and discriminator to maximize the reconstruction capability of the normal image I during training. adv This refers to the cross-entropy obtained by the discriminator between the original image and the reconstructed image. This loss ensures that the generator network G reconstructs the normal image as realistically as possible, while the discriminator network distinguishes between the real input sample I and the corresponding fake sample I generated by the generator. r The classification process involves minimizing G and maximizing D during training to achieve a minimum value. G max D V(D,G). Its expression is:

[0084]

[0085] in, Let I represent the distribution of real data p. I The samples obtained from sampling, D(I), D(I) r ) represent the output obtained after the original image passes through the discriminator in the generator network.

[0086] The adversarial loss defined in S223 and S222 forces the model to generate realistic samples, but does not guarantee the learning of contextual information about the input. To explicitly learn this contextual information to fully capture the normally distributed input data, the original input image I and the reconstructed output image I are compared. r L2 loss is applied between them. con This is the reconstruction error loss between the original input image and the network-reconstructed image. This loss function primarily reduces the difference between the original and reconstructed images at the pixel level, measures visual similarity, and ensures that the model can generate images similar to normal sample contexts. The expression is:

[0087] L con =‖II r ||2

[0088] S224 Feature Matching Loss L lat It is the mean square error between the generated image and the output of the discriminator's hidden layer compared to the real image. The goal is to obtain the input original image I and the reconstructed output image I. r The network generates context-aware latent representations for common examples by using the most similar latent representations possible. Using the last convolutional layer of the discriminator D, I and I' are extracted. r Features to reconstruct their latent representation z and Make z = f(I) and The expression is:

[0089]

[0090] Furthermore, in step S3, the test sample and the anomaly-free reconstructed sample obtained by the generator network are concatenated at the channel and fed into the discriminator network to generate the anomaly score corresponding to a single image. The specific process includes the following steps:

[0091] S31 constructs a discriminative subnetwork;

[0092] S32 trains the discriminant subnetwork by concatenating normal samples and constructed pseudo-abnormal samples with the abnormal reconstruction samples generated by the subnetwork, and then feeding them into the discriminant subnetwork for training as positive and negative samples.

[0093] In the S33 testing phase, thresholds are set based on the probability values ​​of all test images output by the sigmoid activation layer of the discrimination network to achieve binary classification of normal and abnormal images.

[0094] For further details, please refer to Figure 4 In step S31, the discriminant subnetwork model is similar to the VGG structure, consisting of 7 blocks, one convolutional layer, and one sigmoid activation layer. Each block contains two sets of convolutional layers, BN layers, ReLU activation, and a max pooling layer.

[0095] Furthermore, in step S32, the normal samples and the constructed pseudo-abnormal samples are concatenated with the abnormal-free reconstructed samples generated by the sub-network, and then used as positive and negative samples to be fed into the discriminative sub-network for training. The specific process includes the following steps:

[0096] S321 uses the cutpaste method to generate two types of pseudo-abnormal samples, CutpasteNormal and CutpasteScar, from normal samples. Please refer to [link / reference]. Figure 5 ;

[0097] The motivation behind CutPaste augmentation is to generate spatial irregularities as a rough approximation of actual defects. To generate different augmentation effects, the CutPasteNormal method first cuts rectangular patches of different sizes and aspect ratios from normal training images, then randomly rotates or jitters the pixel values ​​in the rectangular patches, and finally pastes the rectangular patches back to random positions in normal samples.

[0098] In addition to the CutpasteNormal class which uses a large rectangular patch, for some datasets with very small local anomalies, such as cracks, a long and thin scar-like small rectangular patch is also used to create local irregularities to supplement pseudo-anomaly samples. As a variant of Cutpaste, this forms the CutpasteScar class.

[0099] S322, based on Perlin noise, samples irregular texture fragments from anomalous source images on an anomalous source image dataset unrelated to the distribution of normal input samples to construct perlin-noise-based pseudo-anomaly samples. (Please refer to...) Figure 6 .

[0100] A noisy image is generated using a Berlin noise generator to capture various anomalous shapes. This image is then binarized using a threshold obtained through random uniform sampling to generate anomaly maps. Anomaly texture source images are sampled from a dataset of anomalous source images whose distribution is independent of the input image. These anomalous texture images are then randomly augmented, including with bias, sharpness, daylighting, equalization, brightness variations, color variations, and automatic contrast adjustments. The augmented texture image and normal samples are masked by the anomalous map, forming anomalous parts. These anomalous parts are then mixed with normal samples, making the constructed pseudo-anomaly samples as close as possible to normal samples, with only localized anomalies—that is, generating just-out-distribution pseudo-anomaly samples. This helps tighten the decision boundaries between normal and anomalous samples in the training network.

[0101] S323 randomly selects pseudo-anomaly samples as negative sample inputs;

[0102] The S324 sample and the non-abnormal sample reconstructed by the generator subnetwork are cascaded at the channel and fed into the discriminant subnetwork. This ensures that the input pseudo-abnormality carries more features of the normal sample, making it as close to the normal sample as possible. This increases the training difficulty of the discriminant subnetwork and prevents overfitting due to easy differentiation.

[0103] S325 uses the focal loss function as the objective function for the training phase of the discriminant subnetwork, expressed as:

[0104] FL=-α i (1-p i ) γ y i log(p i )-α i (p i ) γ (1-y i log(1-p) i )

[0105] Among them, y i p represents the label of sample i, with 1 for positive class and 0 for negative class; i This represents the probability that sample i is predicted as positive. γ is the adjustment factor for this loss function; when γ = 0, this expression degenerates into a binary classification cross-entropy loss. α is a balancing factor, set to: α i =1, γ=2.

[0106] In classification networks, the commonly used objective function during the training phase is the Cross Entropy Loss Function. To minimize loss during training, the model must predict the true label with a higher probability. In other words, the cross entropy loss requires the model to be very confident in its predictions. This can lead to overconfidence in the deep learning model, resulting in overfitting and a decrease in the model's generalization ability. Therefore, for the network in this invention, the model may be unable to distinguish between true anomalies during testing.

[0107] Compared to Cross Entropy Loss, focal loss reduces the training loss for samples that are "classified well" or "highly predicted correctly," preventing overfitting. However, it doesn't significantly reduce the loss for "difficult-to-classify examples," such as those with low prediction probabilities. Therefore, in cases of imbalanced data, focal loss directs the model to focus on sparse and difficult-to-distinguish sample classes.

[0108] For images whose instance-level test results are classified as anomalous, S4 uses the gradient class weighted activation mapping method and the gradient information of the last convolutional layer in the flow discriminant network to generate a localization map of the anomalous region in the image, thereby achieving anomaly localization.

[0109] Since the feature map of the last layer in the discriminative network contains the richest category semantic information, it can be visualized. The gradient-weighted activation mapping method uses the gradient information flowing to the last convolutional layer in the discriminative network to understand the importance of each neuron for category recognition. For category c, the gradient-weighted activation mapping method backpropagates the output vector corresponding to that category, calculates the gradient map corresponding to the feature map, and then averages these gradient maps. This average value corresponds to the weight of each feature map. Then the weights With feature map A i We perform weighted summation and finally obtain the final visualized class activation map by applying the ReLU activation function.

[0110]

[0111]

[0112] Please refer to Figure 7 In this embodiment, data preprocessing is first performed, and operations such as flipping, sharpness change, brightness change, color change, and automatic contrast change are performed on normal samples to achieve random augmentation of normal samples, expanding normal samples to 6 times the original size.

[0113] Then, various colors and shapes of local irregular anomalies are introduced into normal samples to simulate pseudo-abnormal samples, generating pseudo-abnormal samples that are close to normal samples but outside the normal sample data distribution. By supplementing the normal samples with anomalous samples, image augmentation is achieved.

[0114] Next, the first stage of training for this network is carried out. This involves training a generative subnetwork consisting of an autoencoder and a generative adversarial network on normal samples. The overall objective function is a weighted sum of structural similarity loss, image reconstruction loss, feature matching loss, and adversarial loss. This ensures that the generative subnetwork learns only the features and distribution of normal samples and possesses only the ability to reconstruct normal features, thus achieving anomaly-free reconstruction of input samples. After training, the first stage of testing is conducted using augmented normal samples, simulated pseudo-anomaly samples, and real anomaly samples.

[0115] Then, the second stage of training of this network is carried out. Each augmented normal sample and its reconstructed sample generated by the generator sub-network are concatenated at the channel as positive samples. Each simulated pseudo-abnormal sample and its non-abnormal sample reconstructed by the generator sub-network are concatenated at the channel as negative samples. They are fed into the discriminator network for training. In order to increase the robustness of the discriminator network and avoid overfitting, focal loss is used as the objective function, so that the model focuses on the rare and difficult-to-distinguish sample categories.

[0116] Finally, real normal and abnormal images from the publicly available industrial dataset mvtec_ad and the private railway parts dataset were used as test sample inputs. Since the last layer of the discriminant subnetwork uses a sigmoid activation function, its output is between 0 and 1 for any input. Therefore, for each input test sample, the output obtained after passing through the sigmoid activation layer does not need further normalization and can be directly used as the instance-level anomaly score for that image. After obtaining the instance-level anomaly scores for all test samples, a threshold is applied to classify normal and abnormal images. For the identified abnormal images, the abnormal regions in the image are located using a gradient-weighted activation mapping method.

[0117] Please refer to Figure 8 The instance-level detection results of the method provided in this embodiment on the mvtec_ad dataset and the private railway parts dataset are as follows: Figure 8 As shown, using three typical algorithms in the field of anomaly detection of reconstructed images based on generative models as baseline algorithms, the anomaly detection method based on image augmentation and image reconstruction proposed in this invention has an anomaly detection AUC that is about 6% and 7% higher than the best baseline algorithms on two major datasets, respectively.

[0118] Please refer to Figure 9The method provided in this embodiment, on the mvtec_ad dataset and the private railway parts dataset, shows the localization maps of some abnormal samples as follows. Figure 9 It can be seen that the local abnormal areas in the sample have been located relatively accurately.

Claims

1. An anomaly detection method based on image augmentation and image reconstruction, characterized in that: The test image is input into the generator network to achieve anomaly-free reconstruction of the input sample; The test sample and the anomaly-free reconstructed sample obtained by the generator network are concatted at the channel and fed into the discriminant network to generate an anomaly score map corresponding to a single image, thereby achieving instance-level anomaly detection. Based on the gradient-weighted activation mapping method, a localization map of the abnormal region is generated using the gradient information of the last convolutional layer in the flow direction discriminant network to achieve anomaly localization. The method for constructing a generative network includes the following steps: Build a generative network; Input normal samples and use structural similarity loss. Reconstruction error loss Feature matching loss and combat losses The weighted sum is used as the objective function to train the generative network; The generative network uses a bow-tie symmetrical structure of encoder and decoder to achieve mutual mapping from the input image to the latent space, thus reconstructing the input image. An encoder is then added as a discriminator to form adversarial training, ensuring the quality of the reconstructed image. The test sample and the anomaly-free reconstructed sample obtained by the generator network are concatenated at the channel level and fed into the discriminator network to generate the anomaly score corresponding to a single image. The specific process includes the following steps: Build a discriminant network; The discriminant network is trained by concatenating normal samples and constructed pseudo-anomaly samples with the anomaly-free reconstructed samples generated by the generator network, and then feeding them as positive and negative samples into the discriminant network for training. The specific process includes the following steps: Based on the cutpaste method, two types of pseudo-abnormal samples, CutpasteNormal and CutpasteScar, are generated from normal samples. Based on Berlin noise, various irregular texture fragments are formed on the abnormal texture source images sampled on the abnormal source image dataset that is unrelated to the distribution of normal input samples to construct perlin-noise-based pseudo-abnormal samples. Randomly select pseudo-anomaly samples as negative sample inputs; The sample and the abnormal sample reconstructed by the generator network are concatenated at the channel and sent to the discriminant network; During the testing phase, thresholds are set based on the probability values ​​of all test images output by the sigmoid activation layer of the discrimination network to achieve binary classification of normal and abnormal images.

2. The anomaly detection method according to claim 1, characterized in that: Structural similarity loss Measuring the input image With reconstructed image The degree of distortion and similarity are expressed as: ， in and These are the input images. Height and width, It is the input image The number of pixels in Based on image coordinates Input image as center value With reconstructed image Structural similarity values ​​corresponding to the same patch; Context loss It is the input image and reconstructed image The reconstruction error loss between the original and reconstructed images is reduced at the pixel level, and its expression is: ； Feature matching loss It is to reconstruct the image With input image The goal is to obtain the input image by taking the mean square error of the output of the discriminator hidden layer in the generator network. and reconstructed image The most similar potential representation is expressed as: ； in It is the input image After passing through the discriminator hidden layer in the generator network The resulting latent representation, It is to reconstruct the image After passing through the discriminator hidden layer in the generator network The resulting latent representation; Combating losses It is the input image and reconstructed image The cross-entropy obtained after discrimination by the discriminator in the generative network aims to maximize the normal image during training. The reconstruction capability is expressed as: ， in, express To obtain from real data distribution The samples obtained from sampling , These represent the output obtained after the input passes through the discriminator in the generator network.

3. The anomaly detection method according to claim 1, characterized in that: The discriminant network model consists of 7 blocks, one convolutional layer, and one sigmoid activation layer. Each block contains two sets of convolutional layers, a batch normalization (BN) layer, a ReLU activation layer, and a max pooling layer.

4. The anomaly detection method according to claim 1, characterized in that: Using the focal loss function as the objective function for the training phase of the discriminant subnetwork, the expression is: ， in, Indicates sample The label is 1 for positive classes and 0 for negative classes; Indicates sample The probability of predicting it as positive; It is the adjustment of the loss function, when When =0, this expression degenerates into binary cross-entropy loss. It is a balancing factor.

Images