Unified anomaly detection method based on multi-source uncertainty mining

Abstract

The application belongs to the technical field of visual anomaly detection, and discloses a unified anomaly detection method based on multi-source uncertainty mining, which comprises the following steps: firstly, training an anomaly segmentation network based on a semi-supervised learning method of multi-source uncertainty mining; and then, performing unified anomaly detection on a to-be-detected image.In the training process of the anomaly segmentation network, a multi-source uncertainty mining network and an anomaly segmentation network are introduced, and based on a cross-attention mechanism, the multi-source uncertainty mining network and the anomaly segmentation network are trained layer by layer in interaction, which can assist the anomaly segmentation network to better focus on the global features of the image and the local features of the anomaly, so that more accurate positioning is realized.

Description

Technical Field

[0001] This invention belongs to the field of visual anomaly detection technology and relates to a unified anomaly detection method based on multi-source uncertainty mining. Background Technology

[0002] Anomaly detection plays a crucial role in computer vision and industrial applications. The main goal of visual anomaly detection is to accurately identify anomalous images and precisely locate anomalous regions. Existing visual anomaly detection methods can be categorized into three paradigms: unsupervised, semi-supervised, and fully supervised. Unsupervised methods can effectively model normal samples, do not rely on prior anomaly information, and can detect unknown forms of anomaly in products. However, their detection performance is not satisfactory. Semi-supervised and fully supervised methods focus on modeling anomalous samples, requiring the specification of anomaly forms and the labeling of anomaly instances. These methods exhibit good detection performance for known anomalies. However, because they are limited to predefined anomaly forms, their detection performance is significantly reduced when faced with unknown anomaly forms.

[0003] In industrial applications, unsupervised methods are typically used to quickly build detection models. Subsequently, by introducing some anomaly samples, a transition to semi-supervised or fully supervised methods is made to further improve anomaly detection performance. However, this strategy presents a challenge: the two stages are disconnected, meaning the model obtained in the unsupervised stage cannot be carried over to subsequent supervised learning. Furthermore, a unified model for anomaly detection across multiple product categories, capable of detecting multiple products with a single model, is more suitable for real-world production environments. Existing unified models often employ feature reconstruction, but this unification only occurs in the feature reconstruction part, lacking a unified decision boundary during the detection process. Specifically, when all product categories are detected using a unified decision boundary, performance is lower compared to detecting each category separately. Summary of the Invention

[0004] The purpose of this invention is to address the aforementioned problems in the existing technology by providing a unified anomaly detection method based on multi-source uncertainty mining, which can identify various anomaly forms without predefinition and unify the decision boundary.

[0005] To achieve the above objectives, the present invention adopts the following technical solutions.

[0006] This invention provides a unified anomaly detection method based on multi-source uncertainty mining, which includes the following steps:

[0007] S1 trains anomaly segmentation networks using a semi-supervised learning method based on multi-source uncertainty mining, including the following steps:

[0008] S11 uses data containing normal and abnormal samples to construct a training set and obtains multi-source pseudo-labels through several pre-trained basic models;

[0009] S12 uses any base model to obtain the reconstructed image of the sample in the training set, and uses the Euclidean distance between the sample and its reconstructed image as a biased difference input to the anomaly segmentation network to obtain the anomaly segmentation image; the anomaly segmentation network includes several sequentially set feature extraction stages, with the output of the previous stage serving as the input of the next stage; at least some of the image features extracted by the feature extraction stages are output to the multi-source uncertainty mining network.

[0010] The S13 multi-source pseudo-label is input to the multi-source uncertainty mining network, combined with image features from the anomaly segmentation network, and the global attention distribution is obtained based on the cross-attention mechanism to generate an uncertainty weight map.

[0011] S14 constructs a loss function and obtains the loss value based on multi-source pseudo-labels, abnormal segmentation images, and uncertainty weight maps;

[0012] S15 uses the loss value to update the parameters of the anomaly segmentation network and the multi-source uncertainty mining network;

[0013] Repeat steps S12-S15 above until the loss function converges to obtain the anomaly segmentation network of the training number.

[0014] S2 performs unified anomaly detection on the image to be detected, including the following steps:

[0015] S21 uses the basic model to obtain the reconstructed image of the image to be detected, and uses the Euclidean distance between the image to be detected and its reconstructed image as the biased difference;

[0016] S22 inputs the biased difference into the anomaly segmentation network to obtain the anomaly segmentation image.

[0017] In step S12 above, the base model is selected from EdgRec, DRAEM, FastFlow, or MSTAD, etc. Each feature extraction stage in the anomaly segmentation network includes several convolutional modules; each convolutional module consists of a convolutional layer, a batch normalization layer, and a ReLU activation function. After each feature extraction stage except the last one, the images pass through a downsampling layer to enter the next feature extraction stage. The last feature extraction stage passes through an upsampling layer and a convolutional module to obtain the anomaly segmentation image. Except for the first feature extraction stage, the image features extracted in the remaining feature extraction stages are output to the multi-source uncertainty mining network.

[0018] In step S13 above, the multi-source uncertainty mining network includes an encoder, several cross-attention modules, and a decoder arranged sequentially; the encoder is used to encode the input image features; the cross-attention module is used to obtain the global attention distribution of the image features output by the corresponding feature extraction stage of the anomaly segmentation network with respect to the multi-source pseudo-labels based on the cross-attention mechanism, and superimpose it with the encoder output or the output of the previous cross-attention module, and then perform patch fusion; the decoder is used to decode the input image features to obtain the uncertainty weight map.

[0019] In a preferred implementation, the encoder includes three convolutional layers and two downsampling layers, with the two downsampling layers located between adjacent convolutional layers. Their main function is to extract features and resize the input feature image. The decoder includes one convolutional layer and one upsampling layer, primarily used to output an uncertainty weight map that accurately represents the pixel-level uncertainty mining content.

[0020] In a preferred implementation, the number of cross-attention modules is the same as the number of feature extraction stages in the multi-source uncertainty mining network. Each cross-attention module includes a cross-attention layer, a feedback neural network, and a patch fusion layer.

[0021] In step S14 above, the loss function constructed is:

[0022]

[0023] In the formula, θ S and θ ∑ These are the trainable parameters for the anomaly segmentation network and the multi-source uncertainty mining network, respectively. This indicates that the i-th pixel obtained based on the m-th pseudo-label uses a pseudo-label. Nonnormalized score S i Calculate the cross-entropy loss; This represents the predicted log-variance. Indicates the use of pseudo tags The uncertainty weight of the i-th pixel is obtained; M represents the number of pseudo-labels, H represents the height of the image, and W represents the width of the image.

[0024] In step S2 above, to improve anomaly detection in the image to be detected, this step also includes:

[0025] S23 performs global average pooling on the abnormal segmentation image and uses the maximum value as the abnormal score of the image to be detected.

[0026] S24 performs weighted fusion of the abnormal segmentation image and the image to be detected to obtain an abnormal heatmap.

[0027] Compared with existing technologies, the unified anomaly detection method based on multi-source uncertainty mining provided by this invention has the following beneficial effects:

[0028] 1) In the training process of the anomaly segmentation network, this invention introduces a multi-source uncertainty mining network and anomaly segmentation network. Based on the cross-attention mechanism, the multi-source uncertainty mining network and anomaly segmentation network are trained interactively layer by layer, which can help the anomaly segmentation network to better focus on the global features of the image and the local features of the anomaly, thereby achieving more accurate localization.

[0029] 2) This invention utilizes a base model obtained through unsupervised learning to acquire biased differences in anomalies; and further improves the performance of the base model in anomaly detection by utilizing unlabeled anomaly samples.

[0030] 3) This invention utilizes an uncertainty-weighted loss function and treats the model's pseudo-labels as a Gibbs distribution within a Bayesian framework. Through this loss function, a multi-source uncertainty mining network and an anomaly segmentation network can be trained simultaneously, enabling the anomaly segmentation network to perform anomaly detection under unified decision boundary conditions.

[0031] 4) Extensive experiments were conducted on the MVTec AD dataset to demonstrate the effectiveness of the proposed method. Furthermore, anomaly detection performance can still be improved when using a self-stimulating method, i.e., without relying on pseudo-labels from other models. Attached Figure Description

[0032] Figure 1 This is a flowchart illustrating the unified anomaly detection method based on multi-source uncertainty mining.

[0033] Figure 2 The original image and its corresponding label are: (a) the original image, (b) the real label, (c) the localization label obtained by the method of the present invention, (d) the pseudo label obtained by the EdgRec model, (e) the pseudo label obtained by the DRAEM model, (f) the pseudo label obtained by the FastFlow model, and (g) the pseudo label obtained by the MSTAD model.

[0034] Figure 3 The images show the original image and anomaly localization maps obtained using different methods. Detailed Implementation

[0035] The technical solutions of various embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0036] Example 1

[0037] This invention proposes a novel semi-supervised anomaly detection strategy that uses a pixel reconstruction model or a feature reconstruction model as the base model. It introduces a Multi-Source Uncertainty Mining Network (MUMNet) and an Anomaly Segmentation Network (ASNet). During training, a cross-attention mechanism allows MUMNet and ASNet to interact layer by layer, helping ASNet better focus on global image features and local anomaly features, thereby achieving more accurate localization.

[0038] The data used in this embodiment is the MVTec AD dataset, which contains 3629 normal images covering 5 different texture categories and 10 unique object categories. The test set includes 467 normal images and 1258 anomalous images. For each anomalous sample in the test set, the dataset provides the image label and segmentation information as ground truth.

[0039] 1. Basic Model

[0040] The base model used in this embodiment is selected from EdgRec, DRAEM, FastFlow, or MSTAD.

[0041] These foundational models are all unsupervised reconstruction models that utilize the Euclidean distance between the input and output of the aforementioned foundational models to represent biased differences in various forms of anomalies. This method eliminates the reliance on manually labeled anomalies. This embodiment builds upon this foundation with a unified anomaly detection method based on multi-source uncertainty mining (MUM-UAD) and employs a progressive learning strategy. However, since unsupervised models learn only from normal samples and cannot utilize anomalous samples, they cannot be reused in subsequent semi-supervised stages. To address this issue, MUM-UAD employs an uncertainty mining learning strategy, using the foundational model as a link between the unsupervised and semi-supervised learning stages, thereby improving detection performance and better adapting to real-world production environments.

[0042] 2. Anomaly Segmentation Network (ASNet)

[0043] The primary goal of the Anomaly Segmentation Network (ASNet) is to generate high-precision anomaly segmentation maps during the final decision-making process. To achieve this goal, this embodiment makes appropriate modifications to the ResNet34 network architecture.

[0044] The anomaly segmentation network (ASNet) provided in this embodiment, such as Figure 1As shown in (a), it includes four sequentially arranged feature extraction stages (Stage 0-Stage 3), with the output of the previous stage serving as the input to the next stage. Each feature extraction stage includes several convolutional modules, with the numbers represented as N0=3, N1=4, N2=6, and N3=3, respectively; the network structure can be adjusted accordingly based on the type of base model used. Each convolutional module consists of a convolutional layer, a batch normalization layer, and a ReLU activation function. After the feature extraction stages other than the last one, the images are passed through a downsampling layer to enter the next feature extraction stage. The last feature extraction stage passes through an upsampling layer and a convolutional module to obtain the anomaly segmentation image. The image features extracted by the remaining feature extraction stages (Stage 1-Stage 3), except for the first feature extraction stage (Stage 0), are output to the multi-source uncertainty mining network.

[0045] For feature reconstruction, ASNet employs an inverted ResNet34 structure. Specifically, all downsampling operations are replaced with upsampling, and the order of all stages is reversed. For pixel reconstruction models or cases without a base model, ASNet follows the standard ResNet34 structure, but replaces the final linear output layer with an upsampling layer to adjust the output size. Here, ASNet is used to extract multi-scale features from the biased differences obtained from the base model, thereby obtaining an unbiased anomaly segmentation map.

[0046] In this embodiment, ASNet is initialized using the pre-trained weights of ResNet34, and the size of the deepest feature map is limited to 14×14.

[0047] 3. Multi-Source Uncertainty Mining Network (MUMNet)

[0048] MUMNet aims to accurately identify the reliability of each pixel by analyzing pseudo-labels and the input image. Specifically, by simultaneously mining the commonalities and differences among multiple pseudo-labels generated by various anomaly detection models, MUMNet can effectively capture reliable label samples.

[0049] The multi-source uncertainty mining network consists of an encoder, several cross-attention modules, and a decoder arranged sequentially.

[0050] The encoder is used to encode the features of the input image. The encoder consists of three convolutional layers and two downsampling layers, with the two downsampling layers located between two adjacent convolutional layers. Its main function is to extract features and change the size of the input feature image.

[0051] The decoder is used to decode the features of the input image to obtain an uncertainty weight map. The decoder consists of a convolutional layer and an upsampling layer, and is mainly used to output an uncertainty weight map that can accurately represent the pixel-level uncertainty mining content.

[0052] The number of cross-attention modules is used to obtain the global attention distribution of image features output from the corresponding feature extraction stage of the anomaly segmentation network for multi-source pseudo-labels based on the cross-attention mechanism. This attention distribution is then superimposed on the encoder output or the output of the previous cross-attention module, and patched and fused. The number of cross-attention modules is the same as the number of feature extraction stages in the multi-source uncertainty mining network; specifically, in this embodiment, there are three cross-attention modules, namely the first cross-attention module to the third cross-attention module. Figure 1 As shown in (a), the cross-attention module includes a cross-attention layer, a forward feedback neural network, and a patch fusion layer. The feature images extracted from Stages 1 to 3 of the anomaly segmentation network are respectively input to the cross-attention layers of the first to third cross-attention modules.

[0053] The cross-attention layer uses linear projection to transform the image features extracted from ASNet into... The features derived from the pseudo-labels are denoted as follows: Linear projection is and Here, i represents the stage number of the cross-attention module, n represents the number of elements in the feature map, and c represents the dimension of the feature. Details of the i-th stage are as follows:

[0054]

[0055] By utilizing a cross-attention mechanism, the global attention distribution of image features to multi-source pseudo-label features at a specific stage can be obtained. Subsequently, a feedforward neural network (FFN) is used to integrate the information, and downsampling is performed through patch fusion to enable new interactions with feature information in the next stage.

[0056] f i+1 =PatchMerging(FFN(f′)) (2);

[0057] This module enhances the ability of features at different scales in ASNet to perceive anomalous information and helps MUMNet generate more robust uncertainty maps.

[0058] In this embodiment, MUMNet is initialized using a uniform distribution, and the cross-attention module interacts with feature maps of three scales: 56×56, 28×28, and 14×14.

[0059] Based on the above explanation, the unified anomaly detection method based on multi-source uncertainty mining provided in this embodiment includes the following steps:

[0060] S1 trains anomaly segmentation networks using a semi-supervised learning method based on multi-source uncertainty mining, including the following steps:

[0061] S11 constructs a training set using data containing both normal and abnormal samples, and obtains multi-source pseudo-labels through several pre-trained basic models.

[0062] Since the MVTec AD dataset only contains anomalous samples in the test set, this embodiment redefines the dataset. The original MVTec AD training set contains only normal samples and is called the "basic training set," while the test set, which contains anomalous samples and a small number of normal samples, is called the "basic test set." Then, a number of anomalous and normal samples are selected from each product category in the "basic test set" to form the "new training set." The remaining samples in the "basic test set" form the "new test set." Simultaneously, the resolution of all input images is adjusted to 224×224.

[0063] The aforementioned base models were pre-trained using the "base training set," and the parameter weights of each base model were preserved. Furthermore, the threshold value covering 98% of the scores in the anomaly score graphs generated by the four models (EdgRec, DRAEM, FastFlow, MSTAD) trained on the "base training set" was selected.

[0064] The pre-trained basic model described above is used to detect anomalies on the "new training set" to generate anomaly score maps. These anomaly score maps are then binarized according to a threshold and used as multi-source pseudo-labels.

[0065] S12 uses any base model to obtain the reconstructed images of samples in the training set, and uses the Euclidean distance between the sample and its reconstructed image as a biased difference input to the anomaly segmentation network to obtain anomaly segmentation images.

[0066] Meanwhile, the feature images extracted in the feature extraction stages Stage 1-Stage 3 of the anomaly segmentation network are output to the corresponding cross-attention module of the multi-source uncertainty mining network.

[0067] The S13 multi-source pseudo-label is input to the multi-source uncertainty mining network, combined with image features from the anomaly segmentation network, and the global attention distribution is obtained based on the cross-attention mechanism to generate an uncertainty weight map.

[0068] This step utilizes the multi-source uncertainty mining network given earlier to generate an uncertainty weight graph.

[0069] The output of the last cross-attention module is decoded by the decoder to obtain an uncertainty weight map that can accurately represent the pixel-level uncertainty mining content.

[0070] S14 constructs a loss function and obtains the loss value based on multi-source pseudo-labels, abnormal segmentation images, and uncertain weight maps.

[0071] In M anomaly detection methods (corresponding to M pseudo-labels), the pseudo-label of pixel i generated by the m-th method is represented as: Where c=1 represents the abnormal region and c=0 represents the normal region. The corresponding uncertainty weighting diagram is composed of The pseudo-label is modeled as a random variable y that follows a Gibbs distribution under Bayesian theory. When the outlier score is normalized using the Softmax function, the probability distribution of y can be calculated as follows:

[0072]

[0073] Given observed pseudo-labels The negative log-likelihood can be further derived as follows:

[0074]

[0075] in, This indicates that the i-th pixel obtained based on the m-th pseudo-label uses a pseudo-label. Nonnormalized score S i The cross-entropy loss is calculated (given by the anomalous segmentation image). However, in practice, it can be calculated by predicting the log-variance. Increase numerical stability during training. Indicates the use of pseudo tags The uncertainty weight of the i-th pixel is obtained.

[0076] Therefore, the loss can be restated as follows:

[0077]

[0078] In the formula, θ S and θ ∑ These are the trainable parameters of the anomaly segmentation network and the multi-source uncertainty mining network, respectively; Equation (5) is called the uncertainty-weighted loss, which helps the joint learning of the entire network. Therefore, based on Equations (4) and (5), and extended to all M pseudo-labels, the final loss function is obtained as follows:

[0079]

[0080] Under the supervision of Equation (6), MUMNet and ASNet are trained together to enhance MUMNet’s ability to capture detailed anomaly feature information, thereby improving ASNet’s performance in anomaly detection.

[0081] S15 uses the loss value to update the parameters of the anomaly segmentation network and the multi-source uncertainty mining network.

[0082] Repeat steps S12-S15 above until the loss function converges to obtain the anomaly segmentation network of the training number.

[0083] The convergence of the loss function can be determined by changes in the loss value or by setting an upper limit on the number of iterations. When the loss value tends to stabilize or the upper limit on the number of iterations is reached, the loss function is considered to have converged, and training ends.

[0084] In this embodiment, the learning rates of both MUMNet and ASNet networks are set to 1e-4, and the Adam optimizer is used for end-to-end joint training. During training, the "new training set" is divided into 6 batches for batch training of the networks. Five different random seeds are used for evaluation. The entire training process requires approximately 200 epochs on a GeForce 4080 GPU.

[0085] S2 performs unified anomaly detection on the image to be detected.

[0086] In this embodiment, samples from the "new test set" are used as the images to be detected, and unified anomaly detection is performed in the following steps:

[0087] S21 uses the base model to obtain the reconstructed image of the image to be detected, and uses the Euclidean distance between the image to be detected and its reconstructed image as the biased difference.

[0088] S22 inputs the biased difference into the anomaly segmentation network to obtain the anomaly segmentation image.

[0089] S23 performs global average pooling on the abnormal segmentation image and uses the maximum value as the abnormal score of the image to be detected; the obtained abnormal score can be used for abnormal detection.

[0090] In this embodiment, the size of the average pooling used to calculate the outlier score is set to 80.

[0091] S24 performs weighted fusion of the abnormal segmentation image and the image to be detected to obtain an abnormal heatmap; the obtained abnormal heatmap can be used for anomaly localization.

[0092] In this embodiment, the anomaly heatmap is obtained according to α·I+(1-α)·S′, where I represents the input image, S′ represents the score map of the output anomaly segmentation image S after colormap transformation, and α represents the weight coefficient; specifically, it can be referenced Figure 1 As shown in the image.

[0093] In this embodiment, experiments were conducted on four unsupervised anomaly detection reconstruction models: models designed for single-class detection (DRAEM and EdgRerc), models designed for multi-class detection (UniAD and MSTAD), and two semi-supervised anomaly detection models (DRA and BGAD). Specifically, the unsupervised models were trained on the "base training set" of MVTec AD, retaining their weights. In the subsequent semi-supervised stage, one of the unsupervised models was selected as the base model, and combined with the MUM-UAD strategy, MUMNet and ASNet were further trained using a "new training set." Before and after fusing MUM-UAD, the unsupervised models were tested against a unified decision boundary on a "new test set."

[0094] Table 1 compares the results of the unsupervised basic model before and after introducing the MUMAD-UAD strategy under the condition of a unified decision boundary.

[0095]

[0096] “Det.” represents image-level AUROC, “Loc.” represents pixel-level AUROC, the MUM-UAD strategy is represented by “ours”, and the best result in each before-and-after comparison is highlighted in bold.

[0097] AUROC represents the area under the receiver operating characteristic curve, used to evaluate the performance of proposed anomaly detection and localization methods.

[0098] Aupro represents the area under the remainder overlap curve, used to measure anomaly location assessment and ensure that anomalies of all sizes are treated equally during the assessment process.

[0099] Table 2 shows the comparison results of semi-supervised methods under the condition of a unified decision boundary.

[0100]

[0101] “BM” represents the base model (e.g., MSTAD), while “ours” represents the MUM-UAD strategy.

[0102] Ours w / o BM indicates that this embodiment lacks a base model. In this case, the MUM-UAD strategy is still used to train MUMNet and ASNet with the help of pseudo-labels, and then the trained ASNet is tested.

[0103] Best results are highlighted in bold.

[0104] As shown in Table 1, when an unsupervised model is combined with the MUM-UAD strategy, it can be applied to semi-supervised tasks, resulting in varying degrees of performance improvement in detection. Specifically, when the base model focuses on a single-class anomaly detection task, combining this strategy demonstrates improvements exceeding 10% in image-level AUC or pixel-level AUC. This enhancement highlights the effectiveness of MUM-UAD in enabling single-class anomaly detection models to address multi-class anomaly problems. Conversely, when the base model focuses on multi-class anomaly detection tasks, the method of this invention exhibits superior performance, highlighting its applicability to multi-class scenarios. We further investigated the UniAD and MSTAD models. Figure 2 and Figure 3 As shown, by incorporating the strategy of this invention, these models establish more accurate segmentation boundaries for anomalies, effectively reducing the focus on non-anomaly regions and thus significantly improving detection performance. This further confirms that integrating MUM-UAD into ASNet can significantly enhance its local anomaly perception capability and help improve its overall image understanding.

[0105] For semi-supervised models (DRA and BGAD), they are trained directly on a "new training set" and tested on a "new test set." The method of this invention outperforms traditional semi-supervised anomaly detection methods, as shown in Table 2, especially in the absence of an unsupervised base model. Furthermore, the method of this invention achieves even better results when utilizing the knowledge provided by the base model, further validating its effectiveness.

[0106] Those skilled in the art will recognize that the embodiments described herein are intended to help the reader understand the principles of the invention, and should be understood that the scope of protection of the invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific modifications and combinations based on the technical teachings disclosed in this invention without departing from the spirit of the invention, and these modifications and combinations are still within the scope of protection of this invention.

Claims

1. A unified anomaly detection method based on multi-source uncertainty mining, characterized in that, Includes the following steps: S1 trains anomaly segmentation networks using a semi-supervised learning method based on multi-source uncertainty mining, including the following steps: S11 uses data containing normal and abnormal samples to construct a training set and obtains multi-source pseudo-labels through several pre-trained basic models; S12 uses any base model to obtain the reconstructed image of the sample in the training set, and uses the Euclidean distance between the sample and its reconstructed image as a biased difference input to the anomaly segmentation network to obtain the anomaly segmentation image; the anomaly segmentation network includes several sequentially set feature extraction stages, with the output of the previous stage serving as the input of the next stage; at least some of the image features extracted by the feature extraction stages are output to the multi-source uncertainty mining network. The S13 multi-source pseudo-label is input to the multi-source uncertainty mining network, combined with image features from the anomaly segmentation network, and the global attention distribution is obtained based on the cross-attention mechanism to generate an uncertainty weight map. S14 constructs a loss function and obtains the loss value based on multi-source pseudo-labels, abnormal segmentation images, and uncertainty weight maps; S15 uses the loss value to update the parameters of the anomaly segmentation network and the multi-source uncertainty mining network; Repeat steps S12-S15 above until the loss function converges to obtain the anomaly segmentation network of the training number. S2 performs unified anomaly detection on the image to be detected, including the following steps: S21 uses the basic model to obtain the reconstructed image of the image to be detected, and uses the Euclidean distance between the image to be detected and its reconstructed image as the biased difference; S22 inputs the biased difference into the anomaly segmentation network to obtain the anomaly segmentation image.

2. The unified anomaly detection method based on multi-source uncertainty mining according to claim 1, characterized in that, In step S12, the base model is selected from EdgRec, DRAEM, FastFlow, or MSTAD.

3. The unified anomaly detection method based on multi-source uncertainty mining according to claim 1, characterized in that, Each feature extraction stage in the anomaly segmentation network includes several convolutional modules; each convolutional module consists of a convolutional layer, a batch normalization layer, and a ReLU activation function.

4. The unified anomaly detection method based on multi-source uncertainty mining according to claim 3, characterized in that, After the feature extraction stages except for the last one, the image is passed through a downsampling layer to enter the next feature extraction stage. The last feature extraction stage passes through an upsampling layer and a convolution module to obtain the abnormal segmentation image.

5. The unified anomaly detection method based on multi-source uncertainty mining according to any one of claims 1 to 4, characterized in that, In step S13, the multi-source uncertainty mining network includes an encoder, several cross-attention modules, and a decoder arranged sequentially. The encoder is used to encode the input image features. The cross-attention module is used to obtain the global attention distribution of the image features output by the corresponding feature extraction stage of the anomaly segmentation network with respect to the multi-source pseudo-labels based on the cross-attention mechanism, and superimpose it with the encoder output or the output of the previous cross-attention module, and perform patch fusion. The decoder is used to decode the input image features to obtain the uncertainty weight map.

6. The unified anomaly detection method based on multi-source uncertainty mining according to claim 5, characterized in that, The encoder includes three convolutional layers and two downsampling layers, with the two downsampling layers located between two adjacent convolutional layers; the decoder includes one convolutional layer and one upsampling layer.

7. The unified anomaly detection method based on multi-source uncertainty mining according to claim 5, characterized in that, The cross-attention module includes a cross-attention layer, a forward feedback neural network, and a patch fusion layer.

8. The unified anomaly detection method based on multi-source uncertainty mining according to claim 5, characterized in that, In step S14, the constructed loss function is: In the formula, θ S and θ ∑ These are the trainable parameters for the anomaly segmentation network and the multi-source uncertainty mining network, respectively. This indicates that the i-th pixel obtained based on the m-th pseudo-label uses a pseudo-label. Nonnormalized score S i Calculate the cross-entropy loss; This represents the predicted log-variance. Indicates the use of pseudo tags The uncertainty weight of the i-th pixel is obtained; M represents the number of pseudo-labels, H represents the height of the image, and W represents the width of the image.

9. The unified anomaly detection method based on multi-source uncertainty mining according to claim 1, characterized in that, Step S2 also includes: S23 performs global average pooling on the abnormal segmentation image and uses the maximum value as the abnormal score of the image to be detected. S24 performs weighted fusion of the abnormal segmentation image and the image to be detected to obtain an abnormal heatmap.

10. The unified anomaly detection method based on multi-source uncertainty mining according to claim 8, characterized in that, Step S2 also includes: S23 performs global average pooling on the abnormal segmentation image and uses the maximum value as the abnormal score of the image to be detected. S24 performs weighted fusion of the abnormal segmentation image and the image to be detected to obtain an abnormal heatmap.

Images