Abnormity detection method and device based on multi-task self-supervision feature reconstruction

Abstract

The invention provides an anomaly detection method and device based on multi-task self-supervision feature reconstruction, and belongs to the field of anomaly detection.The anomaly detection method comprises the steps that n feature tensors of different dimensions of an input sample are extracted, the extracted feature tensors are input into respective corresponding projection layers, and the projection layers are trained in a multi-task learning mode; the distance between embedding of the normal sample features is minimized through a feature compactness task, so that a projection layer is guided to closely gather the normal features, and compact transmission of the normal features is realized; a pseudo-abnormal sample is simulated through simple noise, reconstruction and contrast loss are minimized through an abnormal feature suppression task, a projection layer is guided to recover normal features from pseudo-abnormal features, and effective suppression of the abnormal features is achieved; and performing feature reconstruction by using a compact feature tensor which is extracted by the projection layer after multi-task learning and is irrelevant to the anomaly to obtain n reconstructed normal features, and further identifying the anomaly condition in the sample by calculating the similarity between the sample feature tensor and the reconstructed feature tensor to obtain an anomaly detection result. According to the invention, the capability of detecting and positioning abnormity can be effectively improved.

Description

Technical Field

[0001] The present invention belongs to the field of image anomaly detection, and in particular relates to an anomaly detection method and device based on multi-task self-supervised feature reconstruction. Background Art

[0002] Anomaly detection is a core technology for identifying and locating anomalies in many fields, especially in the fields of industrial product inspection, medical diagnosis, and video surveillance. It is crucial for timely identifying potential problems, ensuring product quality, improving the efficiency of medical services, and maintaining public safety. Methods to achieve this goal include both traditional supervised learning techniques and modern self-supervised and unsupervised learning strategies. However, due to the limited number of negative samples and the lack of sufficient prior information, traditional supervised methods face challenges in identifying unseen anomalies. In contrast, existing self-supervised methods, although relying on reconstruction techniques, are not expressive enough in distinguishing abnormal features, and the reconstruction accuracy of normal data needs to be improved. These factors together limit the overall performance of anomaly detection. Summary of the invention

[0003] The purpose of the present invention is to propose an anomaly detection method and device based on multi-task self-supervised feature reconstruction, which can effectively improve the detection and positioning capabilities of anomalies.

[0004] The present invention is achieved through the following technical solutions:

[0005] An anomaly detection method based on multi-task self-supervised feature reconstruction includes the following steps:

[0006] Step S1: Extract features of n different dimensions of the input sample Among them, i = 1, 2, ..., n, and a projection layer is integrated after each extracted feature. The projection layer is trained using the self-supervised optimal transmission loss method in the feature compactness task. The optimal transmission distance between normal features is determined by minimizing the debiased Sinkhorn divergence between probability metrics to achieve compact transmission of normal features.

[0007] Step S2: Introduce simplex noise into the input sample to simulate pseudo-abnormal samples and extract features of n different dimensions from the samples with pseudo-abnormalities. By minimizing the reconstruction loss and contrast loss in the abnormal feature suppression task, the projection layer is guided to restore the normal feature representation from these pseudo-abnormal features, thereby achieving effective suppression of abnormal features.

[0008] Step S3: Reconstruct features using the features extracted from the projection layer after multi-task training to obtain n reconstructed normal features F R i, respectively, the features obtained from the input sample and the reconstructed normal features are compared for similarity, the abnormality of each pixel in the sample feature tensor is obtained, and the abnormal pixels are accumulated to obtain the abnormality detection result.

[0009] Furthermore, in step S1, according to the formula Extract features of n different dimensions of the input sample Among them, I represents the input image sample, represents the i-th feature extractor network, which is a pre-trained WideResNet-50 network, c i 、c i 、h i 、w i They represent the number of channels, height, and width of the tensor extracted by the i-th feature extractor network, respectively.

[0010] Furthermore, in the step S1, a projection layer is directly integrated after each extracted feature, and the projection layer is trained using the self-supervised optimal transmission loss method in the feature compactness task to achieve compact transmission of normal features, which specifically includes the following steps:

[0011] Step S11: The integrated projection layer corresponding to each extracted feature consists of convolution Conv(g), two-dimensional instance normalization InNorm(g) and activation function LReLU(g), expressed as According to the formula Get the feature tensor after projection in, is the projection layer network corresponding to the i-th feature;

[0012] Step S12: In the feature compactness task, according to the self-supervised optimal transmission loss formula The projection layer is trained to achieve compact transfer of normal features, where S ò,ρ (g) Debiased Sinkhorn divergence, and are the p-th and q-th feature outputs in the i-th feature projection layer, ò and ρ are the regularization parameter and exponential term respectively, σ(g) represents Sigmoid normalization, i = [1, ..., l], p, q = [1, ..., m];

[0013] Furthermore, in step S2, the abnormal feature suppression task realizes effective suppression of abnormal features, which specifically includes the following steps:

[0014] Step S21: According to the formula Simplex noise is randomly added to the input image as a perturbation term to obtain input samples with pseudo anomalies Among them, I kis the kth input sample, λ is the weighting coefficient of the noise, ξ is the noise matrix of the same size as the input sample, and noise is added only in specific areas;

[0015] Step S22: Using formula Extract features of n different dimensions from samples with pseudo-anomalies Using the formula Get the pseudo-anomaly feature tensor after projection

[0016] Step S23: In the abnormal feature suppression task, we first need to use the formula of minimizing the reconstruction loss function: Train the projection layer and accelerate the ability of the projection layer to suppress abnormal information during reasoning by optimizing this learning objective, where cos(g) is the cosine similarity function.

[0017] Step S24: In the abnormal feature suppression task, the contrast loss function formula composed of cosine embedding loss is then used: The projection layer is trained to push away abnormal information from the normal space, forcing the projection layer to focus on learning deeper representations of normal features, where f is the margin of the cosine embedding and max(g) is the maximum function;

[0018] Furthermore, in step S3, n reconstructed normal features are obtained. Specifically include:

[0019] Step S31: According to the formula Project the n different dimensional features output by the projection layer into the same low-dimensional space and align the dimensions to obtain the fused feature tensor Among them, SubConv n (g) represents the down-sampling convolution operation on n features, ReLU(g) represents the activation function, where c′, h′, and w′ represent the number of channels, height, and width of the fused feature tensor, respectively;

[0020] Step S32: According to the formula Perform feature reconstruction to obtain reconstructed normal features Among them, R i (g) represents the i-th feature reconstruction network, which uses the same The symmetric but opposite structural design consists of symmetric residual decoding blocks built from deconvolutional layers.

[0021] Furthermore, the step S3, obtaining the abnormality detection result specifically includes:

[0022] Step S33: According to the formula Get the i-th sample feature tensor The abnormality of each pixel in A i (h,w), where 1≤h≤h i , 1≤w≤w i ;

[0023] Step S34: In order to locate the anomaly, the score map is formulated as a pixel-by-pixel accumulation of n-layer anomalies to obtain a scalar loss function. Learn and get the optimal A i The anomaly graph represented;

[0024] Step S35: According to the formula Get the final anomaly score map A p , where Φ(g) represents the upsampling operation.

[0025] Furthermore, the input sample is a data set MVTec.

[0026] The present invention is also achieved through the following technical solutions:

[0027] An anomaly detection device based on multi-task self-supervisory feature reconstruction includes a processor and a memory, wherein program instructions are stored in the memory, and the processor calls the program instructions in the memory to implement the following modules:

[0028] Feature compactness task module: extract features of n different dimensions of the input sample Among them, i = 1, 2, ..., n, and a projection layer is integrated after each extracted feature. The projection layer is trained using the self-supervised optimal transmission loss method in the feature compactness task. The optimal transmission distance between normal features is determined by minimizing the debiased Sinkhorn divergence between probability metrics to achieve compact transmission of normal features.

[0029] Abnormal feature suppression task module: used to introduce simplex noise into the input sample to simulate pseudo-abnormal samples and extract features of n different dimensions from samples with pseudo-abnormalities By minimizing the reconstruction loss and contrast loss in the abnormal feature suppression task, the projection layer is guided to restore the normal feature representation from these pseudo-abnormal features, thereby achieving effective suppression of abnormal features.

[0030] Feature reconstruction and detection module: Use the features extracted by the projection layer after multi-task training to reconstruct features and obtain n reconstructed normal features The features obtained from the input sample and the reconstructed normal features are compared for similarity, the abnormality of each pixel in the sample feature tensor is obtained, and the abnormal pixels are accumulated to obtain the abnormality detection result.

[0031] The present invention has the following beneficial effects:

[0032] The present invention first extracts the multi-dimensional features of the sample and integrates a projection layer after each feature to perform multi-task learning. Then, the distance between normal features is optimized by combining the debiased Sinkhorn divergence in the optimal transmission theory with the self-supervision method in the feature compactness task, thereby achieving the compactness of feature representation and enhancing the learning of normal patterns; secondly, simplex noise is added to the input sample to generate pseudo-abnormal samples to simulate abnormal situations, and then the projection layer is prompted to restore normal feature representations from features containing pseudo-abnormalities by minimizing the reconstruction loss and contrast loss in the abnormal feature suppression task, thereby achieving effective suppression of abnormal features. Finally, the multi-scale features extracted by the projection layer of multi-task training are used for feature reconstruction, and then the abnormal situation of each pixel in the sample feature tensor is obtained according to the similarity between the sample feature tensor and the reconstructed feature tensor, and finally the abnormal pixels are accumulated to form an abnormal detection result. In this way, it is possible to focus on the subspace that may contain abnormalities in a complex feature space, accurately identify abnormal features and determine their exact locations, thereby effectively improving the detection and positioning capabilities of abnormalities. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The present invention is further described in detail below with reference to the accompanying drawings.

[0034] Figure 1 Flow chart of the method of the present invention.

[0035] Figure 2 It is a framework diagram of the device of the present invention. DETAILED DESCRIPTION

[0036] like Figure 1 As shown in Figure 1, anomaly detection based on multi-task self-supervised feature reconstruction includes the following steps:

[0037] Step S1: Extract features of n different dimensions of the input sample Among them, i = 1, 2, ..., n, and a projection layer is integrated after each extracted feature. The projection layer is trained using the self-supervised optimal transmission loss method in the feature compactness task. The optimal transmission distance between normal features is determined by minimizing the debiased Sinkhorn divergence between probability metrics to achieve compact transmission of normal features.

[0038] Specifically, according to the formula Extract features of n different dimensions of the input sample Among them, I represents the input image sample, represents the i-th feature extractor network, which is a pre-trained WideResNet-50 network, c i 、c i 、h i 、w iThey represent the number of channels, height, and width of the tensor extracted by the i-th feature extractor network, respectively.

[0039] In this embodiment, the input sample is the MVTec dataset, which is a collection of real-world industrial images with 5 textures and 10 object classes for industrial anomaly detection. The training set consists of 3629 normal images. The test set consists of 1725 normal and abnormal sample images, and each category contains multiple anomalies for testing;

[0040] A projection layer is directly integrated after each extracted feature, and the projection layer is trained using the self-supervised optimal transmission loss method in the feature compactness task to achieve compact transmission of normal features. The specific steps include:

[0041] Step S11: The integrated projection layer corresponding to each extracted feature consists of convolution Conv(g), two-dimensional instance normalization InNorm(g) and activation function LReLU(g), expressed as According to the formula Get the feature tensor after projection in, is the projection layer network corresponding to the i-th feature;

[0042] Step S12: In the feature compactness task, according to the self-supervised optimal transmission loss formula The projection layer is trained to achieve compact transfer of normal features, where S ò,ρ (g) Debiased Sinkhorn divergence function, and are the p-th and q-th feature outputs in the i-th feature projection layer, ò and ρ are the regularization parameter and exponential term respectively, σ(g) represents Sigmoid normalization, i = [1, ..., l], p, q = [1, ..., m];

[0043] also, and Debiased Sinkhorn divergence Defined as: Among them, OT ε,ρ (g) represents the optimal transportation distance function, and As an example, the optimal transportation distance between Where f and g represent the optimal variables to be found to maximize the value of the objective function, is the outer product operation, represents element-by-element addition, and C represents and The cost of transmission between.

[0044] Step S2: Introduce simplex noise into the input sample to simulate pseudo-abnormal samples and extract features of n different dimensions from the samples with pseudo-abnormalities. By minimizing the reconstruction loss and contrast loss in the abnormal feature suppression task, the projection layer is guided to restore the normal feature representation from these pseudo-abnormal features, thereby achieving effective suppression of abnormal features.

[0045] Specifically, simplex noise is introduced into the input sample to simulate pseudo-abnormal samples, and features of n different dimensions are extracted from samples with pseudo-abnormalities. And through the abnormal feature suppression task, the abnormal features can be effectively suppressed;

[0046] The specific steps include:

[0047] Step S21: According to the formula Simplex noise is randomly added to the input image as a perturbation term to obtain input samples with pseudo anomalies Among them, I k is the kth input sample, λ is the weighting coefficient of the noise, ξ is the noise matrix of the same size as the input sample, and noise is added only in specific areas;

[0048] Step S22: Using formula Extract features of n different dimensions from samples with pseudo-anomalies Using the formula Get the pseudo-anomaly feature tensor after projection

[0049] Step S23: In the abnormal feature suppression task, we first need to use the formula of minimizing the reconstruction loss function: Train the projection layer and accelerate the ability of the projection layer to suppress abnormal information during reasoning by optimizing this learning objective, where cos(g) is the cosine similarity function.

[0050] Step S24: In the abnormal feature suppression task, the contrast loss function formula composed of cosine embedding loss is then used: The projection layer is trained to push away abnormal information from the normal space, forcing the projection layer to focus on learning deeper representations of normal features, where f is the margin of the cosine embedding and max(g) is the maximum function.

[0051] Step S3: Reconstruct features using the features extracted from the projection layer after multi-task training to obtain n reconstructed normal features. The features obtained from the input sample and the reconstructed normal features are compared for similarity, the abnormality of each pixel in the sample feature tensor is obtained, and the abnormal pixels are accumulated to obtain the abnormality detection result;

[0052] First, we need to get the reconstructed n normal features Specifically include:

[0053] Step S31: According to the formula Project the n different dimensional features output by the projection layer into the same low-dimensional space and align the dimensions to obtain the fused feature tensor Among them, SubConv n (g) represents the down-sampling convolution operation on n features, ReLU(g) represents the activation function, where c′, h′, and w′ represent the number of channels, height, and width of the fused feature tensor, respectively;

[0054] Step S32: According to the formula Perform feature reconstruction to obtain reconstructed normal features Among them, R i (g) represents the i-th feature reconstruction network, which uses the same The symmetric but opposite structural design consists of symmetric residual decoding blocks built from deconvolutional layers.

[0055] Secondly, we need to detect anomalies, including:

[0056] Step S33: According to the formula Get the i-th sample feature tensor The abnormality of each pixel in A i (h,w), where 1≤h≤h i , 1≤w≤w i ;

[0057] Step S34: In order to locate the anomaly, the score map is formulated as The pixel-by-pixel accumulation of n-layer anomalies gives us the scalar loss function Using this scalar loss function to learn, we get the optimal A i The anomaly graph represented;

[0058] Step S35: According to the formula Get the final anomaly score map A p , where A p Represents an upsampling operation.

[0059] The model is trained by setting reasonable parameters, and the final result uses the area under the receiver operating characteristic curve (AUROC) and each region overlap (AUPRO), the most widely used evaluation indicators in anomaly detection, as evaluation indicators.

[0060] Correspondingly, the anomaly detection device based on multi-task self-supervisory feature reconstruction includes a processor and a memory, wherein the memory stores program instructions, such as Figure 1 As shown, the processor calls the program instructions in the memory to implement the following modules:

[0061] Feature compactness task module: extract features of n different dimensions of the input sample A projection layer is integrated after each extracted feature, and the projection layer is trained using the self-supervised optimal transmission loss method in the feature compactness task. The optimal transmission distance between normal features is determined by minimizing the debiased Sinkhorn divergence between probability metrics, thus achieving compact transmission of normal features.

[0062] More specifically, if Figure 2 As shown in the figure, the feature compactness task module is mainly composed of a feature extractor, a projection layer, an activation function, and a normalization unit. In this module, the feature extractor extracts a multi-scale and comprehensive representation of the input sample; the projection layer mainly processes the corresponding multi-scale features from the feature extractor and maps them into a compact feature representation, which is the processing core of multiple tasks; the convolution layer deeply fuses the predicted multi-scale features; the activation function introduces nonlinear transformations to give the model the ability to express complex nonlinear relationships; the normalization unit stabilizes the network training process and accelerates the convergence speed by standardizing the input, which also helps to alleviate overfitting and improve the generalization performance of the model.

[0063] Abnormal feature suppression task module: used to introduce simplex noise into the input sample to simulate pseudo-abnormal samples and extract features of n different dimensions from samples with pseudo-abnormalities By minimizing the reconstruction loss and contrast loss in the abnormal feature suppression task, the projection layer is guided to restore the normal feature representation from these pseudo-abnormal features, thereby achieving effective suppression of abnormal features.

[0064] Feature reconstruction and detection module: Use the features extracted by the projection layer after multi-task training to reconstruct features and obtain n reconstructed normal features The features obtained from the input sample and the reconstructed normal features are compared for similarity, the abnormality of each pixel in the sample feature tensor is obtained, and the abnormal pixels are accumulated to obtain the abnormality detection result.

[0065] The above description is only a preferred embodiment of the present invention, and therefore cannot be used to limit the scope of implementation of the present invention. That is, equivalent changes and modifications made according to the scope of the patent application of the present invention and the contents of the specification should still fall within the scope covered by the patent of the present invention.

Claims

1. An anomaly detection method and device based on multi-task self-supervised feature reconstruction, Features: The steps include: Step S1: Extract features of n different dimensions of the input sample Among them, i = 1, 2, ..., n, and a projection layer is integrated after each extracted feature. The projection layer is trained using the self-supervised optimal transmission loss method in the feature compactness task. The optimal transmission distance between normal features is determined by minimizing the debiased Sinkhorn divergence between probability metrics to achieve compact transmission of normal features. Step S2: Introduce simplex noise into the input sample to simulate pseudo-abnormal samples and extract features of n different dimensions from the samples with pseudo-abnormalities. By minimizing the reconstruction loss and contrast loss in the abnormal feature suppression task, the projection layer is guided to restore the normal feature representation from these pseudo-abnormal features, thereby achieving effective suppression of abnormal features. Step S3: Reconstruct features using the features extracted from the projection layer after multi-task training to obtain n reconstructed normal features. The features obtained from the input sample and the reconstructed normal features are compared for similarity, the abnormality of each pixel in the sample feature tensor is obtained, and the abnormal pixels are accumulated to obtain the abnormality detection result.

2. According to claim 1, an anomaly detection method based on multi-task self-supervised feature reconstruction, Features: In step S1, according to the formula Extract features of n different dimensions of the input sample Among them, I represents the input image sample, represents the i-th feature extractor network, which is a pre-trained WideResNet-50 network, c i 、c i 、h i 、w i They represent the number of channels, height, and width of the tensor extracted by the i-th feature extractor network, respectively.

3. The anomaly detection method based on multi-task self-supervised feature reconstruction according to claim 2, Features: In step S1, a projection layer is directly integrated after each extracted feature, and the projection layer is trained using the self-supervised optimal transmission loss method in the feature compactness task to achieve compact transmission of normal features, which specifically includes the following steps: Step S11: The integrated projection layer corresponding to each extracted feature consists of convolution Conv(g), two-dimensional instance normalization InNorm(g) and activation function LReLU(g), expressed as According to the formula Get the feature tensor after projection in, is the projection layer network corresponding to the i-th feature; Step S12: In the feature compactness task, according to the self-supervised optimal transmission loss formula The projection layer is trained to achieve compact transfer of normal features, where S ò,ρ (g) Debiased Sinkhorn divergence, and are the p-th and q-th feature outputs in the i-th feature projection layer, ò and ρ are the regularization parameter and the exponential term, respectively, σ(g) represents Sigmoid normalization, i = [1, ..., l], p, q = [1, ..., m].

4. The anomaly detection method based on multi-task self-supervised feature reconstruction according to claim 3, Features: In step S2, the abnormal feature suppression task realizes effective suppression of abnormal features, which specifically includes the following steps: Step S21: According to the formula Simplex noise is randomly added to the input image as a perturbation term to obtain input samples with pseudo anomalies Among them, I k is the kth input sample, λ is the weighting coefficient of the noise, ξ is the noise matrix of the same size as the input sample, and noise is added only in specific areas; Step S22: Using formula Extract features of n different dimensions from samples with pseudo-anomalies Using the formula Get the pseudo-anomaly feature tensor after projection Step S23: In the abnormal feature suppression task, we first need to use the formula of minimizing the reconstruction loss function: Train the projection layer and accelerate the ability of the projection layer to suppress abnormal information during reasoning by optimizing this learning objective, where cos(g) is the cosine similarity function. Step S24: In the abnormal feature suppression task, the contrast loss function formula composed of cosine embedding loss is then used: The projection layer is trained to push away abnormal information from the normal space, forcing the projection layer to focus on learning deeper representations of normal features, where f is the margin of the cosine embedding and max(g) is the maximum function.

5. The anomaly detection method based on multi-task self-supervised feature reconstruction according to claim 4, Features: In step S3, the reconstructed n normal features are obtained Specifically include: Step S31: According to the formula Project the n different dimensional features output by the projection layer into the same low-dimensional space and align the dimensions to obtain the fused feature tensor Among them, SubConv n (g) represents the down-sampling convolution operation on n features, ReLU(g) represents the activation function, where c′, h′, and w′ represent the number of channels, height, and width of the fused feature tensor, respectively; Step S32: According to the formula Perform feature reconstruction to obtain reconstructed normal features Among them, R i (g) represents the i-th feature reconstruction network, which uses the same The symmetric but opposite structural design consists of symmetric residual decoding blocks built from deconvolutional layers.

6. The anomaly detection method based on multi-task self-supervised feature reconstruction according to claim 5, Features: In step S3, obtaining the abnormality detection result specifically includes: Step S33: According to the formula Get the i-th sample feature tensor The abnormality of each pixel in A i (h,w), where 1≤h≤h i , 1≤w≤w i ; Step S34: In order to locate the anomaly, the score map is formulated as a pixel-by-pixel accumulation of n-layer anomalies to obtain a scalar loss function Using this scalar loss function to learn, we get the optimal A i The anomaly graph represented; Step S35: According to the formula Get the final anomaly score map A p , where A p Represents an upsampling operation.

7. The anomaly detection method based on multi-task self-supervised feature reconstruction according to any one of claims 1 to 6, Features: The input sample is the data set MVTec.

8. An anomaly detection device based on multi-task self-supervised feature reconstruction, Features: It includes a processor and a memory, the memory stores program instructions, and the processor calls the program instructions in the memory to implement the following modules: Feature compactness task module: extract features of n different dimensions of the input sample Among them, i = 1, 2, ..., n, and a projection layer is integrated after each extracted feature. The projection layer is trained using the self-supervised optimal transmission loss method in the feature compactness task. The optimal transmission distance between normal features is determined by minimizing the debiased Sinkhorn divergence between probability metrics to achieve compact transmission of normal features. Abnormal feature suppression task module: used to introduce simplex noise into the input sample to simulate pseudo-abnormal samples and extract features of n different dimensions from samples with pseudo-abnormalities By minimizing the reconstruction loss and contrast loss in the abnormal feature suppression task, the projection layer is guided to restore the normal feature representation from these pseudo-abnormal features, thereby achieving effective suppression of abnormal features. Feature reconstruction and detection module: Use the features extracted by the projection layer after multi-task training to reconstruct features and obtain n reconstructed normal features The features obtained from the input sample and the reconstructed normal features are compared for similarity, the abnormality of each pixel in the sample feature tensor is obtained, and the abnormal pixels are accumulated to obtain the abnormality detection result.

Images