Abnormality detection method, device, electronic device and computer readable storage medium

Abstract

The present disclosure provides an anomaly detection method, device, electronic device and computer-readable storage medium; the method comprises: using a feature extraction network to perform feature processing of different scales on an image to be detected to obtain a multi-scale feature sequence; reconstructing based on the multi-scale feature sequence through a reconstruction network to obtain a reconstructed feature sequence; wherein the reconstruction network is obtained by training with positive samples; determining the feature difference at each pixel position according to the reconstructed feature sequence and the multi-scale feature sequence; determining the anomaly score at each pixel position based on the feature difference; and drawing an anomaly detection image corresponding to the image to be detected based on the anomaly score. The present disclosure can improve the accuracy of anomaly detection.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to an anomaly detection method, device, electronic device, and computer-readable storage medium. Background Art

[0002] In recent years, deep learning algorithms have made great progress in various fields and have been implemented in many industrial visual inspection fields. However, deep learning algorithms often require a large amount of high-quality annotated data. In scenarios where samples accumulated in actual production lines are needed to train industrial visual inspection models, abnormal samples are often extremely scarce. The anomaly detection model trained using conventional positive sample deep learning methods has poor recognition ability for abnormal samples, thereby reducing the accuracy of anomaly detection. In addition, current anomaly detection models are usually based on image features extracted by deep convolutional networks for detection and processing, lacking a deep understanding of the semantic information in the image, which further reduces the accuracy of anomaly detection. Summary of the invention

[0003] The embodiments of the present disclosure provide an anomaly detection method, an apparatus, an electronic device, and a computer-readable storage medium, which can improve the accuracy of anomaly detection.

[0004] The technical solution of the embodiment of the present disclosure is implemented as follows:

[0005] The present disclosure provides an anomaly detection method, including:

[0006] Using the feature extraction network, the image to be detected is processed with features of different scales to obtain a multi-scale feature sequence;

[0007] Reconstructing based on the multi-scale feature sequence through a reconstruction network to obtain a reconstructed feature sequence; wherein the reconstruction network is obtained by training with positive samples;

[0008] Determining a feature difference at each pixel position according to the reconstructed feature sequence and the multi-scale feature sequence;

[0009] Based on the feature difference, determining an abnormality score at each pixel position;

[0010] Based on the anomaly score, an anomaly detection image corresponding to the image to be detected is drawn.

[0011] In the above method, the step of performing feature processing of different scales on the image to be detected to obtain a multi-scale feature sequence includes:

[0012] Extracting features of different scales from the image to be detected to obtain features of multiple different scales;

[0013] Fusing the multiple features of different scales to obtain multi-scale features;

[0014] Performing shape transformation on the multi-scale features to obtain the multi-scale feature sequence.

[0015] In the above method, the reconstruction network includes: an encoding network and a decoding network;

[0016] The reconstructing based on the multi-scale feature sequence through the reconstruction network to obtain the reconstructed feature sequence includes:

[0017] Based on the encoding network, encoding the multi-scale feature sequence to obtain an encoding sequence;

[0018] The decoding network is used to perform decoding reconstruction based on the encoding sequence to obtain the reconstructed feature sequence.

[0019] In the above method, determining the feature difference at each pixel position according to the reconstructed feature sequence and the multi-scale feature sequence includes:

[0020] The feature difference at each corresponding sequence position in the reconstructed feature sequence and the multi-scale feature sequence is calculated to obtain the feature difference at each pixel position.

[0021] In the above method, determining the abnormality score at each pixel position based on the feature difference includes:

[0022] The feature differences at each pixel position are averaged by channel to obtain an abnormality score at each pixel position.

[0023] In the above method, the step of drawing the abnormality detection image corresponding to the image to be detected based on the abnormality score includes:

[0024] When the abnormality score at each pixel position is less than or equal to the score threshold, the pixel value corresponding to the pixel position is plotted as the first value;

[0025] When the abnormality score at each pixel position is greater than the score threshold, the pixel value corresponding to the pixel position is drawn as the second value; until the pixel positions of the image to be detected are drawn, the abnormality detection image is obtained.

[0026] In the above method, before reconstructing the multi-scale feature sequence and the preset query word sequence based on the reconstruction network to obtain the reconstructed feature sequence, the method further includes:

[0027] Obtain positive samples and a preset initial query word sequence;

[0028] Based on the positive samples and the initial query word sequence, combined with a preset positive sample loss function, the initial reconstruction network is trained until the reconstruction network is obtained;

[0029] The preset positive sample loss function is used to evaluate the reconstruction performance of the initial reconstruction network on the positive sample according to the feature difference obtained by reconstructing the positive sample using the initial query word sequence.

[0030] In the above method, the initial reconstruction network is trained based on the positive sample and the initial query word sequence in combination with a preset positive sample loss function until the reconstruction network is obtained, including:

[0031] Performing feature processing on the positive sample to obtain a multi-scale positive sample feature sequence;

[0032] Based on the initial reconstruction network, reconstructing the multi-scale positive sample feature sequence and the initial query word sequence to obtain a reconstructed positive sample feature sequence;

[0033] Determining a positive sample feature difference sequence according to the reconstructed positive sample feature sequence and the multi-scale positive sample feature sequence;

[0034] Calculating the current positive sample loss based on the positive sample feature difference sequence and a preset positive sample loss function;

[0035] Based on the current positive sample loss and the preset loss threshold, the initial reconstruction network is trained and adjusted until the reconstruction network having a final loss less than the preset loss threshold is obtained.

[0036] In the above method, the preset positive sample loss function includes: a regression loss function, and the calculation of the current positive sample loss based on the positive sample feature difference sequence and the preset positive sample loss function includes:

[0037] The positive sample feature difference sequence is subjected to gradient smoothing processing by means of the regression loss function to obtain the current positive sample loss.

[0038] In the above method, the method further comprises:

[0039] During the training process of the initial reconstruction network, the initial query word sequence is iteratively updated until the final loss is less than the preset loss threshold, thereby obtaining the preset query word sequence.

[0040] The present disclosure provides an abnormality detection device, including:

[0041] Feature extraction network, used to process features of different scales on the image to be detected to obtain a multi-scale feature sequence;

[0042] A reconstruction network is used to reconstruct based on the multi-scale feature sequence to obtain a reconstructed feature sequence; the reconstruction network is obtained by training with positive samples;

[0043] A determination unit, configured to determine a feature difference at each pixel position according to the reconstructed feature sequence and the multi-scale feature sequence;

[0044] A scoring unit, configured to determine an abnormality score at each pixel position based on the feature difference;

[0045] A detection unit is used to draw an abnormality detection image corresponding to the image to be detected based on the abnormality score.

[0046] In the above device, the feature extraction network is also used to extract features of different scales on the image to be detected to obtain features of multiple scales; fuse the multiple features of different scales to obtain multi-scale features; and perform shape transformation on the multi-scale features to obtain the multi-scale feature sequence.

[0047] In the above device, the reconstruction network includes: an encoding network and a decoding network; the encoding network is used to encode the multi-scale feature sequence to obtain a coded sequence;

[0048] The decoding network is used to perform decoding reconstruction based on the encoding sequence to obtain the reconstructed feature sequence.

[0049] In the above device, the determining unit is further used to calculate the feature difference at each corresponding sequence position in the reconstructed feature sequence and the multi-scale feature sequence to obtain the feature difference at each pixel position.

[0050] In the above device, the scoring unit is further used to perform channel averaging on the feature differences at each pixel position to obtain an abnormality score at each pixel position.

[0051] In the above device, the detection unit is also used to draw the pixel value corresponding to each pixel position as a first value when the abnormality score at the pixel position is less than or equal to the scoring threshold; and to draw the pixel value corresponding to the pixel position as a second value when the abnormality score at each pixel position is greater than the scoring threshold; until the pixel positions of the image to be detected are drawn, the abnormality detection image is obtained.

[0052] In the above device, the anomaly detection device also includes a training unit, which is used to reconstruct the multi-scale feature sequence and the preset query word sequence based on the reconstruction network, and obtain positive samples and a preset initial query word sequence before obtaining the reconstructed feature sequence; based on the positive samples and the initial query word sequence, combined with a preset positive sample loss function, the initial reconstruction network is trained until the reconstructed network is obtained; wherein the preset positive sample loss function is used to evaluate the reconstruction performance of the initial reconstruction network on the positive sample based on the feature difference obtained by reconstructing the positive sample using the initial query word sequence.

[0053] In the above device, the training unit is also used to perform feature processing on the positive sample to obtain a multi-scale positive sample feature sequence; based on the initial reconstruction network, the multi-scale positive sample feature sequence and the initial query word sequence are reconstructed to obtain a reconstructed positive sample feature sequence; according to the reconstructed positive sample feature sequence and the multi-scale positive sample feature sequence, a positive sample feature difference sequence is determined; based on the positive sample feature difference sequence and a preset positive sample loss function, the current positive sample loss is calculated; based on the current positive sample loss and the preset loss threshold, the initial reconstruction network is trained and adjusted until the reconstructed network whose final loss is less than the preset loss threshold is obtained.

[0054] In the above device, the preset positive sample loss function includes: a regression loss function, and the training unit is further used to perform gradient smoothing processing on the positive sample feature difference sequence through the regression loss function to obtain the current positive sample loss.

[0055] In the above method, the training unit is further used to iteratively update the initial query word sequence during the training process of the initial reconstruction network until the final loss is less than the preset loss threshold, thereby obtaining the preset query word sequence.

[0056] An embodiment of the present disclosure provides an electronic device, including:

[0057] A memory for storing executable instructions;

[0058] The processor is used to implement the anomaly detection method provided by the embodiment of the present disclosure when executing the executable instructions stored in the memory.

[0059] The embodiment of the present disclosure provides a computer-readable storage medium storing executable instructions for causing a processor to execute and implement the anomaly detection method provided by the embodiment of the present disclosure.

[0060] The embodiments of the present disclosure have the following beneficial effects:

[0061] The reconstruction network obtained through positive sample training realizes the understanding and modeling of the feature distribution of normal samples. Therefore, when there are abnormal pixels in the image to be detected, the reconstructed features after the abnormal pixels are reconstructed by the reconstruction network will be quite different from the original abnormal pixel features, that is, the feature differences of the reconstruction of abnormal pixels are large. Therefore, the feature differences after reconstruction can be used as anomaly scores, and the pixel-level detection results of the image to be detected can be obtained through anomaly scores, thereby improving the accuracy of anomaly detection. In addition, by reconstructing multi-scale features together with preset query words, it is possible to capture deeper semantic information in the image to be detected, thereby further improving the accuracy of anomaly detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 is an optional flowchart of the anomaly detection method provided by the embodiment of the present disclosure;

[0063] Figure 2 is a schematic diagram of a process of multi-scale feature extraction of an image to be detected provided by an embodiment of the present disclosure;

[0064] Figure 3 is a schematic diagram of a feature extraction network training process provided by an embodiment of the present disclosure;

[0065] Figure 4 is an optional structural diagram of a reconstruction network of an encoder-decoder structure provided in an embodiment of the present disclosure;

[0066] Figure 5 It is a schematic diagram of a process of performing shape transformation on a reconstructed feature sequence to obtain a reconstructed feature provided by an embodiment of the present disclosure;

[0067] Figure 6 It is a schematic diagram of a process of performing anomaly detection on an image to be detected by a reconstruction network provided by an embodiment of the present disclosure to obtain an anomaly score for each pixel position;

[0068] Figure 7 is a schematic diagram of a process of performing anomaly detection on an image to be detected and obtaining an anomaly detection image provided by an embodiment of the present disclosure;

[0069] Figure 8 is a schematic diagram of the effect of anomaly detection images provided by an embodiment of the present disclosure;

[0070] Fig. 9 It is an optional flowchart of the training method of the reconstruction network provided by the embodiment of the present disclosure;

[0071] Fig.10 It is an optional flowchart of the training method of the reconstruction network provided by the embodiment of the present disclosure;

[0072] Fig.11 Schematic diagram of the training loss calculation process corresponding to the complete positive samples provided in the embodiment of the present disclosure;

[0073] Fig.12 is an optional flowchart of the anomaly detection method provided by the embodiment of the present disclosure;

[0074] Fig.13 is a structural schematic diagram of an abnormality detection device provided by an embodiment of the present disclosure;

[0075] Fig.14 It is a structural schematic diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0076] In order to make the objectives, technical solutions and advantages of the present disclosure clearer, the present disclosure will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present disclosure. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present disclosure.

[0077] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0078] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present disclosure described herein can be implemented in an order other than that illustrated or described herein.

[0079] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present disclosure belongs. The terms used herein are only for the purpose of describing the embodiments of the present disclosure and are not intended to limit the present disclosure.

[0080] See also Figure 1 , Figure 1 is an optional flow chart of the anomaly detection method provided by the embodiment of the present disclosure, which is combined with Figure 1 The steps shown are explained.

[0081] S101, using a feature extraction network, performing feature processing of different scales on the image to be detected to obtain a multi-scale feature sequence.

[0082] The anomaly detection method in the embodiments of the present disclosure is applicable to scenarios where anomaly detection is performed through artificial intelligence image processing. For example, by performing anomaly detection on product images on a production line, abnormal products such as defective products are found; or, anomaly detection is performed on batch images to find abnormal images that do not meet preset overall specifications, etc. The specific selection is made based on the actual situation and is not limited by the embodiments of the present disclosure.

[0083] In some embodiments, the electronic device can obtain images of objects to be inspected, such as products on a production line, as images to be inspected through an image acquisition device, and perform abnormality detection on the images to be inspected through the method of the embodiments of the present disclosure to determine whether there is any abnormality in the objects to be inspected.

[0084] In the disclosed embodiment, the electronic device may use a feature extraction network to extract features of different scales on the image to be detected, and obtain features of multiple scales of the image to be detected at multiple scales; and fuse the features of multiple scales to obtain multi-scale features; and perform shape transformation on the multi-scale features, and convert the multi-dimensional feature information into a sequence form to obtain a multi-scale feature sequence.

[0085] In some embodiments, the electronic device can use a CNN network containing multiple feature extraction layers, such as a backbone network of ResNet-34, as a feature extraction network, perform multi-scale feature extraction on the image to be detected, and use the bottom-level feature extraction layer of ResNe-t34 to obtain the apparent features of the image to be detected, such as color, boundary division, contrast, brightness, etc., and use the high-level extraction layer of ResNet-34 to output the high-level semantic information of the image to be detected; the electronic device uses the apparent features of the image to be detected and the high-level semantic information as features of multiple scales, performs feature fusion, and obtains multi-scale features.

[0086] For example, Figure 2 As shown, the electronic device uses the pre-trained CNN network as a feature extraction network to extract features of four scales on the image I to be detected, and obtains features B(I) of four scales, where: B(I)=f1,f2,f3,f4. Here, i represents the i-th network layer of the CNN network. Figure 2 It can be seen that the scale features extracted by different feature extraction layers have different sizes. The electronic device can first adjust the features of different sizes to the same size, such as uniformly adjusting them to H×W, and then perform feature fusion on the four features after the adjustment to obtain multi-scale features, as shown in formula (1):

[0087]

[0088] In formula (1), R represents the resizing operation, and C = ∑ i C i The feature channel representing the multi-scale feature f. It can be seen that in the disclosed embodiment, the features of multiple scales obtained by different feature extraction layers have different levels of receptive fields and are very sensitive to abnormal areas of different degrees that may exist in the image to be detected. Therefore, anomaly detection based on multi-scale features is more helpful to improve the accuracy of anomaly detection.

[0089] In the disclosed embodiment, the electronic device can perform shape transformation on the multi-scale features, thereby converting the multi-scale features into a sequence form as an input for a subsequent reconstruction network. Figure 2 As shown, for a multi-scale feature f with a dimension of H×W×C, the electronic device can merge the dimensions of H and W corresponding to each channel to achieve a shape transformation of the multi-scale feature, and obtain H×W C-dimensional multi-scale feature vectors as a multi-scale feature sequence.

[0090] In some embodiments, the electronic device can use a public training data set and combine it with a task head network of a general visual task, such as a classification network, a target detection network, an image segmentation network, etc. to pre-train the initial feature extraction network to obtain a feature extraction network. Figure 3 As shown, the initial feature extraction network is iteratively trained using images from a public data set. In each round of training, the initial feature extraction network is used to extract multi-scale features from the images of the current round of training to obtain multi-scale image features; the task head network is used to perform image processing based on the multi-scale image features, such as classification prediction, to obtain the prediction results of the current round; the prediction results of the current round are compared with the true values ​​of the images of the current round of training, and the current round loss is calculated based on the comparison results using the loss function, and the parameters of the initial feature extraction network and the task head network are adjusted based on the current round loss. The electronic device iteratively trains the initial feature extraction network and the task head network through the above method until the preset training conditions are met to obtain the feature extraction network. The task head network is discarded, and the feature extraction network is used for the multi-scale feature extraction processing in the disclosed embodiment.

[0091] Here, the electronic device can use a pre-trained task head network and an initial feature extraction network for training, or can use an initial task head network and an initial feature extraction network for joint training. The specific selection is made according to actual conditions and is not limited in the embodiments of the present disclosure.

[0092] In some embodiments, the multi-scale feature sequence can be a feature word sequence containing any form of word vectors, such as a word2vec sequence, a GloVe sequence, a one-hot encoding sequence, etc. The specific selection is based on actual conditions and is not limited in the embodiments of the present disclosure.

[0093] S102. Reconstructing based on the multi-scale feature sequence through a reconstruction network to obtain a reconstructed feature sequence; wherein the reconstruction network is obtained by training with positive samples.

[0094] In the embodiment of the present disclosure, the positive sample is a sample in which all pixels in the image are normal and there are no abnormal pixels. For example, it can be a normal product image or a qualified product image on the production line. The specific selection is based on the actual situation and is not limited in the embodiment of the present disclosure.

[0095] From a statistical point of view, the features of abnormal pixels in the sample image can be regarded as outliers in the feature distribution of normal pixels. Therefore, the disclosed embodiment can simulate the feature distribution of normal pixels so that the initial reconstruction network can learn the feature distribution of normal pixels when reconstructing the features of normal pixels. Therefore, the reconstruction effect of the reconstruction process of the features of abnormal pixels will be greatly reduced, widening the gap between the reconstructed features of normal pixels and abnormal pixels, so that the initial reconstruction network has the ability to identify abnormal situations in the feature distribution of normal pixels, that is, it has the ability to detect abnormal samples.

[0096] In the disclosed embodiment, the electronic device may use a sequence-to-sequence conversion network model as a reconstruction network, use the multi-scale feature sequence in the above sequence form as guidance information, and reconstruct a preset query word sequence in combination with the guidance information to obtain a reconstructed feature sequence, thereby realizing anomaly detection based on the reconstructed feature sequence.

[0097] In some embodiments, the reconstruction network may include an encoding network and a decoding network; illustratively, the reconstruction network may be implemented by a neural network with an encoder-decoder (Transformer) structure. Among them, Transformer is a deep neural network with a global attention mechanism, which first appeared in the field of natural language processing. In recent years, Transformer has gradually been widely used in computer vision tasks due to its powerful representation ability. At present, the application of Transformer has not been involved in anomaly detection scenarios, and the disclosed embodiments can extend the application of Transformer to anomaly detection scenarios, using Transformer's global attention capture of the input word sequence to enhance the reconstruction network's deep understanding of semantic information, so that the reconstruction network can realize feature reconstruction on the premise of understanding the deep semantic features of the image.

[0098] In the disclosed embodiment, the electronic device encodes the multi-scale feature sequence based on the encoding network in the reconstruction network to obtain the encoding sequence; and decodes and reconstructs the encoding sequence output by the encoding network through the decoding network, so as to obtain the reconstructed feature sequence corresponding to the original multi-scale feature sequence. Here, in the process of processing the input multi-scale feature sequence, the reconstruction network can use the characteristics of the positive samples learned in the pre-training stage, such as the characteristic distribution of the positive samples, to encode and decode the multi-scale feature sequence, so that the reconstructed feature sequence obtained by the reconstruction is close to the original multi-scale feature sequence. The training method of the reconstruction network will be explained in the following steps.

[0099] In some embodiments, if the multi-scale feature sequence f is directly reconstructed through the Transformer structure network s After reconstruction, a single-input and single-output reconstructed feature sequence is obtained. in, R represents the reconstruction operation. However, this single-input and single-output method easily leads to the reconstruction network being able to learn only a simple identity mapping, and it is difficult to understand the task objectives that need to be learned from the input image features. In this way, regardless of whether it is a normal pixel or an abnormal pixel in the image, the model can only perform a simple identity mapping on the input image features for output, and cannot learn the ability to identify abnormal pixels. Therefore, the embodiment of the present disclosure introduces a learnable query word sequence, and uses the query word sequence and the multi-scale feature sequence as the dual input of the reconstruction network to perform feature reconstruction, and obtain a reconstructed feature sequence.

[0100] In some embodiments, the reconstructed feature sequence obtained by the dual-input single-output method in the embodiments of the present disclosure can be expressed as Wherein, q represents a preset query word sequence, and the preset query word sequence and the multi-scale feature sequence have the same feature dimension.

[0101] In the disclosed embodiment, the query word sequence is a learnable vector sequence. The electronic device can initialize the vectors in the query word sequence during the startup phase of the model training of the reconstruction network, and iteratively update the query word sequence during the training of the reconstruction network, and finally obtain a preset query word sequence.

[0102] In the disclosed embodiment, the electronic device encodes the multi-scale feature sequence based on the encoder part in the reconstruction network of the Transformer structure to obtain the encoding sequence. The electronic device can use the encoding sequence as guidance information for the decoder part in the reconstruction network to guide the decoding process of the encoder, so as to decode and reconstruct the preset query word sequence through the decoder based on the encoding sequence to obtain the reconstructed feature sequence.

[0103] In some embodiments, the reconstruction network of the Transformer structure can be as follows Figure 4 As shown, it includes an encoder obtained by stacking N encoding modules and a decoder obtained by stacking N decoding modules. Each of the N encoding modules includes a multi-head self-attention layer, a feed forward network (Feed Forward Network, FFN) and a residual and normalization (ADD & Normalize) layer. Exemplarily, the feed forward network can be a fully connected neural network.

[0104] For example, Figure 4 As shown, the electronic device will Figure 4 The multi-scale feature sequence obtained in the above is input into the encoder, and is converted into Q vector, K vector and V vector for attention calculation through three preset weight matrices, and then the attention value is calculated according to the Q vector, K vector and V vector through the multi-head self-attention layer, and the residual calculation and normalization processing are performed according to the calculated attention value and the original Q vector, K vector and V vector through the residual and normalization layer, and the processing result is linearly and nonlinearly transformed through the feedforward network, and then the residual and normalization layer is passed to obtain the encoding sequence corresponding to the multi-scale feature sequence, and the encoding sequence is input into the decoder. Then, in the decoder, the attention value calculation and residual normalization processing are performed on the preset query word sequence through the multi-head self-attention layer and the residual and normalization layer, and the encoding sequence output by the encoder is merged with the processing result of the preset query word sequence, and the attention value calculation is performed again to obtain the merged attention value, and the residual and normalization processing is performed in combination with the processing result of the preset query word sequence and the merged attention value, and then the reconstructed feature sequence is obtained through the feedforward network and the residual normalization network.

[0105] In some embodiments, Figure 4 As shown, the electronic device can also use the permutation invariance of the Transformer to add position information to each layer of the multi-head self-attention layer. For example, it can be sinusoidal position information, so as to further improve the semantic understanding ability of the reconstruction network through position information. The specific selection is made according to the actual situation, and the embodiments of the present disclosure are not limited.

[0106] Here, the electronic device uses the multi-scale feature sequence extracted from the image to be detected as guiding information, and performs feature reconstruction of the learnable query word sequence in a dual-input manner, so that the reconstruction network can combine the semantics in the query word sequence and perform decoding and reconstruction based on deep semantic understanding, thereby improving the accuracy of the reconstructed features and further improving the accuracy of anomaly detection.

[0107] S103: Determine the feature difference at each pixel position according to the reconstructed feature sequence and the multi-scale feature sequence.

[0108] In the disclosed embodiment, the multi-scale feature sequence represents the original feature of each pixel position in the image to be detected, and the reconstructed feature sequence represents the reconstructed feature at each pixel position obtained by reconstructing based on the original feature. The electronic device can determine the feature difference at each pixel position, that is, the reconstruction effect of the reconstruction network on each pixel point of the image to be detected, by comparing the difference between the reconstructed feature sequence and the multi-scale feature sequence.

[0109] In some embodiments, the electronic device may calculate the feature difference at each corresponding sequence position between the reconstructed feature sequence and the multi-scale feature sequence, compare the multi-scale feature sequence with the reconstructed feature sequence in sequence form, and obtain a feature difference sequence, wherein each feature difference in the feature difference sequence corresponds to each pixel position. The electronic device may obtain the feature difference at each pixel position through the feature difference sequence.

[0110] In some embodiments, Figure 5 As shown, Figure 5 The multi-scale feature processing and feature reconstruction processes in are consistent with the descriptions in S101 and S102, and will not be repeated here; Figure 5 As shown, after the electronic device obtains the reconstructed feature sequence, it can also perform shape transformation on the reconstructed feature sequence to obtain a matrix form, such as a reconstructed feature in the form of a three-dimensional feature map. Exemplarily, the electronic device can restore H×W C-dimensional reconstructed feature sequences to H×W×C-dimensional reconstructed features. The electronic device obtains the C-dimensional pixel reconstruction features corresponding to each pixel position based on the H and W dimensions in the reconstructed features. Similarly, the electronic device obtains the multi-scale pixel features at each pixel position based on the H×W×C-dimensional multi-scale features corresponding to the multi-scale feature sequence. The electronic device calculates the difference between the pixel reconstruction features at each pixel position and the multi-scale pixel features to obtain the feature difference at each pixel position.

[0111] S104: Determine an abnormality score at each pixel position based on feature differences.

[0112] In the disclosed embodiment, the electronic device may determine an abnormality score at each pixel position based on the reconstruction effect at each pixel position represented by the feature difference sequence, where the abnormality score is used to represent whether the corresponding pixel point is an abnormal pixel point.

[0113] In some embodiments, the electronic device may calculate the abnormality score at each pixel position by performing channel averaging on the feature difference at each pixel position. For example, as shown in formula (2), as follows:

[0114]

[0115] In formula (2), d'(u,i) is the feature difference at each pixel position, C is the number of feature channels corresponding to each feature difference, and s(u) is the anomaly score obtained by averaging the feature channel dimension of the feature difference at each pixel position.

[0116] In the embodiment of the present disclosure, the electronic device may also adopt other calculation methods according to actual conditions to evaluate the reconstruction effect of the pixel point according to the feature difference at each pixel position to obtain an abnormality score. The specific selection is based on actual conditions and is not limited by the embodiment of the present disclosure.

[0117] In some embodiments, based on Figure 2 and Figure 4 , the electronic device uses the reconstruction network to detect anomalies in the image to be detected, and the process of obtaining the anomaly score of each pixel position can be as follows Figure 6 shown.

[0118] S105 . Based on the anomaly score, draw an anomaly detection image corresponding to the image to be detected.

[0119] In the disclosed embodiment, the electronic device can distinguish and draw normal pixels and abnormal pixels based on the abnormality score, and obtain an abnormality detection image corresponding to the image to be detected. In this way, the abnormal area can be automatically marked in the abnormality detection image.

[0120] In some embodiments, the electronic device may compare the abnormality score of each pixel position with a preset score interval, and determine the target pixel value corresponding to each pixel position according to the target score interval in which the abnormality score falls and the correspondence between the preset score interval and the preset pixel value. The electronic device draws an abnormality detection image according to the target pixel value corresponding to each pixel position.

[0121] Exemplarily, when the abnormality score at each pixel position is less than or equal to the score threshold, it means that the reconstruction feature of the pixel point reconstructed by the reconstruction network is close to the original feature, and the reconstruction effect is good. The electronic device draws the pixel value corresponding to the pixel position as the first value to characterize the pixel point as a normal pixel point; when the abnormality score at each pixel position is greater than the score threshold, it means that the reconstruction feature of the pixel point reconstructed by the reconstruction network is greatly different from the original feature, and the reconstruction effect is not good. The electronic device draws the pixel value corresponding to the pixel position as the second value to characterize the pixel point as an abnormal pixel point; until the pixel position of the image to be detected is drawn, the abnormal detection image is obtained. In this way, normal pixels and abnormal pixels can be marked in the abnormal detection image respectively by different pixel values, so as to realize the abnormal detection of the image to be detected.

[0122] In some embodiments, Figure 7 As shown, the electronic device can extract features of different scales from the image to be detected through a deep convolutional neural network to obtain a multi-scale feature sequence, add feature information to the multi-scale feature sequence, and embed the multi-scale feature sequence with added position information into a latent feature space with a global attention mechanism through an encoder in the reconstruction network for encoding to obtain a coding sequence. And through the decoder in the reconstruction network, the preset query word vector with added position information is decoded using the coding sequence as the guiding information of the decoding process to obtain a feature reconstruction sequence. The electronic device obtains the feature difference of each pixel position in the image to be detected by the difference between the multi-scale feature sequence and the feature reconstruction sequence, and determines the abnormality score at each pixel position based on the feature difference of each pixel position, obtains the corresponding pixel value at each pixel position and draws it, and fuses the drawn image with the original image to be detected to obtain an abnormal detection image. For example, as Figure 8 As shown, based on Figure 6 According to the abnormality detection method, the electronic device performs the abnormality detection on the image to be detected 70 to obtain an abnormality detection image as shown in 71. It can be seen that Figure 8 Through different display effects, the abnormal image area of ​​the seed germination part is marked in the seed image. Thus, the function of abnormal detection of products is realized.

[0123] It can be understood that in the disclosed embodiment, the reconstruction network obtained through positive sample training realizes the understanding and modeling of the feature distribution of normal samples. Therefore, when there are abnormal pixels in the image to be detected, the reconstructed features after the abnormal pixels are reconstructed by the reconstruction network will be greatly different from the original abnormal pixel features, that is, the feature differences of the reconstruction of the abnormal pixels are large. Therefore, the feature differences after reconstruction can be used as an abnormality score, and the pixel-level detection results for the image to be detected can be obtained through the abnormality score, thereby improving the accuracy of anomaly detection. In addition, by reconstructing multi-scale features together with preset query words, it is possible to capture deeper semantic information in the image to be detected, thereby further improving the accuracy of anomaly detection.

[0124] Next, we will combine Fig. 9 , the positive sample training process of the reconstruction network provided by the embodiment of the present disclosure is explained.

[0125] S01. Obtain positive samples and a preset initial query word sequence.

[0126] S02. Based on the positive sample and the initial query word sequence, combined with the preset positive sample loss function, the initial reconstruction network is trained until the reconstructed network is obtained; wherein the preset positive sample loss function is used to evaluate the reconstruction performance of the initial reconstruction network on the positive sample according to the feature difference obtained by reconstructing the positive sample using the initial query word sequence. In the disclosed embodiment, the electronic device can obtain the positive sample as the training sample, and initialize the query word sequence to obtain the initial query word sequence, thereby combining the preset positive sample loss function to iteratively train the initial reconstruction network until the preset training conditions are met to obtain the reconstructed network. Here, the preset positive sample loss function is used to evaluate the reconstruction performance of the initial reconstruction network on the positive sample according to the feature difference obtained by reconstructing the positive sample using the initial query word sequence.

[0127] In some embodiments, based on Fig. 9 , S02 can be implemented by executing the process of S021-S025, such as Fig.10 As shown, each step will be explained in combination:

[0128] S021. Perform feature processing on the positive sample to obtain a multi-scale positive sample feature sequence.

[0129] In the disclosed embodiment, for each round of iterative training, the electronic device may perform feature processing on the positive samples through a feature extraction network to obtain a multi-scale positive sample feature sequence.

[0130] In some embodiments, the electronic device can perform feature processing on the positive sample through a pre-trained feature extraction network, or use the initial feature extraction network to perform collaborative training with the initial reconstruction network, and synchronously adjust the network parameters of the initial feature extraction network and the initial reconstruction network in each round of training until the preset training conditions are met to obtain the trained feature extraction network and reconstruction network. The specific selection is made according to the actual situation, and the embodiments of the present disclosure are not limited.

[0131] In the disclosed embodiment, the process of the electronic device performing feature processing on the positive sample is consistent with the process description of S101 and will not be repeated here.

[0132] S022. Based on the initial reconstruction network, reconstruct the multi-scale positive sample feature sequence and the initial query word sequence to obtain a reconstructed positive sample feature sequence.

[0133] In the disclosed embodiment, the electronic device reconstructs the preset initial query word sequence based on the initial reconstruction network and takes the multi-scale positive sample feature sequence as guidance information to obtain a reconstructed positive sample feature sequence. The above reconstruction process is consistent with the process description of S102 and will not be repeated here.

[0134] S023. Determine a positive sample feature difference sequence based on the reconstructed positive sample feature sequence and the multi-scale positive sample feature sequence.

[0135] In the disclosed embodiment, the positive sample feature difference sequence characterizes the reconstruction of the multi-scale pixel features of each pixel by the reconstruction network, and the difference between the obtained pixel reconstruction features and the original multi-scale pixel features, that is, characterizes the reconstruction effect of the reconstruction network for each pixel. The electronic device can calculate the positive sample loss of the current round of training based on the positive sample feature difference sequence, and adjust the training of the initial reconstruction network based on the current positive sample loss and the preset loss threshold until a reconstruction network with a final loss less than the preset loss threshold is obtained.

[0136] In the disclosed embodiment, the electronic device may perform a difference comparison between the reconstructed positive sample feature sequence and the multi-scale positive sample feature sequence in a sequence form to obtain a positive sample feature difference sequence, and obtain the positive sample feature difference at each pixel position according to the positive sample feature difference sequence.

[0137] In the disclosed embodiment, the electronic device may also transform the shape of the reconstructed positive sample feature sequence to restore it to a reconstructed positive sample feature in matrix form, and then calculate the feature difference between the reconstructed positive sample pixel features at each pixel position contained in the reconstructed positive sample features and the multi-scale positive sample pixel features at each pixel position in the multi-scale sample features corresponding to the multi-scale positive sample feature sequence, to obtain the positive sample feature difference at each pixel position.

[0138] In some embodiments, the electronic device may calculate the positive sample feature difference at each pixel position by formula (3), as follows:

[0139]

[0140] In formula (3), f(u,i) is the multi-scale positive sample feature, is the reconstructed positive sample feature; u is the index coordinate of the pixel position, which can be, for example, the coordinate of each pixel position in the feature map of size H×W. i is the index of the feature channel. The electronic device can perform pixel position subtraction between the multi-scale sample feature and the reconstructed sample feature by formula (3) to obtain the positive sample feature difference d(u,i) at each pixel position u.

[0141] S024. Calculate the current positive sample loss based on the positive sample feature difference sequence and the preset positive sample loss function.

[0142] In the disclosed embodiment, the electronic device may use a preset positive sample loss function and, based on the positive sample feature difference sequence, evaluate the effect of the initial reconstruction network on the feature reconstruction of the positive sample during the current round of training to obtain the current positive sample loss.

[0143] In some embodiments, the positive sample may include: a completely positive sample without a label, that is, a reliable normal image without a true value annotation. The preset positive sample loss function includes: a regression loss function, and the electronic device can use the regression loss function as the preset positive sample loss function, and calculate the current positive sample loss according to the positive sample feature difference sequence through the regression loss function.

[0144] Exemplarily, the regression loss function may include smooth L1 loss, MSE loss, etc., which is specifically selected according to actual conditions and is not limited in the embodiments of the present disclosure.

[0145] In some embodiments, the process of calculating the current positive sample loss through the regression loss function can be as follows: Fig.11 As shown, the electronic device first obtains the positive sample feature difference sequence based on the reconstructed positive sample feature sequence and the multi-scale positive sample feature sequence, thereby obtaining the positive sample feature difference at each pixel position; and then, through the smoothing loss function, the positive sample feature difference at each pixel position is smoothed to obtain the current positive sample loss. As shown in formula (4), it is as follows:

[0146]

[0147] In formula (4), L nor Represents the current positive sample loss.

[0148] In some embodiments, the positive sample may include: a positive sample corresponding to a true value annotated at the image level. For example, the positive sample may be a sample in a training sample set that contains both positive samples and abnormal samples, and the true value represents that the image is a normal image. The preset positive sample loss function includes: the image level loss function. The electronic device calculates the current positive sample loss based on the positive sample feature difference sequence. The process can be as follows: Fig.12 As shown, including S201-S204, as follows:

[0149] S201, performing channel dimension scalarization processing on the positive sample feature difference sequence to obtain the positive sample feature difference measurement at each pixel position.

[0150] In the disclosed embodiment, the electronic device may convert the feature difference representing each pixel position in the positive sample feature difference sequence into a scalar of the channel dimension, so that each pixel position corresponds to only one scalarized positive sample feature difference metric.

[0151] In some embodiments, the electronic device may convert the positive sample feature difference sequence into a positive sample feature difference in matrix form, such as a positive sample feature difference of H×W×C dimensions, and perform channel dimension averaging on the C-dimensional positive sample pixel feature difference at each pixel position, i.e., C-dimensional averaging, to obtain a positive sample feature difference metric at each pixel position. For example, as shown in formula (5):

[0152]

[0153] In formula (5), the electronic device can perform C-dimensional mean square error processing on the positive sample pixel feature difference at each pixel position u to obtain As the initial scalarization result, and further Smoothing is performed to obtain the positive sample feature difference metric φ(u).

[0154] S202. Determine K maximum positive sample feature difference metrics with the largest values ​​from the positive sample feature difference metrics at each pixel position; K is a positive integer greater than 1.

[0155] In the disclosed embodiment, the electronic device can evaluate the reconstruction effect of each pixel point based on the positive sample feature difference metric at each pixel position, and select pixel points with poor reconstruction effect, that is, pixel points with larger φ(u) values ​​to calculate the current positive sample loss, so as to perform targeted optimization on the pixel points with poor reconstruction effect in the positive sample.

[0156] In some embodiments, the electronic device may determine K maximum positive sample feature difference metrics with the largest values ​​from the positive sample feature difference metrics at each pixel position; wherein K is a positive integer greater than 1.

[0157] S203 , averaging the K largest positive sample feature difference metrics to obtain a positive sample feature average difference metric.

[0158] In the disclosed embodiment, the electronic device may average the K largest positive sample feature difference metrics to obtain an average positive sample feature difference metric.

[0159] For example, the electronic device may calculate the average difference metric of the positive sample features through formula (6), as follows:

[0160]

[0161] In formula (6), q is the average difference measure of positive sample features.

[0162] 204. Based on the average difference metric of the positive sample features, the true value of the positive sample and the image-level loss function, the current positive sample loss is obtained.

[0163] In the disclosed embodiment, the electronic device may calculate the current positive sample loss corresponding to the positive sample by using an image-level loss function that is compatible with positive samples and abnormal samples.

[0164] In some embodiments, the image-level loss function includes: a normal image loss part and an abnormal image loss part. When the true value representation image is a normal image, the electronic device can determine the average difference metric of the positive sample features as the current positive sample loss based on the normal image loss part. The average difference metric of the positive sample features is positively correlated with the current positive sample loss. The electronic device can use the current positive sample loss to adjust the parameters of the initial reconstruction network with the goal of reducing the average difference metric of the positive sample features, so that the reconstructed features of the normal image reconstructed by the initial reconstruction network are close to the original features.

[0165] Here, the abnormal image loss part is used to calculate the current abnormal sample loss corresponding to the abnormal sample when processing the abnormal sample. Among them, the abnormal sample is a sample whose true value characterization image is an abnormal image. The electronic device can perform feature extraction processing, reconstruction, and feature difference sequence calculation similar to S101-S103 based on the abnormal sample to obtain the abnormal sample feature difference sequence, and perform channel dimension scalarization processing and maximum abnormal sample feature difference metric average processing similar to S201-S203 based on the abnormal sample feature difference sequence to obtain the abnormal sample feature difference metric. Then, the electronic device can obtain the current abnormal sample loss through the abnormal image loss part based on the abnormal sample feature difference metric corresponding to the abnormal sample. Among them, the average difference metric of the abnormal sample features is negatively correlated with the current abnormal sample loss. The current abnormal sample loss is used to adjust the initial reconstruction network with the goal of increasing the average difference metric of the abnormal sample features, so that the reconstructed features of the abnormal image reconstructed by the initial reconstruction network are far away from the original features.

[0166] In some embodiments, the image-level loss function may be as shown in formula (7), as follows:

[0167] L img =(1-y)q-αylog(1-exp(-q)) (7)

[0168] In formula (7), y is the true value and α is the weight factor. Taking the true value corresponding to the positive sample as 0 and the true value corresponding to the abnormal sample as 1 as an example, (1-y)q is the loss part of the normal image, and -αylog(1-exp(-q)) is the loss part of the abnormal image. When y=0, L img is the current positive sample loss. When y=1, L imgis the current abnormal sample loss. It can be seen that for normal images, the smaller q is, the smaller the distance between the reconstructed features of the normal image and the original features is, and accordingly, the current positive sample loss is also smaller. In other words, the current positive sample loss is positively correlated with the average difference measure of the positive sample features. For abnormal images, the larger q is, the larger the distance between the reconstructed features of the abnormal image and the original features is, and accordingly, the current abnormal sample loss is smaller. In other words, the current abnormal sample loss is negatively correlated with the average difference measure of the abnormal sample features.

[0169] It can be understood that in the embodiments of the present disclosure, an image-level loss function that is compatible with positive samples and abnormal samples is used to calculate the current positive sample loss corresponding to the positive sample, and the initial reconstruction network is trained. This provides a basis for obtaining the initial reconstruction network by training with different types of samples, thereby improving the flexibility and compatibility of the training method of the initial reconstruction network.

[0170] S025. Based on the current positive sample loss and the preset loss threshold, the initial reconstruction network is trained and adjusted until a reconstructed network with a final loss less than the preset loss threshold is obtained.

[0171] In the disclosed embodiment, when the electronic device obtains the current positive sample loss by using a preset regression loss function or averaging the positive sample feature difference metric, the electronic device can compare the current positive sample loss with a preset loss threshold, and when the current positive sample loss is greater than the preset loss threshold, the initial reconstruction network is updated with the goal of reducing the current positive sample loss, and the initial reconstruction network with updated parameters is used for the next round of training. The electronic device iteratively trains the initial reconstruction network through the above process, and obtains a reconstructed network when the final loss is less than the preset loss threshold.

[0172] It should be noted that, during the training process of the initial reconstruction network, the initial query word sequence can be updated with each round of updates of the initial reconstruction network, or can be updated according to its own update cycle, until the final loss is less than the preset loss threshold, and a preset query word sequence is obtained. The specific selection is made according to the actual situation, and the embodiments of the present disclosure are not limited.

[0173] It can be understood that the electronic device can evaluate the reconstruction ability of the initial reconstruction network for completely positive samples by calculating the gap between the original features and the reconstructed features through the positive sample loss function, thereby improving the reconstruction effect of the reconstruction network for completely positive samples, so that the reconstructed features of normal pixels reconstructed by the reconstruction network are as close to the original features as possible. In this way, the generalization gap between the reconstructed features of abnormal pixels can be widened, so that the reconstructed features of normal pixels and abnormal pixels processed by the reconstruction network show obvious differences in quality, thereby realizing anomaly detection using the reconstructed features output by the reconstruction network and improving the accuracy of anomaly detection.

[0174] The present disclosure also provides an abnormality detection device. Fig.13 A schematic diagram of the structure of an abnormality detection device provided in an embodiment of the present disclosure; Fig.13 As shown, the abnormality detection device 500 includes:

[0175] Feature extraction network 501; used to perform feature processing of different scales on the image to be detected to obtain a multi-scale feature sequence;

[0176] A reconstruction network 502, used to reconstruct based on the multi-scale feature sequence to obtain a reconstructed feature sequence; the reconstruction network is obtained by training with positive samples;

[0177] A determining unit 503, configured to determine a feature difference at each pixel position according to the reconstructed feature sequence and the multi-scale feature sequence;

[0178] A scoring unit 504, configured to determine an abnormality score at each pixel position based on the feature difference;

[0179] The detection unit 505 is configured to draw an abnormality detection image corresponding to the image to be detected based on the abnormality score.

[0180] In some embodiments, the feature extraction network 501 is further used to extract features of different scales on the image to be detected to obtain features of multiple scales; fuse the multiple features of different scales to obtain multi-scale features; and perform shape transformation on the multi-scale features to obtain the multi-scale feature sequence.

[0181] In some embodiments, the reconstruction network 502 includes: an encoding network and a decoding network; the encoding network is used to encode the multi-scale feature sequence to obtain a coded sequence;

[0182] The decoding network is used to perform decoding reconstruction based on the encoding sequence to obtain the reconstructed feature sequence.

[0183] In some embodiments, the determining unit 503 is further configured to calculate a feature difference at each corresponding sequence position between the reconstructed feature sequence and the multi-scale feature sequence to obtain the feature difference at each pixel position.

[0184] In some embodiments, the scoring unit 504 is further configured to perform channel averaging on the feature differences at each pixel position to obtain an abnormality score at each pixel position.

[0185] In some embodiments, the detection unit 505 is further used to draw the pixel value corresponding to each pixel position as a first value when the anomaly score at the pixel position is less than or equal to the scoring threshold; and to draw the pixel value corresponding to the pixel position as a second value when the anomaly score at each pixel position is greater than the scoring threshold; until the pixel positions of the image to be detected are drawn, the anomaly detection image is obtained.

[0186] In some embodiments, the anomaly detection device further includes: a training unit; the training unit is used to reconstruct the multi-scale feature sequence and the preset query word sequence based on the reconstruction network, and obtain positive samples and a preset initial query word sequence before obtaining the reconstructed feature sequence; based on the positive samples and the initial query word sequence, combined with a preset positive sample loss function, the initial reconstruction network is trained until the reconstructed network is obtained; wherein the preset positive sample loss function is used to evaluate the reconstruction performance of the initial reconstruction network on the positive sample based on the feature difference obtained by reconstructing the positive sample using the initial query word sequence.

[0187] In some embodiments, the training unit is further used to perform feature processing on the positive sample to obtain a multi-scale positive sample feature sequence; based on the initial reconstruction network, reconstruct the multi-scale positive sample feature sequence and the initial query word sequence to obtain a reconstructed positive sample feature sequence; determine a positive sample feature difference sequence based on the reconstructed positive sample feature sequence and the multi-scale positive sample feature sequence; calculate the current positive sample loss based on the positive sample feature difference sequence and a preset positive sample loss function; based on the current positive sample loss and the preset loss threshold, train and adjust the initial reconstruction network until the reconstructed network with a final loss less than the preset loss threshold is obtained.

[0188] In some embodiments, the preset positive sample loss function includes: a regression loss function, and the training unit is further used to perform gradient smoothing processing on the positive sample feature difference sequence through the regression loss function to obtain the current positive sample loss.

[0189] In some embodiments, the preset positive sample loss function includes: an image-level loss function, the training unit is further used to perform channel dimension scalarization processing on the positive sample feature difference sequence to obtain a positive sample feature difference metric at each pixel position; from the positive sample feature difference metric at each pixel position, determine the K largest positive sample feature difference metrics with the largest values; K is a positive integer greater than 1; average the K largest positive sample feature difference metrics to obtain a positive sample feature average difference metric;

[0190] The current positive sample loss is obtained based on the average difference measure of the positive sample features, the true value of the positive sample and the image-level loss function.

[0191] In some embodiments, the image-level loss function includes: a normal image loss part and an abnormal image loss part,

[0192] The training unit is further configured to determine the average difference measure of the positive sample features as the current positive sample loss based on the normal image loss part when the true value representation image is a normal image; the average difference measure of the positive sample features is positively correlated with the current positive sample loss; wherein,

[0193] The abnormal image loss part is used to obtain the current abnormal sample loss based on the abnormal sample feature difference metric corresponding to the abnormal sample when processing the abnormal sample whose true value representation image is an abnormal image; the abnormal sample feature average difference metric is negatively correlated with the current abnormal sample loss; the current abnormal sample loss is used to adjust the initial reconstruction network with the goal of increasing the abnormal sample feature average difference metric.

[0194] In some embodiments, the training unit is further used to iteratively update the initial query word sequence during the training process of the initial reconstruction network until the final loss is less than the preset loss threshold, thereby obtaining the preset query word sequence.

[0195] In some embodiments, the detection unit 505 is further used to perform channel averaging on the feature differences at each pixel position to obtain an abnormality score at each pixel position.

[0196] In some embodiments, the detection unit 505 is further used to draw the pixel value corresponding to each pixel position as a first value when the anomaly score at the pixel position is less than or equal to the scoring threshold; and to draw the pixel value corresponding to the pixel position as a second value when the anomaly score at each pixel position is greater than the scoring threshold; until the pixel positions of the image to be detected are drawn, the anomaly detection image is obtained.

[0197] It should be noted that the description of the above device embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the device embodiment of the present disclosure, please refer to the description of the method embodiment of the present disclosure for understanding.

[0198] The present disclosure also provides an electronic device, Fig.14 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure, such as Fig.14 As shown, the electronic device 2 includes: a memory 21 and a processor 22, wherein the memory 21 and the processor 22 are connected via a communication bus 23; the memory 21 is used to store an executable computer program; the processor 22 is used to implement the anomaly detection method provided by the embodiment of the present disclosure when executing the executable computer program stored in the memory 21.

[0199] The embodiment of the present disclosure provides a computer-readable storage medium storing a computer program for causing the processor 22 to execute and implement the anomaly detection method provided by the embodiment of the present disclosure.

[0200] In some embodiments of the present disclosure, the storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be various devices including one or any combination of the above memories.

[0201] In some embodiments of the present disclosure, executable instructions may be in the form of programs, software, software modules, scripts or codes, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as stand-alone programs or as modules, components, subroutines or other units suitable for use in a computing environment.

[0202] As an example, executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions).

[0203] By way of example, executable instructions may be deployed to be executed on one computing device, or on multiple computing devices located at one site, or on multiple computing devices distributed across multiple sites and interconnected by a communication network.

[0204] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EP ROM, EEPROM, flash memory, magnetic surface storage, optical disk, or CD-ROM; or it may be various devices including one or any combination of the above memories.

[0205] In some embodiments, executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0206] As an example, executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions).

[0207] By way of example, executable instructions may be deployed to be executed on one computing device, or on multiple computing devices located at one site, or on multiple computing devices distributed across multiple sites and interconnected by a communication network.

[0208] In summary, the reconstruction network obtained through positive sample training realizes the understanding and modeling of the feature distribution of normal samples. Therefore, when there are abnormal pixels in the image to be detected, the reconstructed features after the abnormal pixels are reconstructed by the reconstruction network will be quite different from the original abnormal pixel features, that is, the feature differences of the reconstruction of abnormal pixels are large. Therefore, the feature differences after reconstruction can be used as anomaly scores, and the pixel-level detection results of the image to be detected can be obtained through anomaly scores, thereby improving the accuracy of anomaly detection. In addition, by reconstructing multi-scale features together with preset query words, it is possible to capture deeper semantic information in the image to be detected, thereby further improving the accuracy of anomaly detection.

[0209] The above description is only an embodiment of the present disclosure and is not intended to limit the protection scope of the present disclosure. Any modification, equivalent replacement and improvement made within the spirit and scope of the present disclosure are included in the protection scope of the present disclosure.

Claims

1. An anomaly detection method, characterized in that: include: Obtain positive samples and a preset initial query word sequence; Based on the positive samples and the initial query word sequence, combined with a preset positive sample loss function, the initial reconstruction network is trained until the reconstruction network is obtained; The preset positive sample loss function is used to evaluate the reconstruction performance of the initial reconstruction network on the positive sample according to the feature difference obtained by reconstructing the positive sample using the initial query word sequence; Using the feature extraction network, the image to be detected is processed with features of different scales to obtain a multi-scale feature sequence; Reconstructing based on the multi-scale feature sequence through the reconstruction network to obtain a reconstructed feature sequence; wherein the reconstruction network is obtained by training with positive samples; Determining a feature difference at each pixel position according to the reconstructed feature sequence and the multi-scale feature sequence; Based on the feature difference, determining an abnormality score at each pixel position; Based on the anomaly score, an anomaly detection image corresponding to the image to be detected is drawn.

2. The method according to claim 1, characterized in that The process of performing feature processing of different scales on the image to be detected to obtain a multi-scale feature sequence includes: Extracting features of different scales from the image to be detected to obtain features of multiple different scales; Fusing the multiple features of different scales to obtain multi-scale features; Performing shape transformation on the multi-scale features to obtain the multi-scale feature sequence.

3. The method according to claim 1 or 2, characterized in that: The reconstruction network includes: an encoding network and a decoding network; The reconstructing based on the multi-scale feature sequence through the reconstruction network to obtain the reconstructed feature sequence includes: Based on the encoding network, encoding the multi-scale feature sequence to obtain an encoding sequence; The decoding network is used to perform decoding reconstruction based on the encoding sequence to obtain the reconstructed feature sequence.

4. The method according to claim 1 or 2, characterized in that: The determining the feature difference at each pixel position according to the reconstructed feature sequence and the multi-scale feature sequence includes: The feature difference at each corresponding sequence position in the reconstructed feature sequence and the multi-scale feature sequence is calculated to obtain the feature difference at each pixel position.

5. The method according to claim 1, characterized in that The determining, based on the feature difference, an abnormality score at each pixel position comprises: The feature differences at each pixel position are averaged by channel to obtain an abnormality score at each pixel position.

6. The method according to claim 1 or 5, characterized in that: Drawing the abnormality detection image corresponding to the image to be detected based on the abnormality score includes: When the abnormality score at each pixel position is less than or equal to the score threshold, the pixel value corresponding to the pixel position is plotted as the first value; When the abnormality score at each pixel position is greater than the score threshold, the pixel value corresponding to the pixel position is drawn as the second value; until the pixel positions of the image to be detected are drawn, the abnormality detection image is obtained.

7. The method according to claim 1, characterized in that The initial reconstruction network is trained based on the positive sample and the initial query word sequence in combination with a preset positive sample loss function until the reconstruction network is obtained, including: Performing feature processing on the positive sample to obtain a multi-scale positive sample feature sequence; Based on the initial reconstruction network, reconstructing the multi-scale positive sample feature sequence and the initial query word sequence to obtain a reconstructed positive sample feature sequence; Determining a positive sample feature difference sequence according to the reconstructed positive sample feature sequence and the multi-scale positive sample feature sequence; Calculating the current positive sample loss based on the positive sample feature difference sequence and a preset positive sample loss function; Based on the current positive sample loss and the preset loss threshold, the initial reconstruction network is trained and adjusted until the reconstruction network having a final loss less than the preset loss threshold is obtained.

8. The method according to claim 7, characterized in that The preset positive sample loss function includes: a regression loss function, and the calculation of the current positive sample loss based on the positive sample feature difference sequence and the preset positive sample loss function includes: The positive sample feature difference sequence is subjected to gradient smoothing processing by means of the regression loss function to obtain the current positive sample loss.

9. The method according to claim 7 or 8, characterized in that: The method further comprises: During the training process of the initial reconstruction network, the initial query word sequence is iteratively updated until the final loss is less than the preset loss threshold, thereby obtaining the preset query word sequence.

10. An abnormality detection device, characterized in that: include: A training unit, used to obtain positive samples and a preset initial query word sequence; Based on the positive sample and the initial query word sequence, combined with a preset positive sample loss function, the initial reconstruction network is trained until the reconstruction network is obtained; wherein the preset positive sample loss function is used to evaluate the reconstruction performance of the initial reconstruction network on the positive sample according to the feature difference obtained by reconstructing the positive sample using the initial query word sequence; Feature extraction network: used to process features of different scales on the image to be detected and obtain a multi-scale feature sequence; A reconstruction network is used to reconstruct based on the multi-scale feature sequence to obtain a reconstructed feature sequence; the reconstruction network is obtained by training with positive samples; A determination unit, configured to determine a feature difference at each pixel position according to the reconstructed feature sequence and the multi-scale feature sequence; A scoring unit, configured to determine an abnormality score at each pixel position based on the feature difference; A detection unit is used to draw an abnormality detection image corresponding to the image to be detected based on the abnormality score.

11. An electronic device, characterized in that: include: A memory for storing executable instructions; A processor, configured to implement the method according to any one of claims 1 to 9 when executing the executable instructions stored in the memory.

12. A computer-readable storage medium, characterized in that: Executable instructions are stored, and when executed by a processor, the method described in any one of claims 1 to 9 is implemented.

Images