Anomaly detection method, system, medium, electronic device and anomaly detection model

Abstract

The present invention provides an anomaly detection method, system, medium, electronic device and anomaly detection model; the anomaly detection model comprises: an anomaly filter, a global semantic embedding block, a multi-scale channel autoencoder and a student feature restoration network; the global semantic embedding block is connected with the anomaly filter, and the global semantic embedding block is used to obtain a deep semantic information flow; the multi-scale channel autoencoder is used to obtain a normal detail information flow; the student feature restoration network is used to obtain a student feature map based on the deep semantic information flow and the normal detail information flow; the student feature map is compared with the teacher feature map to obtain an anomaly map, so as to realize anomaly detection of an image to be tested based on the anomaly map; the present invention uses the dual information flow of the deep semantic information flow and the normal detail information flow for anomaly detection, which helps the student feature restoration network to simultaneously reconstruct semantic and detail features, and effectively solves the detection problem of large defects and semantic structure defects.

Description

Technical Field

[0001] The present invention relates to the field of physics, and in particular to computer vision detection technology, in particular to an anomaly detection method, system, medium, electronic equipment and anomaly detection model. Background Art

[0002] Surface defect detection of industrial products is becoming increasingly important in today's large-scale manufacturing scenarios. However, in real industrial scenarios, due to the scarcity of actual abnormal data, unsupervised anomaly detection that relies on the data distribution of normal samples to detect unknown abnormal data has gradually occupied an important position.

[0003] At present, the anomaly detection method based on knowledge distillation has shown excellent accuracy. By allowing the model to use normal data for training and taking advantage of the difference between students and teachers in the ability to extract abnormal features, the purpose of anomaly detection is achieved. The introduction of the reverse distillation method reduces students' generalization ability for abnormal areas and improves the performance of anomaly detection; however, because students receive a single information stream and cannot balance semantic and detailed information, the current reverse distillation anomaly detection cannot detect large defects and semantic structure defects well. Summary of the invention

[0004] The object of the present invention is to provide an anomaly detection method, system, medium, electronic device and anomaly detection model to solve the problems pointed out in the above background technology.

[0005] In a first aspect, the present invention provides an anomaly detection model, which is used to realize anomaly detection of the image to be tested based on the teacher feature map of the input image to be tested, and the anomaly detection model includes: an anomaly filter, a global semantic embedding block, a multi-scale channel autoencoder and a student feature restoration network; wherein the anomaly filter is used to filter the abnormal information in the teacher feature map to obtain a processing feature map; the global semantic embedding block is connected to the anomaly filter, the global semantic embedding block is used to receive the processing feature map, and is used to enhance the semantic information of the processing feature map to obtain a deep semantic information flow; the multi-scale channel autoencoder is used to extract the normal detail information flow in the teacher feature map; the student feature restoration network is respectively connected to the global semantic embedding block and the multi-scale channel autoencoder, and is used to obtain a student feature map based on the deep semantic information flow and the normal detail information flow; compare the student feature map with the teacher feature map to obtain an anomaly map, so as to realize anomaly detection of the image to be tested based on the anomaly map.

[0006] In the present invention, an anomaly detection model is provided, which performs anomaly detection by using dual information flows of deep semantic information flow and normal detail information flow, thereby helping the student feature restoration network to simultaneously reconstruct semantic and detail features, and effectively solving the detection problem of large defects and semantic structure defects.

[0007] In an implementation of the first aspect, the abnormal filter includes: a feature fusion layer, a feature embedding layer, and a convolution and self-attention block; wherein the feature fusion layer is used to process the teacher feature map to obtain at least one first feature map, and is used to perform feature fusion on the teacher feature map and the at least one first feature map to obtain a fused feature map; the feature embedding layer is connected to the feature fusion layer, the feature embedding layer is used to receive the fused feature map, and is used to perform feature embedding on the fused feature map to obtain a second feature map; the convolution and self-attention block is connected to the feature embedding layer, the convolution and self-attention block is used to receive the second feature map, and is used to perform feature recombination on the second feature map to obtain a third feature map, and is used to perform self-attention feature selection on the second feature map to obtain a fourth feature map, and finally splice the third feature map and the fourth feature map to obtain the processed feature map.

[0008] In an implementation of the first aspect, the global semantic embedding block includes: a global average pooling module, a first feature concatenation module, a feature multiplication module, at least one resolution reduction module and at least one channel attention module; wherein the resolution reduction module is connected to the channel attention module in a one-to-one correspondence, and the resolution reduction module is used to receive the processed feature map, and reduce the resolution of the processed feature map to obtain a fifth feature map; when the number of the resolution reduction modules is at least two, the resolutions of the fifth feature map obtained after being processed by at least two resolution reduction modules are different; the channel attention module is used to receive the fifth feature map, and to perform local cross-channel interaction on the fifth feature map to obtain a sixth feature map; the global average pooling module is used to receive the processed feature map, and perform global average pooling processing on the processed feature map to obtain a seventh feature map; when the number of the resolution reduction modules is at least two, the global semantic embedding block also includes: a second feature concatenation module; the second feature concatenation module is connected to the channel attention module, and is used to receive the sixth feature map, and perform local cross-channel interaction on at least two sixth feature maps. Feature splicing to obtain an eighth feature map; the feature multiplication module is connected to the second feature splicing module and the abnormal filter, and is used to receive the eighth feature map and the processed feature map respectively, and is used to multiply the eighth feature map and the processed feature map pixel by pixel to obtain a ninth feature map; the first feature splicing module is connected to the global average pooling module and the feature multiplication module, and is used to receive the seventh feature map and the ninth feature map respectively, and is used to perform feature splicing on the seventh feature map and the ninth feature map to obtain the deep semantic information flow; when the number of the resolution reduction modules is one, the feature multiplication module is connected to the channel attention module and the abnormal filter, and is used to receive the sixth feature map and the processed feature map respectively, and is used to multiply the sixth feature map and the processed feature map pixel by pixel to obtain a tenth feature map; the first feature splicing module is connected to the global average pooling module and the feature multiplication module, and is used to receive the seventh feature map and the tenth feature map respectively, and is used to perform feature splicing on the seventh feature map and the tenth feature map to obtain the deep semantic information flow.

[0009] In a second aspect, the present invention provides an anomaly detection method implemented based on the above-mentioned anomaly detection model, and the anomaly detection method includes: preprocessing the target image to be tested to obtain a target processed image; inputting the target processed image into a teacher feature extraction network to obtain a target teacher feature map; inputting the target teacher feature map into the anomaly detection model to obtain a target anomaly map, so as to implement anomaly detection of the target image to be tested based on the target anomaly map.

[0010] In an implementation manner of the second aspect, the preprocessing includes at least: data enhancement processing and / or normalization processing.

[0011] In an implementation of the second aspect, before the step of inputting the target teacher feature map into the anomaly detection model, the anomaly detection method also includes: preprocessing normal training sample data to obtain processed data; the normal training sample data includes: a plurality of test images without abnormalities; inputting the processed data into the teacher feature extraction network to obtain a teacher feature map; inputting the teacher feature map into the anomaly detection model to obtain a student feature map; constructing a loss function of the anomaly detection model based on the teacher feature map and the student feature map to train the anomaly detection model based on the loss function to obtain a trained anomaly detection model; inputting the target teacher feature map into the anomaly detection model includes: inputting the target teacher feature map into the trained anomaly detection model.

[0012] In an implementation of the second aspect, the teacher feature extraction network adopts a pre-trained deep feature extraction network.

[0013] In the third aspect, the present invention provides an anomaly detection system implemented based on the above-mentioned anomaly detection model, and the anomaly detection system includes: a preprocessing module, used to preprocess the target image to be tested and obtain a target processed image; a feature extraction module, used to input the target processed image into a teacher feature extraction network to obtain a target teacher feature map; an anomaly detection module, used to input the target teacher feature map into the anomaly detection model to obtain a target anomaly map, so as to realize anomaly detection of the target image to be tested based on the target anomaly map.

[0014] In a fourth aspect, the present invention provides an electronic device, comprising: a processor and a memory; the memory is used to store a computer program; the processor is used to execute the computer program stored in the memory, so that the electronic device performs the above-mentioned abnormality detection method.

[0015] In a fifth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which implements the above-mentioned anomaly detection method when executed by an electronic device.

[0016] As described above, the anomaly detection method, system, medium, electronic device and anomaly detection model of the present invention have the following beneficial effects:

[0017] (1) Compared with the prior art, the present invention uses dual information streams for anomaly detection. By diverting the deep semantic information stream and the normal detail information stream into the student feature restoration network, it helps it to simultaneously reconstruct the semantic and detail features, thereby solving the detection problem of large defects and semantic structure defects.

[0018] (2) In the semantic information flow, the present invention designs a global semantic embedding block to enrich the semantic information of the features required by the student feature restoration network, and adds a learnable anomaly filter to suppress the abnormal information transmitted by the teacher feature extraction network. In addition, a multi-scale channel autoencoder is designed to directly extract the detail information flow from the teacher output, assist the student in reconstructing the teacher's features, and help the anomaly detection model clarify the anomaly detection boundary. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 Shown is a schematic diagram of the structure of the teacher feature extraction network described in an embodiment of the present invention.

[0020] Figure 2 Shown is a schematic diagram of the structure of BottleNeck according to an embodiment of the present invention.

[0021] Figure 3 Shown is a structural schematic diagram of the anomaly detection model described in an embodiment of the present invention.

[0022] Figure 4 Shown is a schematic diagram of the structure of the abnormal filter according to an embodiment of the present invention.

[0023] Figure 5 Shown is a schematic diagram of the structure of the global semantic embedding block according to an embodiment of the present invention.

[0024] Figure 6 Shown is a schematic diagram of the structure of the multi-scale channel autoencoder according to an embodiment of the present invention.

[0025] Figure 7 Shown is a flow chart of the anomaly detection method according to an embodiment of the present invention.

[0026] Figure 8 Shown is a flowchart of training an anomaly detection model and obtaining a trained anomaly detection model as described in an embodiment of the present invention.

[0027] Fig. 9 Shown is a structural schematic diagram of the anomaly detection system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0028] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0029] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention. The illustrations only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0030] See also Figures 1 to 9 . The following embodiments of the present invention provide anomaly detection methods, systems, media, electronic devices and anomaly detection models. Compared with the prior art, the present invention uses dual information streams for anomaly detection, and by diverting the deep semantic information stream and the normal detail information stream into the student feature restoration network, it helps to simultaneously reconstruct the semantic and detail features, thereby solving the detection problem for large defects and semantic structure defects; in the semantic information stream, the present invention designs a global semantic embedding block to enrich the semantic information of the features required by the student feature restoration network, and adds a learnable anomaly filter to suppress the abnormal information transmitted by the teacher feature extraction network; in addition, a multi-scale channel autoencoder is designed to directly extract the detail information stream from the teacher output, assist the student in reconstructing the teacher's features together, and help the anomaly detection model clarify the anomaly detection boundary.

[0031] In one embodiment, the present invention provides an anomaly detection model, which is used to detect anomalies of an image to be tested based on a teacher feature map of the input image to be tested.

[0032] In one embodiment, the teacher feature map is obtained by inputting the image to be tested into a teacher feature extraction network.

[0033] In one embodiment, the teacher feature extraction network uses a pre-trained deep feature extraction network, such as WideResNet50, and other deep feature extraction networks can also be selected independently to extract intermediate features.

[0034] The following is an example of how an anomaly detection model implements anomaly detection on an image to be tested based on teacher feature maps at three scales corresponding to the image to be tested.

[0035] like Figure 1As shown, in one embodiment, the teacher feature extraction network extracts three layers of intermediate features, the first two layers use 3 layers of BottleNeck, and the third layer uses 6 layers of BottleNeck. The extraction of multi-scale features helps to detect defects of different scales well.

[0036] like Figure 2 As shown in the figure, the BottleNeck layer includes two convolutional layers, one normalization layer and two groups: convolutional layer → normalization layer → activation function layer; specifically, the input is extracted through two groups of convolutional layers → normalization layer → activation function layer, and then passed to the next convolutional layer and normalization layer, and the residuals are linked and input into the activation function layer together to obtain the output.

[0037] It should be noted that all BottleNeck structures in the present invention are similar, and the convolution kernel size and step size are selected according to whether downsampling is performed.

[0038] like Figure 3 As shown, in one embodiment, the anomaly detection model includes: an anomaly filter 11, a global semantic embedding block 12, a multi-scale channel autoencoder 13 and a student feature restoration network (not shown in the figure).

[0039] Specifically, the abnormal filter 11 is used to filter abnormal information in the teacher feature map to obtain a processing feature map; the global semantic embedding block 12 is connected to the abnormal filter 11, and the global semantic embedding block 12 is used to receive the processing feature map, and to enhance the semantic information of the processing feature map to obtain a deep semantic information flow; the multi-scale channel autoencoder 13 is used to extract the normal detail information flow in the teacher feature map (for subsequent student feature restoration network restoration feature to help comparison); the student feature restoration network is respectively connected to the global semantic embedding block 12 and the multi-scale channel autoencoder 13, and is used to obtain a student feature map based on the deep semantic information flow and the normal detail information flow; the student feature map and the teacher feature map are compared to obtain an abnormal map, so as to realize abnormality detection of the image to be tested based on the abnormal map.

[0040] It should be noted that the extraction of normal detail information stream by the multi-scale channel autoencoder 13 adopts conventional technical means in this field, so its working principle will not be described in detail here.

[0041] In one embodiment, the structure of the student feature restoration network is similar to the above-mentioned teacher feature extraction network. Only the downsampling convolution layer in the teacher feature extraction network is replaced with an upsampling deconvolution structure to obtain the student feature restoration network. Therefore, the specific structure and working principle of the student feature restoration network will not be described in detail here.

[0042] like Figure 3 and Figure 4 As shown, in one embodiment, the abnormal filter 11 includes: a feature fusion layer 111, a feature embedding layer 112 and a convolution and self-attention block 113.

[0043] Specifically, the feature fusion layer 111 is used to process the teacher feature map to obtain at least one first feature map, and to perform feature fusion on the teacher feature map and the at least one first feature map to obtain a fused feature map; the feature embedding layer 112 is connected to the feature fusion layer 111, and the feature embedding layer 112 is used to receive the fused feature map, and to perform feature embedding on the fused feature map to suppress abnormal disturbances in the information and obtain a second feature map; the convolution and self-attention block 113 is connected to the feature embedding layer 112, and the convolution and self-attention block 113 is used to receive the second feature map, and to perform feature recombination on the second feature map to obtain a third feature map, and to perform feature selection of self-attention on the second feature map to obtain a fourth feature map, and finally to splice the third feature map and the fourth feature map to obtain the processed feature map.

[0044] like Figure 4 As shown, in one embodiment, the feature fusion layer 111 includes: a channel combination layer 1111 and two convolutional layers 1112 .

[0045] Specifically, the teacher feature map is convolved through two convolution layers 1112 to obtain two first feature maps, and then the two first feature maps and the teacher feature map are fused through the channel combination layer 1111 to obtain a fused feature map.

[0046] like Figure 4 As shown, in one embodiment, the feature embedding layer 112 includes three BottleNeck layers 1121 .

[0047] like Figure 4 As shown, in one embodiment, the convolution and self-attention block 113 includes four convolution layers 1131, a self-attention layer 1132 and a splicing layer 1133.

[0048] Specifically, the second feature map first passes through three convolutional layers 1131 to generate a rich set of intermediate features, and then is input into the residual convolutional layer 1131 and the self-attention layer 1132 respectively to obtain the feature reorganization of the convolution (i.e., the third feature map) and the feature selection of the self-attention (i.e., the fourth feature map). Finally, the features of the two are adaptively combined through the splicing layer 1133 to obtain the features after abnormal filtering (i.e., the processed feature map).

[0049] like Figure 3 and Figure 5As shown, in one embodiment, the global semantic embedding block 12 includes: a global average pooling module 121, a first feature splicing module 122, a feature multiplication module 123, at least one resolution reduction module 124 and at least one channel attention module 125.

[0050] Specifically, the resolution reduction module 124 is connected to the channel attention module 125 in a one-to-one correspondence, and the resolution reduction module 124 is used to receive the processed feature map, and reduce the resolution of the processed feature map to obtain the fifth feature map; when the number of the resolution reduction modules is at least two, the resolutions of the fifth feature map obtained after processing by at least two resolution reduction modules are different; the channel attention module 125 is used to receive the fifth feature map, and to perform local cross-channel interaction on the fifth feature map to obtain the sixth feature map; the global average pooling module 121 is used to receive the processed feature map, and perform global average pooling on the processed feature map to obtain the seventh feature map.

[0051] like Figure 5 As shown, in one embodiment, the resolution reduction module 124 includes two convolutional layers 1241 .

[0052] Specifically, by passing the processed feature map through two convolutional layers 1241 in sequence, the resolution of the processed feature map is reduced.

[0053] like Figure 5 As shown, in one embodiment, the number of the resolution reduction modules 124 is at least two ( Figure 4 When the global semantic embedding block 12 further includes: a second feature splicing module 126.

[0054] Specifically, the second feature splicing module 126 is connected to the channel attention module 125, and is used to receive the sixth feature map, and perform feature splicing on at least two sixth feature maps to obtain an eighth feature map; the feature multiplication module 123 is connected to the second feature splicing module 126 and the splicing layer 1133 of the exception filter 11, and is used to receive the eighth feature map and the processed feature map respectively, and to perform pixel-by-pixel multiplication on the eighth feature map and the processed feature map to obtain a ninth feature map; the first feature splicing module 122 is connected to the global average pooling module 121 and the feature multiplication module 123, and is used to receive the seventh feature map and the ninth feature map respectively, and to perform feature splicing on the seventh feature map and the ninth feature map to obtain the deep semantic information flow.

[0055] It should be noted that the high-dimensional features after abnormal filtering (corresponding to the processed feature map) are gradually integrated at a deeper level through the global semantic embedding block 12 until the minimum resolution (1×1) of the feature map, which helps the model extract the most critical global semantic information.

[0056] like Figure 5 As shown, after the feature map resolution of each branch (corresponding to each resolution reduction module 124) is reduced, the features are passed to the efficient channel attention module 125. The efficient channel attention module 125 can perform local cross-channel interaction of features without dimensionality reduction. This interaction between channels helps to enhance the global information; then, features of different scales are gradually fused and multiplied pixel by pixel with the original features (i.e., the processed feature maps); finally, through the first feature splicing module 122, it is fused with the output of the global average pooling module 121 to ensure further enhancement of the global semantic information and generate a deep semantic information flow, which can well extract the global semantic information expressed by the features to enhance the model's detection capability for large-scale defects.

[0057] In one embodiment, when the number of the resolution reduction modules 124 is one, the feature multiplication module 123 is connected to the channel attention module 125 and the splicing layer 1133 of the exception filter 11, for respectively receiving the sixth feature map and the processed feature map, and for performing pixel-by-pixel multiplication of the sixth feature map and the processed feature map to obtain the tenth feature map; the first feature splicing module 122 is connected to the global average pooling module 121 and the feature multiplication module 123, for respectively receiving the seventh feature map and the tenth feature map, and for performing feature splicing on the seventh feature map and the tenth feature map to obtain the deep semantic information flow.

[0058] like Figure 3 and Figure 6 As shown, the multi-scale channel autoencoder 13 (including multi-layer encoders and multi-layer decoders) directly receives multi-scale features from the teacher, extracts the normal detail information in the features and adds it pixel by pixel with the features of the corresponding scale decoded by it, and the fused features are used as the final output of each layer of decoder and passed to the next layer of the decoder; in this way, the student only receives the deep semantic information flow in the initial decoding layer, restores the semantics of the normal sample, and then fills in the normal detail information flow from the multi-scale channel autoencoder layer by layer, which is convenient for the restoration of the normal detail features of the next two layers; finally, the output of the multi-scale channel autoencoder 13 is merged with the features restored by the student to further supplement the student with the detail information of the normal part, and help the model to clearly define the abnormality detection boundary.

[0059] In one embodiment, the multi-scale channel autoencoder 13 includes a four-layer encoder and a four-layer decoder, and the four-layer encoder and the four-layer decoder are symmetrical.

[0060] like Figure 6 As shown in the figure, for each channel autoencoder, the input features will be gradually aggregated along the channel, from C to C / 16, and then directly aggregated to a single channel. The final output is obtained by fusion of features with different numbers of channels after decoding. In each channel fusion BaseBlock, the features will be sampled by convolution and then the channel attention weights will be extracted. The weights will be multiplied by the features after convolution sampling and then added, and finally output. Here, convolution sampling uses instance normalization to ensure that the normal information extraction of this instance will not be interfered with by other instances, which helps the accuracy of student network restoration.

[0061] It should be noted that the anomaly filter 11 filters the anomaly information from the teacher's features and then passes it to the global semantic embedding block 12 for semantic information enhancement, forming a semantic information stream that enters the student decoder. At the same time, the multi-scale channel autoencoder 13 receives the features from the teacher and passes the normal detail information stream to the student to help it better restore the corresponding scale features. The model learns by minimizing the sum of the multi-scale cosine distances between the teacher and the student. During the reasoning process, the larger cosine distance between the features corresponding to the teacher output and the student output indicates an anomaly, and the final anomaly map is the accumulation of multi-scale anomaly maps.

[0062] In one embodiment, an anomaly map is obtained by comparing and upsampling multi-scale features to perform anomaly detection and positioning.

[0063] It should be noted that the present invention forms a teacher-student network by combining a teacher feature extraction network (teacher network) and a student feature restoration network (student network).

[0064] Student–Teacher Networks, originally proposed by Bergmann et al., is a student-teacher (ST) framework for anomaly detection, where the teacher is a pre-trained frozen CNN. The student network is trained to simulate the teacher's output on the training images. Since the students have not seen abnormal images during training, they are usually unable to predict the teacher's output on these images, thereby achieving anomaly detection.

[0065] like Figure 7 As shown, in one embodiment, the present invention further provides an anomaly detection method implemented based on the above anomaly detection model; specifically, the anomaly detection method includes:

[0066] Step S1, preprocessing the target image to be tested to obtain a target processed image.

[0067] It should be noted that the preprocessing operation in step S1 can be autonomously combined according to the available morphology and conditions of the target to be detected.

[0068] In one embodiment, the preprocessing includes at least but is not limited to: data enhancement processing and / or normalization processing.

[0069] In one embodiment, the data enhancement processing includes at least but is not limited to any one of the following: horizontal and vertical inversion, Gaussian blur, changes in brightness, saturation, and contrast, various types of affine transformations, and center cropping.

[0070] Step S3: input the target processed image into the teacher feature extraction network to obtain a target teacher feature map.

[0071] In one embodiment, the teacher feature extraction network adopts a pre-trained deep feature extraction network.

[0072] Step S5: input the target teacher feature map into the anomaly detection model to obtain a target anomaly map, so as to implement anomaly detection of the target image to be tested based on the target anomaly map.

[0073] In one embodiment, after the target anomaly map is obtained in step S5, an anomaly score may be further obtained to obtain defect detection and location results.

[0074] In one embodiment, before the step of inputting the target teacher feature map into the anomaly detection model, the anomaly detection method further includes: training the anomaly detection model to obtain a trained anomaly detection model.

[0075] like Figure 8 As shown, in one embodiment, the training of the anomaly detection model to obtain the trained anomaly detection model includes:

[0076] Step S41: pre-process normal training sample data to obtain processed data.

[0077] It should be noted that the normal training sample data includes: a plurality of images to be tested without abnormalities.

[0078] It should be noted that the normal training sample data requires normal samples of the target to be detected, and the defects to be detected do not need to be known. They can be texture defects of the target to be detected, such as scratches and breakages on the surface, or structural defects, such as bending and missing.

[0079] It should be noted that the preprocessing operation in step S41 is based on the same principle as the preprocessing operation in step S1 above, so it will not be described in detail here.

[0080] Step S42: input the processed data into the teacher feature extraction network to obtain a teacher feature map.

[0081] Step S43: input the teacher feature map into the anomaly detection model to obtain the student feature map.

[0082] Step S44: construct a loss function of the anomaly detection model according to the teacher feature map and the student feature map, so as to train the anomaly detection model based on the loss function and obtain a trained anomaly detection model.

[0083] In one embodiment, the loss function uses cosine distance.

[0084] In one embodiment, the loss function uses the sum of multi-scale cosine distances between the teacher feature map and the student feature map.

[0085] In this embodiment, in step S5, inputting the target teacher feature map into the anomaly detection model includes: inputting the target teacher feature map into the trained anomaly detection model.

[0086] It should be noted that the present invention proposes a method for reverse distillation anomaly detection using dual information streams, which separates semantic information and detail information into student decoders to help them simultaneously reconstruct semantic and detail features, while solving the detection and positioning of local detail defects and semantic structure defects, thereby improving detection accuracy; in addition, the present invention also proposes an attention mechanism that introduces an attention mechanism combined with convolution to manufacture anomaly filters, and proposes a form of multi-scale fusion to the minimum resolution, and uses efficient channel attention to interact with channel information for global semantic enhancement; through multi-scale channel self-attention, normal detail information is extracted and passed to the student decoder to restore features to help comparison; feature restoration is performed by fusing the detail information provided by the dual information streams with the student restored information, and the student decoder is helped to jointly restore semantics and details.

[0087] The protection scope of the anomaly detection method described in the embodiment of the present invention is not limited to the execution order of the steps listed in this embodiment. All solutions implemented by adding, reducing or replacing steps in the prior art based on the principles of the present invention are included in the protection scope of the present invention.

[0088] An embodiment of the present application also provides an electronic device, which includes: a processor and a memory; the memory is used to store a computer program; the processor is used to execute the computer program stored in the memory, so that the electronic device performs the above-mentioned abnormality detection method.

[0089] In one embodiment, the present application provides a development platform (corresponding to the above-mentioned electronic device) as a server system.

[0090] In one embodiment, the server system is ubuntu18.04, the GPU is NVIDIA 3080, and the CPU is interi9-10850; the experimental environment is python3.9, the deep learning framework pytorch 1.11.0, cudatoolkit 11.3.1, torchvision0.13.1.

[0091] An embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by an electronic device, the above-mentioned abnormality detection method is implemented.

[0092] A person of ordinary skill in the art can understand that all or part of the steps in the method for implementing the above-mentioned embodiment can be completed by instructing a processor through a program, and the program can be stored in a computer-readable storage medium, and the storage medium is a non-transitory medium, such as a random access memory, a read-only memory, a flash memory, a hard disk, a solid-state hard disk, a magnetic tape, a floppy disk, an optical disc, and any combination thereof. The above-mentioned storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that includes one or more available media. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a digital video disc (DVD)), or a semiconductor medium (for example, a solid-state disk (SSD)), etc.

[0093] An embodiment of the present invention further provides an anomaly detection system, which can implement the anomaly detection method described in the present invention. However, the implementation device of the anomaly detection method described in the present invention includes but is not limited to the structure of the anomaly detection system listed in this embodiment. All structural deformations and replacements of the prior art made according to the principles of the present invention are included in the protection scope of the present invention.

[0094] like Fig. 9 As shown, in one embodiment, the present invention provides an anomaly detection system implemented based on the above anomaly detection model, and the anomaly detection system includes:

[0095] The preprocessing module 91 is used to preprocess the target image to be tested to obtain a target processed image.

[0096] The feature extraction module 92 is used to input the target processed image into the teacher feature extraction network to obtain a target teacher feature map.

[0097] The anomaly detection module 93 is used to input the target teacher feature map into the anomaly detection model to obtain a target anomaly map, so as to realize anomaly detection of the target image to be tested based on the target anomaly map.

[0098] It should be noted that the structures and principles of the preprocessing module 91, the feature extraction module 92 and the anomaly detection module 93 correspond one-to-one to the steps in the above-mentioned anomaly detection method (step S1, step S3 and step S5). The specific working principles can also be referred to the introduction of the anomaly detection method in the above-mentioned embodiment, so they will not be repeated here.

[0099] In the several embodiments provided by the present invention, it should be understood that the disclosed system, device or method can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules / units is only a logical function division, and there may be other division methods in actual implementation, such as multiple modules or units can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules or units, which can be electrical, mechanical or other forms.

[0100] The modules / units described as separate components may or may not be physically separated, and the components displayed as modules / units may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules / units may be selected according to actual needs to achieve the purpose of the embodiments of the present invention. For example, the functional modules / units in the various embodiments of the present invention may be integrated into one processing module, or each module / unit may exist physically separately, or two or more modules / units may be integrated into one module / unit.

[0101] Those of ordinary skill in the art should further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0102] The descriptions of the processes or structures corresponding to the above-mentioned figures have different emphases. For parts that are not described in detail in a certain process or structure, please refer to the relevant descriptions of other processes or structures.

[0103] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Anyone familiar with the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by a person of ordinary skill in the art without departing from the spirit and technical concept disclosed by the present invention shall still be covered by the claims of the present invention.

Claims

1. An anomaly detection method based on an anomaly detection model, characterized in that: The anomaly detection method comprises: Preprocessing the target image to be tested to obtain a target processed image; Inputting the target processed image into a teacher feature extraction network to obtain a target teacher feature map; Inputting the target teacher feature map into the anomaly detection model to obtain a target anomaly map, so as to implement anomaly detection of the target image to be tested based on the target anomaly map; The anomaly detection model is used to detect anomalies of the image to be tested based on the teacher feature map of the input image to be tested. The anomaly detection model includes: an anomaly filter, a global semantic embedding block, a multi-scale channel autoencoder and a student feature restoration network; wherein, The abnormal filter is used to filter abnormal information in the teacher feature map to obtain a processing feature map; The global semantic embedding block is connected to the abnormal filter, and the global semantic embedding block is used to receive the processing feature graph, and to enhance the semantic information of the processing feature graph to obtain a deep semantic information flow; The multi-scale channel autoencoder is used to extract the normal detail information flow in the teacher feature map; The student feature restoration network is respectively connected to the global semantic embedding block and the multi-scale channel autoencoder, and is used to obtain a student feature map based on the deep semantic information flow and the normal detail information flow; Comparing the student feature map with the teacher feature map to obtain an anomaly map, so as to implement anomaly detection of the image to be tested based on the anomaly map; The abnormal filter includes: a feature fusion layer, a feature embedding layer, and a convolution and self-attention block; wherein, The feature fusion layer is used to process the teacher feature map to obtain at least one first feature map, and to perform feature fusion on the teacher feature map and the at least one first feature map to obtain a fused feature map; The feature embedding layer is connected to the feature fusion layer, and the feature embedding layer is used to receive the fused feature map, and to perform feature embedding on the fused feature map to obtain a second feature map; The convolution and self-attention block is connected to the feature embedding layer, and the convolution and self-attention block is used to receive the second feature map, and to perform feature reorganization on the second feature map to obtain a third feature map, and to perform feature selection of self-attention on the second feature map to obtain a fourth feature map, and finally to splice the third feature map and the fourth feature map to obtain the processed feature map; The global semantic embedding block includes: a global average pooling module, a first feature concatenation module, a feature multiplication module, at least one resolution reduction module and at least one channel attention module; wherein, The resolution reduction module is connected to the channel attention module in a one-to-one correspondence, and the resolution reduction module is used to receive the processed feature map, and reduce the resolution of the processed feature map to obtain a fifth feature map; when the number of the resolution reduction modules is at least two, the resolutions of the fifth feature maps obtained after being processed by at least two resolution reduction modules are different; The channel attention module is used to receive the fifth feature map, and to perform local cross-channel interaction on the fifth feature map to obtain a sixth feature map; The global average pooling module is used to receive the processed feature map, and perform global average pooling processing on the processed feature map to obtain a seventh feature map; When the number of the resolution reduction modules is at least two, the global semantic embedding block further includes: a second feature splicing module; the second feature splicing module is connected to the channel attention module, and is used to receive the sixth feature map, and perform feature splicing on at least two sixth feature maps to obtain an eighth feature map; the feature multiplication module is connected to the second feature splicing module and the abnormal filter, and is used to receive the eighth feature map and the processed feature map respectively, and is used to multiply the eighth feature map and the processed feature map pixel by pixel to obtain a ninth feature map; the first feature splicing module is connected to the global average pooling module and the feature multiplication module, and is used to receive the seventh feature map and the ninth feature map respectively, and is used to perform feature splicing on the seventh feature map and the ninth feature map to obtain the deep semantic information flow; When the number of the resolution reduction modules is one, the feature multiplication module is connected to the channel attention module and the exception filter, for respectively receiving the sixth feature map and the processed feature map, and for performing pixel-by-pixel multiplication on the sixth feature map and the processed feature map to obtain the tenth feature map; the first feature splicing module is connected to the global average pooling module and the feature multiplication module, for respectively receiving the seventh feature map and the tenth feature map, and for performing feature splicing on the seventh feature map and the tenth feature map to obtain the deep semantic information flow.

2. The anomaly detection method based on anomaly detection model according to claim 1, characterized in that: The preprocessing includes at least: data enhancement processing and / or normalization processing.

3. The anomaly detection method based on anomaly detection model according to claim 1, characterized in that: Before the step of inputting the target teacher feature map into the anomaly detection model, the anomaly detection method further includes: Preprocessing normal training sample data to obtain processed data; the normal training sample data includes: a plurality of images to be tested without abnormalities; Inputting the processed data into the teacher feature extraction network to obtain a teacher feature graph; Inputting the teacher feature map into the anomaly detection model to obtain the student feature map; Constructing a loss function of the anomaly detection model according to the teacher feature graph and the student feature graph, so as to train the anomaly detection model based on the loss function and obtain a trained anomaly detection model; The inputting the target teacher feature map into the anomaly detection model includes: inputting the target teacher feature map into the trained anomaly detection model.

4. The anomaly detection method based on anomaly detection model according to claim 1, characterized in that: The teacher feature extraction network adopts a pre-trained deep feature extraction network.

5. An anomaly detection system based on an anomaly detection model, characterized in that: The anomaly detection system comprises: A preprocessing module is used to preprocess the target image to be tested and obtain a target processed image; A feature extraction module, used for inputting the target processed image into a teacher feature extraction network to obtain a target teacher feature map; An anomaly detection module, used for inputting the target teacher feature map into the anomaly detection model to obtain a target anomaly map, so as to realize anomaly detection of the target image to be tested based on the target anomaly map; The anomaly detection model is used to detect anomalies of the image to be tested based on the teacher feature map of the input image to be tested. The anomaly detection model includes: an anomaly filter, a global semantic embedding block, a multi-scale channel autoencoder and a student feature restoration network; wherein, The abnormal filter is used to filter abnormal information in the teacher feature map to obtain a processing feature map; The global semantic embedding block is connected to the abnormal filter, and the global semantic embedding block is used to receive the processing feature graph, and to enhance the semantic information of the processing feature graph to obtain a deep semantic information flow; The multi-scale channel autoencoder is used to extract the normal detail information flow in the teacher feature map; The student feature restoration network is respectively connected to the global semantic embedding block and the multi-scale channel autoencoder, and is used to obtain a student feature map based on the deep semantic information flow and the normal detail information flow; Comparing the student feature map with the teacher feature map to obtain an anomaly map, so as to implement anomaly detection of the image to be tested based on the anomaly map; The abnormal filter includes: a feature fusion layer, a feature embedding layer, and a convolution and self-attention block; wherein, The feature fusion layer is used to process the teacher feature map to obtain at least one first feature map, and to perform feature fusion on the teacher feature map and the at least one first feature map to obtain a fused feature map; The feature embedding layer is connected to the feature fusion layer, and the feature embedding layer is used to receive the fused feature map, and to perform feature embedding on the fused feature map to obtain a second feature map; The convolution and self-attention block is connected to the feature embedding layer, and the convolution and self-attention block is used to receive the second feature map, and to perform feature reorganization on the second feature map to obtain a third feature map, and to perform feature selection of self-attention on the second feature map to obtain a fourth feature map, and finally to splice the third feature map and the fourth feature map to obtain the processed feature map; The global semantic embedding block includes: a global average pooling module, a first feature concatenation module, a feature multiplication module, at least one resolution reduction module and at least one channel attention module; wherein, The resolution reduction module is connected to the channel attention module in a one-to-one correspondence, and the resolution reduction module is used to receive the processed feature map, and reduce the resolution of the processed feature map to obtain a fifth feature map; when the number of the resolution reduction modules is at least two, the resolutions of the fifth feature maps obtained after being processed by at least two resolution reduction modules are different; The channel attention module is used to receive the fifth feature map, and to perform local cross-channel interaction on the fifth feature map to obtain a sixth feature map; The global average pooling module is used to receive the processed feature map, and perform global average pooling processing on the processed feature map to obtain a seventh feature map; When the number of the resolution reduction modules is at least two, the global semantic embedding block further includes: a second feature splicing module; the second feature splicing module is connected to the channel attention module, and is used to receive the sixth feature map, and perform feature splicing on at least two sixth feature maps to obtain an eighth feature map; the feature multiplication module is connected to the second feature splicing module and the abnormal filter, and is used to receive the eighth feature map and the processed feature map respectively, and is used to multiply the eighth feature map and the processed feature map pixel by pixel to obtain a ninth feature map; the first feature splicing module is connected to the global average pooling module and the feature multiplication module, and is used to receive the seventh feature map and the ninth feature map respectively, and is used to perform feature splicing on the seventh feature map and the ninth feature map to obtain the deep semantic information flow; When the number of the resolution reduction modules is one, the feature multiplication module is connected to the channel attention module and the exception filter, for respectively receiving the sixth feature map and the processed feature map, and for performing pixel-by-pixel multiplication on the sixth feature map and the processed feature map to obtain the tenth feature map; the first feature splicing module is connected to the global average pooling module and the feature multiplication module, for respectively receiving the seventh feature map and the tenth feature map, and for performing feature splicing on the seventh feature map and the tenth feature map to obtain the deep semantic information flow.

6. An electronic device, characterized in that: The electronic device comprises: a processor and a memory; The memory is used to store computer programs; The processor is used to execute the computer program stored in the memory so that the electronic device executes the anomaly detection method based on the anomaly detection model described in any one of claims 1 to 4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by an electronic device, the abnormality detection method based on the abnormality detection model described in any one of claims 1 to 4 is implemented.

Images