Unsupervised anomaly detection method and system based on self-supervised data enhancement and multi-resolution knowledge distillation

Abstract

The present invention relates to an unsupervised anomaly detection method and system based on self-supervised data enhancement and multi-resolution knowledge distillation. The method comprises: cutting and randomly pasting normal detection images to generate abnormal image samples, extracting feature representations of abnormal image samples at different levels by using multi-resolution knowledge distillation technology through deep convolutional neural network, and performing feature fusion; locating abnormal areas according to image gradient information; and constructing an abnormal scoring system to calculate the degree of abnormality in abnormal areas. By cutting and pasting images, the diversity of data can be enriched; by using multi-resolution knowledge distillation technology to extract feature representations at different levels, a richer feature space is provided for abnormal detection; based on gradient information, abnormal areas can be accurately located, and an abnormal scoring system constructed according to the fused features can evaluate the degree of abnormality, provide decision support for abnormal detection, and improve the accuracy and real-time performance of abnormal detection.

Description

Technical Field

[0001] The present invention relates to the technical field of industrial visual inspection, and in particular to an unsupervised anomaly detection method and system based on self-supervised data enhancement and multi-resolution knowledge distillation. Background Art

[0002] In the field of industrial visual inspection, the development of anomaly detection technology is of great significance for improving production efficiency and ensuring product quality. With the advancement of automation and intelligent manufacturing technology, the accuracy and real-time requirements for anomaly detection are becoming increasingly stringent. Traditional anomaly detection technologies mostly rely on supervised learning, which requires a large number of labeled normal and abnormal samples for training. However, in actual industrial applications, due to the diversity and unpredictability of anomaly types, obtaining enough abnormal samples is not only costly but also almost impossible. In addition, existing methods still face challenges in processing complex abnormal patterns and accurately locating abnormal areas.

[0003] Traditional anomaly detection techniques, such as autoencoders, generative adversarial networks (GANs), and single-class classifiers, mainly identify anomalies through reconstruction errors or the discriminative ability of generative models. Although these methods can reveal the abnormality of data to a certain extent, they are insufficient in simulating the diversity and complexity of anomalies, especially in small sample learning scenarios, and their performance is difficult to meet the needs of industrial-grade applications. In addition, the generalization ability of feature representation and the accuracy of anomaly localization of existing technologies need to be improved. Single-class classifiers usually rely on a single data enhancement strategy, such as Cutout, which can introduce local irregularities but is difficult to fully simulate the types of anomalies in the real world. In terms of knowledge distillation, previous work has mainly focused on imitating the output layer of the teacher network, but has failed to fully utilize the rich features of the intermediate layers of the network.

[0004] Therefore, traditional anomaly detection methods have the problems of low efficiency and low accuracy. Summary of the invention

[0005] Based on this, in order to solve the above technical problems, an unsupervised anomaly detection method and system based on self-supervised data enhancement and multi-resolution knowledge distillation are provided, which can improve the accuracy and real-time performance of anomaly detection.

[0006] An unsupervised anomaly detection method based on self-supervised data enhancement and multi-resolution knowledge distillation, the method comprising:

[0007] Acquire a normal detection image, cut and randomly paste the normal detection image, and generate a simulated abnormal image sample;

[0008] Inputting the abnormal image samples into a deep convolutional neural network, and extracting feature representations of the abnormal image samples at different levels by using a multi-resolution knowledge distillation technique through the deep convolutional neural network;

[0009] Acquire corresponding image gradient information based on the abnormal image sample, and locate the abnormal area according to the image gradient information;

[0010] Performing feature fusion on each of the feature representations at different levels to obtain fused features; and constructing an anomaly scoring system based on the fused features;

[0011] The abnormal region is scored by the abnormal scoring system to calculate the degree of abnormality of the abnormal region.

[0012] In one embodiment, obtaining a normal detection image, cutting and randomly pasting the normal detection image to generate a simulated abnormal image sample includes:

[0013] Acquire a normal detection image, randomly select a target area from the normal detection image, and cut the target area to obtain a cut area image;

[0014] The cut region image is randomly pasted to any position in the normal detection image to generate a simulated abnormal image sample.

[0015] In one embodiment, randomly pasting the cut region image to any position in the normal detection image includes:

[0016] Transform the cut region image, and randomly paste the transformed region image to any position in the normal detection image;

[0017] The transformation process includes rotation, scaling and color dithering.

[0018] In one embodiment, the method further comprises:

[0019] Marking the abnormal image samples, and inputting the marked abnormal image samples and the normal detection images into an image detection model;

[0020] Outputting an image distinction detection result through the image detection model, and adjusting parameters in the image detection model according to the image distinction detection result, so as to complete the training optimization of the image detection model;

[0021] The image detection model is used to distinguish abnormal image samples from normal detection images.

[0022] In one embodiment, the feature representation of the abnormal image sample at different levels is extracted by the deep convolutional neural network using multi-resolution knowledge distillation technology, including:

[0023] Obtain a pre-trained deep convolutional neural network, and capture local and texture information in the abnormal image sample through a first network layer in the deep convolutional neural network; and capture semantic information in the abnormal image sample through a second network layer in the deep convolutional neural network;

[0024] Constructing a feature representation of multi-resolution information as a multi-resolution feature according to the local and texture information and the semantic information;

[0025] The multi-resolution features are transferred using multi-resolution knowledge distillation technology to extract feature representations at different levels.

[0026] In one embodiment, obtaining corresponding image gradient information based on the abnormal image sample, and locating the abnormal area according to the image gradient information includes:

[0027] Calculating image gradient information based on the abnormal image sample, and analyzing the size and distribution of the image gradient information to obtain an analysis result;

[0028] The position information of the abnormal area is determined according to the analysis result, and an abnormal map is generated according to the position information and the image gradient information up-sampled to the spatial resolution to highlight the abnormal area.

[0029] In one embodiment, the feature representations at different levels are fused to obtain fused features, including:

[0030] Determine the feature weights corresponding to the feature representations at different levels respectively;

[0031] The feature representations at different levels are weighted and summed according to the corresponding feature weights to obtain fused features.

[0032] In one embodiment, the method further comprises:

[0033] Determine anomaly detection decision thresholds;

[0034] The fused features are input into the anomaly scoring system, and the anomaly scoring system converts the fused features into anomaly scores based on the anomaly detection decision threshold.

[0035] An unsupervised anomaly detection system based on self-supervised data enhancement and multi-resolution knowledge distillation, the system comprising:

[0036] An image processing module is used to obtain a normal detection image, cut and randomly paste the normal detection image, and generate a simulated abnormal image sample;

[0037] A multi-resolution feature extraction module, used to input the abnormal image samples into a deep convolutional neural network, and extract feature representations of the abnormal image samples at different levels by using the multi-resolution knowledge distillation technology through the deep convolutional neural network;

[0038] An abnormality detection and positioning module, used to obtain corresponding image gradient information based on the abnormal image sample, and locate the abnormal area according to the image gradient information;

[0039] A feature fusion module is used to fuse the feature representations at different levels to obtain fused features; and to construct an anomaly scoring system based on the fused features;

[0040] The abnormality scoring module is used to score the abnormal area through the abnormality scoring system and calculate the abnormality degree of the abnormal area.

[0041] The above-mentioned unsupervised anomaly detection method and system based on self-supervised data enhancement and multi-resolution knowledge distillation can enrich the diversity of data by cutting and pasting normal detection images; multi-resolution knowledge distillation technology is used to extract feature representations at different levels, providing a richer feature space for anomaly detection; based on gradient information, the abnormal area can be accurately located, and the anomaly scoring system constructed according to the fused features can evaluate the degree of anomaly, provide decision support for anomaly detection, and improve the accuracy and real-time performance of anomaly detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 is an application environment diagram of an unsupervised anomaly detection method based on self-supervised data enhancement and multi-resolution knowledge distillation in one embodiment;

[0043] Figure 2 1 is a flow chart of an unsupervised anomaly detection method based on self-supervised data enhancement and multi-resolution knowledge distillation in one embodiment;

[0044] Figure 3 is a structural block diagram of an unsupervised anomaly detection system based on self-supervised data enhancement and multi-resolution knowledge distillation in one embodiment;

[0045] Figure 4 Schematic diagram of the structure of a self-supervised data enhancement module in one embodiment;

[0046] Figure 5 is a schematic diagram of the structure of a multi-resolution feature extraction module in one embodiment;

[0047] Figure 6 A schematic diagram of the structure of an abnormality detection and positioning module in one embodiment;

[0048] Figure 7 Schematic diagram of the structure of a feature fusion module and anomaly scoring module in one embodiment;

[0049] Figure 8 Schematic diagram of the structure of an unsupervised anomaly detection system based on self-supervised data enhancement and multi-resolution knowledge distillation in one embodiment;

[0050] Fig. 9 is a schematic diagram of a process flow applied to a semiconductor manufacturing process in one embodiment;

[0051] Fig.10 is a schematic diagram of a process for quality control of automobile parts in one embodiment;

[0052] Fig.11 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0054] The unsupervised anomaly detection method based on self-supervised data enhancement and multi-resolution knowledge distillation provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Figure 1 As shown, the application environment includes a computer device 110. The computer device 110 can obtain a normal detection image, cut and randomly paste the normal detection image, and generate a simulated abnormal image sample; the computer device 110 can input the abnormal image sample into a deep convolutional neural network, and extract the feature representation of the abnormal image sample at different levels by using the multi-resolution knowledge distillation technology through the deep convolutional neural network; the computer device 110 can obtain the corresponding image gradient information based on the abnormal image sample, and locate the abnormal area according to the image gradient information; the computer device 110 can perform feature fusion on each feature representation at different levels to obtain the fused feature; construct an abnormal scoring system according to the fused feature; the computer device 110 can score the abnormal area through the abnormal scoring system and calculate the abnormal degree of the abnormal area. Among them, the computer device 110 can be, but not limited to, various personal computers, laptops, smart phones, robots and other devices.

[0055] In one embodiment, Figure 2As shown, an unsupervised anomaly detection method based on self-supervised data enhancement and multi-resolution knowledge distillation is provided, including the following steps:

[0056] Step 202, obtaining a normal detection image, cutting and randomly pasting the normal detection image, and generating a simulated abnormal image sample.

[0057] The computer device can obtain a normal detection image and exercise the ability to identify abnormal images by implementing self-supervised data enhancement. In this embodiment, the computer device can generate simulated abnormal image samples by cutting a local area of ​​the normal detection image and randomly pasting the cut area to another position. By simulating the diversity of abnormalities in the real world, rich training signals can be provided.

[0058] In one embodiment, an unsupervised anomaly detection method based on self-supervised data enhancement and multi-resolution knowledge distillation is provided, which may also include an image processing process, wherein the specific process includes: obtaining a normal detection image, randomly selecting a target area from the normal detection image, and cutting the target area to obtain a cut area image; and randomly pasting the cut area image to any position in the normal detection image to generate a simulated abnormal image sample.

[0059] Specifically, the computer device can randomly select a region R from the normal detection image X, and the size and position of the region R are randomly determined. The cutting operation can be described by a mathematical expression: ;in, is the randomly selected X upper left corner coordinate of the normal detection image, are the width and height of region R.

[0060] The computer device can randomly paste the cut region image to any position in the normal detection image. Specifically, the computer device can randomly paste the cut region R to a new position of the image X. The pasting operation can be expressed as: ;in, is the upper left corner coordinate of the randomly selected new position.

[0061] In one embodiment, an unsupervised anomaly detection method based on self-supervised data enhancement and multi-resolution knowledge distillation is provided, which may also include a process of transforming an image, wherein the specific process includes: transforming the cut region image, and randomly pasting the transformed region image to any position in a normal detection image; wherein the transformation process includes rotation, scaling, and color jittering.

[0062] In order to enhance the diversity of data, transformations such as rotation, scaling, and color jitter can be introduced during the cutting and pasting process, so that the enhanced samples are closer to the diversity of real anomalies.

[0063] In one embodiment, an unsupervised anomaly detection method based on self-supervised data enhancement and multi-resolution knowledge distillation is provided, which may also include a model training process, wherein the specific process includes: marking abnormal image samples, and inputting the marked abnormal image samples and normal detection images into the image detection model; outputting image distinction detection results through the image detection model, and adjusting the parameters in the image detection model according to the image distinction detection results, to complete the training optimization of the image detection model; wherein the image detection model is used to distinguish abnormal image samples from normal detection images.

[0064] Image detection models need to learn the representation of normal patterns from unlabeled data. Specifically, through the simulated abnormal image samples generated by the data augmentation strategy, the image detection model can be trained to distinguish normal images from abnormal images. The specific objective function can be expressed as: ;in, is the data distribution of normal images, is the forward propagation function of the model, is the image after applying the data enhancement strategy, is the mean square error loss function, which is used to measure the consistency of the original image and the enhanced image in the feature space. Through the samples generated by the data augmentation strategy, the model is trained to recognize irregular patterns in the image, that is, abnormal images.

[0065] In this embodiment, in order to train a model that can effectively identify anomalies, it is necessary to design a loss function that comprehensively considers data enhancement and feature learning. The loss function should not only consider the consistency of features, but also the sensitivity of the model to anomalies. The loss function can be expressed as: ;in, represents the parameters in the model, is the regularization coefficient, which is used to control the strength of the regularization term to prevent overfitting. Self-supervised data enhancement can generate rich simulated abnormal samples and train the model through self-supervised learning to enable it to recognize irregular patterns.

[0066] By cutting and randomly pasting normal images, simulated abnormal samples are generated. This process not only enriches the diversity of data, but also enhances the model's ability to recognize irregular patterns, providing a more comprehensive data representation for subsequent feature learning.

[0067] Step 204: Input the abnormal image samples into a deep convolutional neural network, and use the deep convolutional neural network to extract feature representations of the abnormal image samples at different levels using multi-resolution knowledge distillation technology.

[0068] After computer equipment has undergone data enhancement and self-supervised learning, it can use pre-trained deep convolutional neural networks to extract feature representations of abnormal image samples at different levels through multi-resolution knowledge distillation technology, which can enable the student network to learn the deep features of normal data at multiple scales, providing a richer feature space for anomaly detection.

[0069] Among them, the pre-trained expert network is crucial for multi-resolution feature extraction. In this embodiment, a deep convolutional network pre-trained on a large-scale dataset (such as ImageNet) can be used as an expert network, which can capture rich visual features and provide a basis for subsequent knowledge distillation. The adaptation process includes adjusting the network to adapt to specific industrial visual inspection tasks, which involves fine-tuning network parameters or modifying the network structure to better capture task-related features.

[0070] In one embodiment, an unsupervised anomaly detection method based on self-supervised data enhancement and multi-resolution knowledge distillation is provided, which may also include a multi-resolution feature extraction process, the specific process including: obtaining a pre-trained deep convolutional neural network, capturing local and texture information in abnormal image samples through a first network layer in the deep convolutional neural network; capturing semantic information in abnormal image samples through a second network layer in the deep convolutional neural network; constructing a feature representation of multi-resolution information based on local and texture information and semantic information as a multi-resolution feature; and using multi-resolution knowledge distillation technology to transfer multi-resolution features to extract feature representations at different levels.

[0071] Multi-resolution feature extraction is achieved by analyzing images at different levels. In a deep convolutional network, the first network layer, the bottom layer, usually captures local and texture information, while the second network layer, the upper layer, captures more abstract semantic information. By extracting features at multiple levels, a feature representation that integrates information at different resolutions can be constructed. The formula can be expressed as: ;in, is the input image, It is Layer network parameters, It is The feature extraction function of the layer, It is Features extracted by the layer.

[0072] Next, the computer device can use a knowledge distillation strategy to transfer multi-resolution features, thereby extracting feature representations at different levels. Specifically, knowledge distillation (KD) is a model compression technology that is implemented by transferring the knowledge of a large and complex model (teacher network) to a small and simple model (student network). In this embodiment, knowledge distillation is used to transfer multi-resolution features: ;in, is the number of levels considered, It is The weight coefficient of the layer, Is the teacher network The output features of the layer, Is the student network The output features of the layer.

[0073] Step 206: Acquire corresponding image gradient information based on the abnormal image sample, and locate the abnormal area according to the image gradient information.

[0074] Combining the advantages of multi-resolution features and self-supervised learning, the abnormal area is accurately located by analyzing the output image gradient information; pixel-level anomaly mapping is achieved, providing high-precision anomaly location results for industrial visual inspection.

[0075] In one embodiment, an unsupervised anomaly detection method based on self-supervised data enhancement and multi-resolution knowledge distillation is provided, which may also include a process of locating abnormal areas based on gradient information. The specific process includes: calculating image gradient information based on abnormal image samples, and analyzing the size and distribution of the image gradient information to obtain analysis results; determining the position information of the abnormal area based on the analysis results, and generating an abnormal map based on the position information and the image gradient information up-sampled to the spatial resolution to highlight the abnormal area.

[0076] Among them, gradient information is the key in the back-propagation process, which indicates the degree of influence of the input data on the output result. In the context of anomaly detection, gradient information can be used to identify areas in the image that cause the model to output abnormally high scores: ;in, is the anomaly score, is the input image, Represents the gradient with respect to the input image.

[0077] By analyzing the size and distribution of the gradient, the computer device can determine which areas in the image contribute most to the anomaly score, thereby locating the abnormal area. Specifically, the implementation steps of anomaly location may include: gradient calculation: first, calculating the gradient of the anomaly score relative to the input image; gradient thresholding: applying a threshold to determine significant gradient values, which indicate the location of the abnormal area; anomaly map generation: based on the thresholded gradient, generating an anomaly map to highlight the abnormal area.

[0078] Among them, pixel-level anomaly mapping is achieved by upsampling the gradient information to the spatial resolution of the input image. The pixel-level anomaly mapping process can be completed using upsampling techniques such as nearest neighbor interpolation, bilinear interpolation, or more advanced convolutional neural network layers: ;in, is the exception mapping, is an upsampling operation.

[0079] Step 208, performing feature fusion on the feature representations at different levels to obtain fused features; and constructing an anomaly scoring system based on the fused features.

[0080] After obtaining the multi-resolution feature representation, the feature information at different levels can be integrated through feature fusion technology to obtain a more comprehensive image representation. Based on the fused features, an anomaly scoring system is constructed in this embodiment to evaluate the degree of anomaly of an image or image block and provide decision support for anomaly detection.

[0081] Among them, feature fusion is the process of integrating features of different levels and resolutions. In this embodiment, a feature weighted network can be used to capture the global and local information of the image to improve the accuracy of anomaly detection.

[0082] In one embodiment, an unsupervised anomaly detection method based on self-supervised data enhancement and multi-resolution knowledge distillation is provided, which may also include a feature fusion process, and the specific process includes: determining the feature weights corresponding to each feature representation at different levels respectively; performing weighted summation of each feature representation at different levels according to the corresponding feature weights to obtain a fused feature.

[0083] Feature weighting is achieved by assigning different weights to features at different levels, and these weights can be learned through an optimization process: ;in, It is a feature representation that is fused after feature weighting. According to their respective weights The feature representation obtained after weighted summation is used to improve the accuracy of anomaly detection. In order to ensure that the fused features can effectively reconstruct the original input image and improve the accuracy of anomaly detection, a feature fusion reconstruction loss function is designed in this embodiment. , which measures the fusion feature representation Through a reconstruction network Reconstructed image With the original input image The difference between.

[0084] Step 210: Score the abnormal area through an abnormality scoring system to calculate the abnormality degree of the abnormal area.

[0085] Based on the fused features, an abnormal scoring system is constructed in this embodiment. To evaluate the degree of abnormality of each image or image patch. The scoring network is a fully connected or convolutional neural network that converts the fused features into anomaly scores: in, is the feature representation after fusion.

[0086] In one embodiment, an unsupervised anomaly detection method based on self-supervised data enhancement and multi-resolution knowledge distillation is provided, which may also include a process of setting a decision threshold to convert features into anomaly scores. The specific process includes: determining anomaly detection decision threshold; inputting the fused features into an anomaly scoring system, and converting the fused features into anomaly scores based on the anomaly detection decision threshold through the anomaly scoring system.

[0087] In order to convert the anomaly score into the final result of anomaly detection, a decision threshold needs to be set. The decision threshold is optimized based on the performance on the validation set: Among them, the calculation of the anomaly score is through the scoring network The fused features are Convert to anomaly score ; The decision process of anomaly detection is based on a decision threshold , which is the threshold used to convert the anomaly score into a binary anomaly label .

[0088] In this embodiment, in order to ensure the accuracy of the anomaly scoring system and make the decision threshold To optimally distinguish normal and abnormal samples, an anomaly score loss function is needed that not only penalizes the prediction error of the anomaly score, but also penalizes the difference between the decision threshold and the actual label. The anomaly score loss function is defined as: ; where N is the total number of samples, is the weight coefficient , is the anomaly score of the i-th sample predicted by the model, that is, the actual anomaly label of the i-th sample (a value between 0 and 1, indicating the probability or degree of anomaly).

[0089] In one embodiment, in order to ensure that the unsupervised anomaly detection method based on self-supervised data enhancement and multi-resolution knowledge distillation can effectively learn from normal samples and accurately detect anomalies, a set of comprehensive loss functions is designed in this embodiment, including data enhancement loss , knowledge distillation loss and anomaly score loss Through a carefully designed optimization strategy, it is possible to balance the various losses during the training process, ensuring high sensitivity to abnormal samples while maintaining a high recognition rate for normal samples. The total loss function is the sum of the above losses, that is, , allowing for balancing different objectives during training.

[0090] In the unsupervised anomaly detection method based on self-supervised data enhancement and multi-resolution knowledge distillation provided in this application, simulated abnormal samples are generated through self-supervised data enhancement, which enhances the recognition ability of irregular patterns. Compared with traditional unsupervised methods based on reconstruction or clustering, on the MVTec AD dataset, experimental results show that the method in this application achieves higher image-level AUC and AP values ​​than the existing method ARAE, with an average AUC increase of about 5% and an AP value increase of about 4%, showing higher detection accuracy.

[0091] Since this application adopts a multi-resolution knowledge distillation strategy, the student network can learn the deep features of normal data at multiple scales, so that it can show better generalization ability when facing new types of anomalies. On the VisA dataset, when facing new types of anomalies, it shows better generalization ability than the existing method ARAE, and the image-level AUC is improved by about 3%.

[0092] Anomaly detection and localization accurately locates abnormal areas by analyzing the output gradient information, effectively solving the problem of "normal forgetting" that may occur in the student network during training. In the complex texture category in the MVTec AD dataset, the method in this application reduces the false detection rate by about 10%. Compared with the memory-based method, this application does not need to store a large number of normal features, but integrates feature information at different levels through feature fusion and anomaly scoring, significantly reducing storage requirements.

[0093] Through pixel-level anomaly mapping, high-precision anomaly localization results are provided for industrial visual inspection, which is of great significance for subsequent anomaly analysis and decision-making, especially in industrial application scenarios that require rapid response and precise maintenance.

[0094] Since the method in this application is unsupervised, it does not require annotation information of abnormal samples, which greatly reduces the cost and workload of manual annotation in practical applications. And through feature fusion and anomaly scoring, it can not only detect and locate anomalies, but also provide interpretable information about abnormal areas, which is crucial for understanding the decision-making process of the model and improving user trust.

[0095] It should be understood that, although the various steps in the above-mentioned flow chart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the above-mentioned flow chart may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0096] In one embodiment, Figure 3 As shown, an unsupervised anomaly detection system based on self-supervised data enhancement and multi-resolution knowledge distillation is provided, including: an image processing module 310, a multi-resolution feature extraction module 320, an anomaly detection and positioning module 330, a feature fusion module 340 and an anomaly scoring module 350, wherein:

[0097] The image processing module 310 is used to obtain a normal detection image, cut and randomly paste the normal detection image, and generate a simulated abnormal image sample;

[0098] A multi-resolution feature extraction module 320 is used to input the abnormal image samples into a deep convolutional neural network, and extract feature representations of the abnormal image samples at different levels by using the multi-resolution knowledge distillation technology through the deep convolutional neural network;

[0099] Anomaly detection and positioning module 330, used to obtain corresponding image gradient information based on abnormal image samples, and locate abnormal areas according to the image gradient information;

[0100] The feature fusion module 340 is used to fuse the feature representations at different levels to obtain fused features; and to construct an anomaly scoring system based on the fused features;

[0101] The abnormality scoring module 350 is used to score the abnormal area through the abnormality scoring system and calculate the abnormality degree of the abnormal area.

[0102] In one embodiment, the image processing module 310 is also used to obtain a normal detection image, randomly select a target area from the normal detection image, and cut the target area to obtain a cut area image; randomly paste the cut area image to any position in the normal detection image to generate a simulated abnormal image sample.

[0103] In one embodiment, the image processing module 310 is further used to transform the cut region image, and randomly paste the transformed region image to any position in the normal detection image; wherein the transformation process includes rotation, scaling and color jittering.

[0104] In one embodiment, since the image processing module can realize self-supervised data enhancement by performing image processing, in this embodiment, the image processing module can be called a self-supervised data enhancement module. The self-supervised data enhancement module is the basis for realizing anomaly detection, and enhances the model's ability to recognize irregular patterns by simulating abnormal samples. The structure of the self-supervised data enhancement module is as follows: Figure 4 As shown, it includes: image input: responsible for providing normal image data to be processed, which is the starting point of the data enhancement process; cropping and pasting operations: including the operations of randomly cropping a certain area of ​​the image and pasting it to a new position to simulate abnormalities; enhanced image feature extraction: feature extraction of data-enhanced images to provide necessary feature representation for self-supervised learning; self-supervised learning distinction: using the extracted features, the model is trained through a self-supervised learning framework to distinguish between normal images and enhanced images.

[0105] like Figure 4 As shown in the figure, arrows are used to represent data flow and information transfer in each part. The image input provides data for the cropping and pasting operations to generate enhanced image features, which are then input into the enhanced image feature extraction. The extracted features are input into the self-supervised learning distinction to train the model to distinguish between normal and enhanced images; finally, the outputs of all modules are summarized at the end point B, indicating the completion of the self-supervised data enhancement process.

[0106] In one embodiment, an unsupervised anomaly detection system based on self-supervised data enhancement and multi-resolution knowledge distillation is provided, which may also include a model training module for marking abnormal image samples, and inputting the marked abnormal image samples and normal detection images into an image detection model; outputting image distinction detection results through the image detection model, and adjusting parameters in the image detection model according to the image distinction detection results to complete training optimization of the image detection model; wherein the image detection model is used to distinguish abnormal image samples from normal detection images.

[0107] In one embodiment, the multi-resolution feature extraction module 320 is also used to obtain a pre-trained deep convolutional neural network, capture local and texture information in abnormal image samples through the first network layer in the deep convolutional neural network; capture semantic information in abnormal image samples through the second network layer in the deep convolutional neural network; construct a feature representation of multi-resolution information based on the local and texture information and semantic information as a multi-resolution feature; use the multi-resolution knowledge distillation technology to transfer the multi-resolution features and extract feature representations at different levels.

[0108] In one embodiment, the multi-resolution feature extraction module uses a pre-trained expert network to extract feature representations of the image at different levels through multi-resolution knowledge distillation technology. The structure of the multi-resolution feature extraction module is as follows: Figure 5 As shown in the figure, it includes: pre-trained expert network: a deep convolutional network pre-trained on a large-scale dataset, which provides a basis for multi-resolution feature extraction; student network construction: construct a simplified student network to learn the features of the expert network through knowledge distillation; multi-resolution feature extraction: extract feature representations of images at different levels to capture information at different scales; feature fusion strategy: use feature fusion technology to integrate features at different levels to obtain a more comprehensive image feature representation; anomaly scoring system: based on the fused features, construct an anomaly scoring system to evaluate the degree of abnormality of images or image blocks.

[0109] like Figure 5 As shown in the figure, arrows are used to represent data flow and information transfer in each part. The pre-trained expert network provides knowledge to the student network to build the feature representation of the learning expert network; the student network extracts multi-resolution features, which are input into the multi-resolution feature extraction; the output features are fused in the feature fusion strategy to generate fused feature representations, which are input into the anomaly scoring system to evaluate the anomaly score. Finally, the outputs of all modules are summarized at the end point B, indicating the completion of the multi-resolution feature extraction process.

[0110] In one embodiment, the anomaly detection and positioning module 330 is also used to calculate image gradient information based on abnormal image samples, and analyze the size and distribution of the image gradient information to obtain analysis results; determine the location information of the abnormal area according to the analysis results, and generate an abnormal map based on the location information and the image gradient information upsampled to the spatial resolution to highlight the abnormal area.

[0111] In one embodiment, the anomaly detection and location module can demonstrate how to accurately locate the anomaly area through the output gradient information. Figure 6 As shown, the anomaly detection and localization module may include: input image processing: preprocessing the image to be detected in preparation for feature extraction; feature representation extraction: responsible for extracting deep feature representations from the input image, which are crucial for subsequent anomaly scoring; anomaly score calculation: calculating the anomaly score based on the extracted feature representation to quantify the degree of anomaly in the image; gradient information acquisition: calculating the gradient of the anomaly score relative to the input image, and the gradient information indicates the location of the abnormal area; abnormal area localization: using gradient information to determine and locate the abnormal area in the image.

[0112] like Figure 6As shown in the figure, arrows are used to represent data flow and information transmission in each part. Input image processing prepares image data, which is provided to feature representation extraction. The extracted feature representation is input to the anomaly score calculation; the calculated anomaly score drives the gradient information acquisition, and the acquired gradient information guides the location of the abnormal area. The located abnormal area is fed back to the anomaly score calculation and gradient information acquisition to optimize the scoring and gradient calculation process. Finally, the outputs of all modules are summarized at the end point B, indicating the completion of the anomaly detection and location process.

[0113] In one embodiment, the feature fusion module 340 is further used to respectively determine the feature weights corresponding to each feature representation at different levels; perform weighted summation on each feature representation at different levels according to the corresponding feature weights to obtain fused features.

[0114] In one embodiment, the anomaly scoring module 350 is further used to determine an anomaly detection decision threshold; the fused features are input into an anomaly scoring system, and the anomaly scoring system converts the fused features into an anomaly score based on the anomaly detection decision threshold.

[0115] In one embodiment, the feature fusion module and the anomaly scoring module integrate multi-resolution features and perform anomaly scoring to provide decision support for anomaly detection. Figure 7 As shown, the structure of the feature fusion module and the anomaly scoring module may include: multi-resolution feature set: it brings together features extracted from different resolutions to provide raw materials for feature fusion; feature fusion operation: it is responsible for effectively fusing multi-resolution features to generate a comprehensive feature representation; anomaly scoring network construction: it builds and trains a network for anomaly scoring based on fused features; anomaly detection decision: it makes the final decision of anomaly detection based on the anomaly score and the preset threshold.

[0116] like Figure 7 As shown in the figure, arrows are used to represent data flow and information transfer in each part. The multi-resolution feature set provides a feature set, which is input into the feature fusion operation. The fused features are input into the anomaly scoring network construction. The constructed network outputs anomaly scores, which are input into anomaly detection decisions. The decisions made are fed back to the anomaly scoring network construction to optimize the scoring network. Finally, the outputs of all modules are summarized at the end point B, indicating the completion of the feature fusion and anomaly scoring process.

[0117] In one embodiment, Figure 8As shown, an unsupervised anomaly detection system based on self-supervised data enhancement and multi-resolution knowledge distillation can include: a self-supervised data enhancement module: responsible for generating simulated abnormal samples, enriching the diversity of data through data enhancement strategies, and enhancing the model's recognition ability of irregular patterns; a multi-resolution feature extraction module: using a pre-trained expert network, extracting feature representations of images at different levels through multi-resolution knowledge distillation technology; anomaly detection and positioning module: combining the advantages of multi-resolution features and self-supervised learning, accurately locating abnormal areas by analyzing the gradient information output by the model; feature fusion and anomaly scoring module: integrating feature information at different levels through feature fusion technology, building an anomaly scoring system, and providing decision support for anomaly detection.

[0118] like Figure 8 As shown in the figure, arrows are used to represent data flow and information transmission in each part. The output of the self-supervised data enhancement module points to the multi-resolution feature extraction module through an arrow, indicating the transmission of simulated abnormal samples; the output of the multi-resolution feature extraction module is passed to the anomaly detection and localization module through an arrow, indicating the transmission of multi-resolution features; the output of the anomaly detection and localization module is passed to the feature fusion and anomaly scoring module, indicating the transmission of abnormal area positioning information; the feature fusion and anomaly scoring module calculates the anomaly score and feeds back the scoring result to each module to guide dynamic adjustment.

[0119] In one embodiment, an unsupervised anomaly detection method and system based on self-supervised data enhancement and multi-resolution knowledge distillation in the present application is applied to a defect detection system in a semiconductor manufacturing process, aiming to improve the accuracy and efficiency of defect detection and ensure high quality standards of semiconductor products.

[0120] Specifically, the specific process is as follows Fig. 9 As shown, including:

[0121] Image data acquisition: During the semiconductor manufacturing process, high-resolution cameras capture images of silicon wafers. These images contain microscopic details of the wafer surface.

[0122] Self-supervised data enhancement: The self-supervised data enhancement module of the present invention is used to generate simulated abnormal samples by cutting and randomly pasting normal silicon wafer images, thereby enhancing the model's ability to recognize defect patterns;

[0123] Multi-resolution feature extraction: Using pre-trained expert networks and multi-resolution knowledge distillation technology, feature representations of silicon wafer images at different levels are extracted to provide a rich feature space for anomaly detection;

[0124] Anomaly detection and location: Combining the advantages of multi-resolution features and self-supervised learning, the model outputs gradient information to accurately locate abnormal areas in silicon wafer images and achieve pixel-level anomaly mapping.

[0125] Feature fusion and anomaly scoring: Feature fusion technology is used to integrate feature information at different levels, build an anomaly scoring system, evaluate the degree of anomaly in the image, and provide decision support for defect detection;

[0126] Defect data processing: Based on the results of the anomaly scoring system, the detected defects are classified and recorded for subsequent analysis and processing;

[0127] Feedback and model iteration: Feedback defect detection results to the model training process, continuously optimize and iterate the model to adapt to new types of defects that may occur in the production process.

[0128] In one embodiment, an unsupervised anomaly detection method and system based on self-supervised data enhancement and multi-resolution knowledge distillation in the present application is applied to the quality control of automobile parts to achieve rapid detection and high-precision positioning of surface defects of automobile parts.

[0129] Specifically, the specific process is as follows Fig.10 As shown, including:

[0130] Parts image acquisition: On the automotive parts production line, high-definition cameras are used to capture images of parts for subsequent quality inspection and analysis;

[0131] Self-supervised data enhancement: Use the self-supervised data enhancement module to perform data enhancement on normal part images, generate simulated abnormal samples, and provide diverse training signals for the model;

[0132] Multi-resolution feature extraction: Through multi-resolution knowledge distillation technology, the features of part images at different levels are extracted to capture information from local details to global structures;

[0133] Anomaly detection and location: Using gradient information and multi-resolution features, the abnormal area on the surface of the part can be accurately located to provide accurate defect location information for quality control;

[0134] Feature fusion and anomaly scoring: Integrate feature information of different resolutions, build an anomaly scoring system, and quantitatively evaluate the degree of anomaly of part images;

[0135] Quality control decision: Based on the results of the abnormal scoring system, the parts are judged as qualified or not, and the unqualified parts are marked and isolated;

[0136] Production process optimization: Use the test results to optimize the production process, adjust process parameters, reduce defects, and improve product quality;

[0137] Continuous model learning: New normal samples and abnormal samples are continuously introduced into model training to enable the model to continuously adapt to changes in the production process and maintain detection accuracy.

[0138] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Fig.11 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, an unsupervised anomaly detection method based on self-supervised data enhancement and multi-resolution knowledge distillation is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0139] Those skilled in the art will understand that Fig.11 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0140] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the steps of an unsupervised anomaly detection method based on self-supervised data enhancement and multi-resolution knowledge distillation are implemented.

[0141] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of an unsupervised anomaly detection method based on self-supervised data enhancement and multi-resolution knowledge distillation are implemented.

[0142] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0143] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0144] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. An unsupervised anomaly detection method based on self-supervised data enhancement and multi-resolution knowledge distillation, characterized in that: The method comprises: Acquire a normal detection image, cut and randomly paste the normal detection image, and generate a simulated abnormal image sample; The abnormal image samples are input into a deep convolutional neural network, and feature representations of the abnormal image samples at different levels are extracted by the deep convolutional neural network using a multi-resolution knowledge distillation technique, including: obtaining a pre-trained deep convolutional neural network, capturing local and texture information in the abnormal image samples through a first network layer in the deep convolutional neural network; capturing semantic information in the abnormal image samples through a second network layer in the deep convolutional neural network; constructing feature representations of multi-resolution information as multi-resolution features based on the local and texture information and semantic information; and transferring the multi-resolution features using a multi-resolution knowledge distillation technique to extract feature representations at different levels; Acquire corresponding image gradient information based on the abnormal image sample, and locate the abnormal area according to the image gradient information; Performing feature fusion on each of the feature representations at different levels to obtain fused features; and constructing an anomaly scoring system based on the fused features; The abnormal region is scored by the abnormal scoring system to calculate the degree of abnormality of the abnormal region.

2. The unsupervised anomaly detection method based on self-supervised data enhancement and multi-resolution knowledge distillation according to claim 1, characterized in that: Acquire a normal detection image, cut and randomly paste the normal detection image, and generate a simulated abnormal image sample, including: Acquire a normal detection image, randomly select a target area from the normal detection image, and cut the target area to obtain a cut area image; The cut region image is randomly pasted to any position in the normal detection image to generate a simulated abnormal image sample.

3. The unsupervised anomaly detection method based on self-supervised data enhancement and multi-resolution knowledge distillation according to claim 2 is characterized in that: Randomly pasting the cut region image to any position in the normal detection image includes: Transform the cut region image, and randomly paste the transformed region image to any position in the normal detection image; The transformation processing includes rotation, scaling, and color jitter.

4. The unsupervised anomaly detection method based on self-supervised data enhancement and multi-resolution knowledge distillation according to claim 1, characterized in that: The method further comprises: Marking the abnormal image samples, and inputting the marked abnormal image samples and the normal detection images into an image detection model; Outputting an image distinction detection result through the image detection model, and adjusting parameters in the image detection model according to the image distinction detection result, so as to complete the training optimization of the image detection model; The image detection model is used to distinguish abnormal image samples from normal detection images.

5. The unsupervised anomaly detection method based on self-supervised data enhancement and multi-resolution knowledge distillation according to claim 1, characterized in that: Acquiring corresponding image gradient information based on the abnormal image sample, and locating the abnormal area according to the image gradient information, including: Calculating image gradient information based on the abnormal image sample, and analyzing the size and distribution of the image gradient information to obtain an analysis result; The position information of the abnormal area is determined according to the analysis result, and an abnormal map is generated according to the position information and the image gradient information up-sampled to the spatial resolution to highlight the abnormal area.

6. The unsupervised anomaly detection method based on self-supervised data enhancement and multi-resolution knowledge distillation according to claim 1, characterized in that: The feature representations at different levels are fused to obtain fused features, including: Determine the feature weights corresponding to the feature representations at different levels respectively; The feature representations at different levels are weighted and summed according to the corresponding feature weights to obtain fused features.

7. The unsupervised anomaly detection method based on self-supervised data enhancement and multi-resolution knowledge distillation according to claim 1, characterized in that: The method further comprises: Determine anomaly detection decision thresholds; The fused features are input into the anomaly scoring system, and the anomaly scoring system converts the fused features into anomaly scores based on the anomaly detection decision threshold.

8. An unsupervised anomaly detection system based on self-supervised data enhancement and multi-resolution knowledge distillation, characterized in that: The system comprises: An image processing module is used to obtain a normal detection image, cut and randomly paste the normal detection image, and generate a simulated abnormal image sample; A multi-resolution feature extraction module is used to input the abnormal image samples into a deep convolutional neural network, and extract the feature representations of the abnormal image samples at different levels by using the multi-resolution knowledge distillation technology through the deep convolutional neural network, including: obtaining a pre-trained deep convolutional neural network, capturing the local and texture information in the abnormal image samples through the first network layer in the deep convolutional neural network; capturing the semantic information in the abnormal image samples through the second network layer in the deep convolutional neural network; constructing the feature representation of multi-resolution information according to the local and texture information and the semantic information as a multi-resolution feature; and transferring the multi-resolution features by using the multi-resolution knowledge distillation technology to extract the feature representations at different levels; An abnormality detection and positioning module, used to obtain corresponding image gradient information based on the abnormal image sample, and locate the abnormal area according to the image gradient information; A feature fusion module is used to fuse the feature representations at different levels to obtain fused features; and to construct an anomaly scoring system based on the fused features; The abnormality scoring module is used to score the abnormal area through the abnormality scoring system and calculate the abnormality degree of the abnormal area.

9. The unsupervised anomaly detection system based on self-supervised data enhancement and multi-resolution knowledge distillation according to claim 8, characterized in that: The image processing module is also used to: obtain a normal detection image, randomly select a target area from the normal detection image, and cut the target area to obtain a cut area image; randomly paste the cut area image to any position in the normal detection image to generate a simulated abnormal image sample.

Images