Abnormity detection system and method based on distillation training

Abstract

The invention provides an anomaly detection system and method based on distillation training, and belongs to the field of artificial intelligence and industrial quality inspection, the anomaly detection system comprises an anomaly detection large model and a real-time anomaly detection model, the anomaly detection large model is used as a teacher model, the real-time anomaly detection model is used as a student model, and through channel-by-channel knowledge distillation training, an anomaly detection result is obtained. And the anomaly detection knowledge in the anomaly detection large model is migrated to the real-time anomaly detection model, so that the detection efficiency is improved while the anomaly detection precision is ensured. According to the method, the zero sample detection capability of defect detection items is realized, the influence of the problems of long collection period of high-precision workpiece defect samples, insufficient robustness of long-tail sample detection and the like on an industrial quality inspection solution can be greatly relieved, and the method is convenient to popularize and apply in the field of industrial quality inspection.

Description

Technical Field

[0001] This invention belongs to the fields of artificial intelligence and industrial quality inspection, and specifically relates to an anomaly detection system and method based on distillation training. Background Technology

[0002] In the field of intelligent manufacturing, industrial quality inspection is an important part of the production process. The current mainstream solution for intelligent industrial quality inspection is the segmentation, classification and target detection based on supervised learning. However, the defect rate in actual production scenarios is already below 0.5%, defects are difficult to collect, and the amount of data is seriously insufficient. This leads to problems such as long project cycles, unsatisfactory detection accuracy and excessive costs.

[0003] However, for the inspection of high-precision workpieces, it is often not required to subdivide the defect types, but there are still problems such as long defect sample collection cycle and insufficient robustness of "long tail" samples (abnormal samples that occur infrequently). Summary of the Invention

[0004] In order to overcome the shortcomings of the prior art, the purpose of this invention is to provide an anomaly detection system and method based on distillation training, which can solve the above-mentioned problems.

[0005] Design Principle: Anomaly detection methods fully utilize the similarity of images of good products, transforming the open-set problem of target detection and recognition into a closed-set problem of standard product similarity clustering. Anomaly detection, with its advantages of not requiring defective samples and annotation information, and its ability to effectively detect unknown defects, has become an emerging technology in industrial quality inspection. Mainstream anomaly detection methods include FastFlow and EfficientAD. Therefore, the applicant designed a novel and efficient anomaly detection method and applied it to the field of industrial quality inspection to solve the aforementioned problems. The overall design scheme is as follows.

[0006] An anomaly detection system based on distillation training is disclosed. The system includes: a large anomaly detection model, comprising a text encoder, a weighted image encoder, an image decoder, a vector acquisition module, an aggregation module, and an inference and judgment module; and a real-time anomaly detection model, comprising a small model backbone network and multiple subsequent sampling modules. The large anomaly detection model serves as the teacher model, and the real-time anomaly detection model serves as the student model. Through channel-wise knowledge distillation training, the anomaly detection knowledge in the large anomaly detection model is transferred to the real-time anomaly detection model, thereby improving detection efficiency while ensuring anomaly detection accuracy.

[0007] Furthermore, in the large-scale anomaly detection model, the text encoder receives the text description of the defect, the weighted image encoder receives the anomaly map and the good product map, and the image decoder uses a lightweight decoder to output the anomaly map result.

[0008] Furthermore, the vector acquisition module obtains four types of vector features: the encoded features based on the anomaly image and the good image are added together and input into the linear layer to obtain feature vector V1; the prompt learning layer provides learnable feature vector V2; the image decoder outputs features through convolution, pooling and fully connected layers to obtain feature vector V3; and the image encoder obtains feature vector V4 through convolution, pooling and fully connected layers.

[0009] Furthermore, the aggregation module is used to connect and aggregate feature vectors V1, V2, V3, and V4.

[0010] Furthermore, the inference and judgment module performs inference and judgment on the aggregated feature vector data based on the LLaVA-Llama model to obtain the training or inference results of the large model.

[0011] Furthermore, the training loss function for the large anomaly detection model includes a pixel-level error loss function for anomaly detection and a prediction loss function for training the LLaVA large model.

[0012] Furthermore, the real-time anomaly detection model adopts a small model backbone network of YOLOv11, and the sampling module consists of three lightweight upsampling modules.

[0013] Furthermore, the lightweight upsampling module includes convolutional layers, ReLU activation layers, and MaxPooling pooling layers.

[0014] This application also provides an anomaly detection method based on distillation training. The anomaly detection method includes: S1, training a large anomaly detection model using a prompt-based learning fine-tuning strategy; S2, distillation training, using the large anomaly detection model as the teacher model and the real-time anomaly detection model as the student model, and training the model using a channel-wise knowledge distillation training method; S3, independently using the distilled real-time anomaly detection model for model inference; and S4, performing anomaly detection on the product to be detected based on the real-time anomaly detection model.

[0015] Furthermore, the distillation training employs a channel-wise knowledge distillation-based training method to align the second and third level features of the real-time anomaly detection model.

[0016] Compared with the prior art, the beneficial effects of the present invention are as follows: the present application realizes the zero-sample detection capability for defect detection items, and can greatly alleviate the impact of problems such as long collection cycle of high-precision workpiece defect samples and insufficient robustness of "long tail" sample detection on industrial quality inspection solutions, which is convenient for promotion and application in the field of industrial quality inspection. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the anomaly detection process of the present invention; Figure 2 This is a schematic diagram of the structure of a large-scale anomaly detection model; Figure 3 This is a schematic diagram of a real-time anomaly detection model; Figure 4 This is a schematic diagram of the channel-by-channel knowledge distillation training method. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] An anomaly detection system based on distillation training, the overall process is as follows: Figure 1 As shown, the anomaly detection system includes...

[0020] The large-scale anomaly detection model includes a text encoder, a weighted image encoder, an image decoder, a vector acquisition module, an aggregation module, and an inference and judgment module.

[0021] The real-time anomaly detection model consists of a small model backbone network and multiple subsequent sampling modules.

[0022] The large-scale anomaly detection model serves as the teacher model, while the real-time anomaly detection model serves as the student model. Through channel-wise knowledge distillation training, the anomaly detection knowledge in the large-scale anomaly detection model is transferred to the real-time anomaly detection model, thereby improving detection efficiency while ensuring anomaly detection accuracy.

[0023] The overall approach is as follows: First, a prompt-based learning fine-tuning strategy is used to train a large-scale anomaly detection model. Then, the large-scale anomaly detection model is used as the teacher model, and the real-time anomaly detection model is used as the student model; distillation training is then employed to train the real-time anomaly detection model. Finally, the real-time anomaly detection model is extracted separately for model inference. This approach ensures both good accuracy and speed in the anomaly detection model.

[0024] In the large-scale anomaly detection model, the text encoder receives the text description of the defect, the weighted image encoder receives the anomaly image and the good product image, and the image decoder uses a lightweight decoder to output the anomaly image result.

[0025] The vector acquisition module obtains four types of vector features: the encoded features based on the anomaly image and the good product image are added together and input into the linear layer to obtain the feature vector V1; the prompt learning layer provides the learnable feature vector V2; the image decoder outputs features through convolution, pooling and fully connected layers to obtain the feature vector V3; and the image encoder obtains the feature vector V4 through convolution, pooling and fully connected layers.

[0026] The aggregation module is used to connect and aggregate feature vectors V1, V2, V3, and V4.

[0027] The inference and judgment module performs inference and judgment on the aggregated feature vector data based on the LLaVA-Llama model to obtain the training or inference results of the large model.

[0028] The training loss function for the large anomaly detection model includes a pixel-level error loss function for anomaly detection and a prediction loss function for training the LLaVA large model.

[0029] The pixel-level error loss function consists of two parts: one part is the DICE loss function used for semantic segmentation; the other part is the mean squared error loss function for normal sample reconstruction.

[0030] Specifically, the structure of the large-scale anomaly detection model is as follows: Figure 2 As shown, this invention employs a text editor based on the LLaVA-Llama model and Clip ViT-L / 14. The image decoder uses a lightweight decoder based on UNet, and the output is an anomaly map. The encoded features of the input anomaly map and the good sample map are added together and input into a linear layer to obtain feature vector V1. The hint learning layer provides a learnable feature vector V2. The output features of the image decoder are processed through convolution, pooling, and fully connected layers to obtain feature vector V3. The image encoder is processed through convolution, pooling, and fully connected layers to obtain feature vector V4. V1, V2, V3, and V4 are concatenated and input into the LLaVA-Llama model to obtain the large model training or inference result. The training loss functions are a pixel-level error loss function for anomaly detection and a prediction loss function for training the LLaVA large model. The pixel-level error loss function consists of two parts: one part is the DICE loss function for semantic segmentation; the other part is the mean squared error loss function for normal sample reconstruction.

[0031] Considering the timeliness of anomaly detection, this invention designs a real-time anomaly detection model, the structure of which is as follows: Figure 3 As shown, the backbone network adopts a small model backbone network of YOLOv11, followed by three lightweight upsampling modules. The lightweight upsampling modules consist of convolutional layers, ReLU activation layers, and MaxPooling pooling layers.

[0032] To efficiently train the real-time anomaly detection model, we adopted a training method based on channel-wise knowledge distillation, such as... Figure 4 As shown. The distillation method in this invention aligns only the second and third level features of the model, and this alignment method performs best in downstream task performance verification.

[0033] This application also provides an anomaly detection method based on distillation training, the anomaly detection method including the following steps.

[0034] S1. A prompt-based learning fine-tuning strategy is used to train a large-scale anomaly detection model.

[0035] S2. Distillation training: The large anomaly detection model is used as the teacher model, and the real-time anomaly detection model is used as the student model. The model is trained using a channel-wise knowledge distillation training method.

[0036] S3. Use the real-time anomaly detection model trained by distillation independently for model inference; S4. Perform anomaly detection on the product to be tested based on the real-time anomaly detection model.

[0037] Among them, the distillation training adopts a training method based on channel-wise knowledge distillation to align the second and third level features of the real-time anomaly detection model.

[0038] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An anomaly detection system based on distillation training, characterized in that, The anomaly detection system includes: The large-scale anomaly detection model includes a text encoder, a weighted image encoder, an image decoder, a vector acquisition module, an aggregation module, and an inference and judgment module. The real-time anomaly detection model consists of a small model backbone network and multiple subsequent sampling modules; The large-scale anomaly detection model serves as the teacher model, while the real-time anomaly detection model serves as the student model. Through channel-wise knowledge distillation training, the anomaly detection knowledge in the large-scale anomaly detection model is transferred to the real-time anomaly detection model.

2. The anomaly detection system according to claim 1, characterized in that: In the large-scale anomaly detection model, the text encoder receives the text description of the defect, the weighted image encoder receives the anomaly image and the good product image, and the image decoder uses a lightweight decoder to output the anomaly image result.

3. The anomaly detection system according to claim 1, characterized in that, The vector acquisition module obtains four types of vector features: the encoded features based on the anomaly image and the good product image are added together and input into the linear layer to obtain the feature vector V1; the prompt learning layer provides the learnable feature vector V2; the image decoder outputs features through convolution, pooling and fully connected layers to obtain the feature vector V3; and the image encoder obtains the feature vector V4 through convolution, pooling and fully connected layers.

4. The anomaly detection system according to claim 3, characterized in that: The aggregation module is used to connect and aggregate feature vectors V1, V2, V3, and V4.

5. The anomaly detection system according to claim 4, characterized in that: The inference and judgment module performs inference and judgment on the aggregated feature vector data based on the LLaVA-Llama model to obtain the training or inference results of the large model.

6. The anomaly detection system according to claim 4, characterized in that: The training loss function for the large anomaly detection model includes a pixel-level error loss function for anomaly detection and a prediction loss function for training the LLaVA large model.

7. The anomaly detection system according to claim 1, characterized in that: The real-time anomaly detection model uses a small model backbone network of YOLOv11, and the sampling module consists of three lightweight upsampling modules.

8. The anomaly detection system according to claim 7, characterized in that: The lightweight upsampling module includes convolutional layers, ReLU activation layers, and MaxPooling pooling layers.

9. An anomaly detection method based on distillation training, characterized in that, Anomaly detection methods include: S1. Employ a prompt-based learning fine-tuning strategy for training a large-scale anomaly detection model; S2. Distillation training: The large anomaly detection model is used as the teacher model, and the real-time anomaly detection model is used as the student model. The model is trained using a channel-wise knowledge distillation training method. S3. Use the real-time anomaly detection model trained by distillation independently for model inference; S4. Perform anomaly detection on the product to be tested based on the real-time anomaly detection model.

10. The anomaly detection method according to claim 9, characterized in that: The distillation training employs a channel-wise knowledge distillation-based training method to align the second and third level features of the real-time anomaly detection model.

Images