Unsupervised chip anomaly detection method based on distillation learning
By adopting an unsupervised detection method based on distillation learning in chip abnormality detection, the problem of high error detection rate during reconstruction in the prior art is solved, and a higher detection accuracy and degree of automation is achieved.
Patent Information
- Application Number
- CN202411413254.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-11
- Publication Date
- 2025-02-07
AI Technical Summary
The existing defect detection methods based on image reconstruction are difficult to ensure the complete and accurate reconstruction of normal areas during the reconstruction process, which is prone to false detection.
Adopt unsupervised chip abnormality detection method based on distillation learning, and reconstruct the chip image after the abnormality is processed by training the teacher network, freeze the weight of the teacher network, and use the good product image to train the student network, so that the student network learning can imitate the teacher Network reconstruction of good product images.
Reduce misdetection caused by reconstruction deviation, improve detection accuracy, reduce dependence on manual labeled data, reduce costs and improve the degree of automation of the detection process.
Smart Images
Figure CN119399124A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of defect detection, and in particular to an unsupervised chip anomaly detection method based on distillation learning. Background Art
[0002] Image recognition and inspection technology is playing an increasingly important role in the broad fields of quality control and defect identification. Traditionally, these tasks rely on complex parameter settings to achieve more effective inspection results. However, with the rapid development of artificial intelligence, especially deep learning technology, data-driven inspection methods are gradually emerging, bringing revolutionary changes to the field of defect detection.
[0003] Although deep learning technology has demonstrated strong learning capabilities and the ability to accurately capture complex image features, its application is not an easy journey. A significant challenge is that building efficient deep learning models usually requires massive training data sets as support, which must be composed of a large number of carefully annotated images. This process not only requires the collection of sufficiently extensive and diverse image samples, but also involves tedious and time-consuming manual annotation work, that is, professionals review the images one by one and accurately mark the defective areas. This high dependence on human resources and the resulting high costs have become the main bottleneck limiting the widespread popularization and application of deep learning technology in the field of defect detection. Therefore, current research and practice are actively exploring various strategies to alleviate or overcome this problem. For example, developing more efficient image annotation tools, using semi-supervised or weakly supervised learning techniques to reduce reliance on manual annotation, and expanding the diversity and number of training samples through data augmentation techniques. These efforts aim to lower the application threshold of deep learning technology in the field of defect detection and promote its development in a more automated and intelligent direction. Another option is to achieve this through anomaly detection methods that only train good products.
[0004] In the field of image anomaly detection, an important branch is the method based on image reconstruction, that is, training a network to reconstruct both good and defective images into good images. When defective products are input, the defective images can be found by comparing the reconstructed images with the original images. Conventional defect detection methods based on image reconstruction are as follows: Figure 1 However, one drawback of this method is that, during the reconstruction process, it is difficult to ensure that the normal area is completely accurately reconstructed as normal. Once the normal area deviates during the reconstruction process, it will cause new defects and lead to false detection.
[0005] It should be noted that the information disclosed in the above background technology section is only used for understanding the background of the present application, and therefore may include information that does not constitute prior art known to ordinary technicians in the field. Summary of the invention
[0006] The main purpose of the present invention is to solve the problems existing in the above-mentioned background technology and provide an unsupervised chip anomaly detection method based on distillation learning.
[0007] To achieve the above object, the present invention adopts the following technical solutions:
[0008] An unsupervised chip anomaly detection method based on distillation learning includes the following steps:
[0009] S1. Teacher network training: Use a data set containing good chip images and good chip images processed by abnormalization to train the teacher network, so that the teacher network can reconstruct good chip images and corresponding abnormal images;
[0010] S2. Distillation learning: Freeze the weights of the trained teacher network, and train the student network using good chip images through distillation learning, so that the student network can learn and imitate the teacher network's ability to reconstruct good chip images, and learn to reconstruct any input image into a good image;
[0011] S3. Anomaly detection: Input the image to be detected into the trained teacher network and student network to obtain their respective reconstructed images; analyze and compare the reconstructed images of the two through the trained detection network, and output an anomaly score map; determine whether the image to be detected contains anomalies based on the anomaly score map, and determine the location and degree of the abnormal area.
[0012] Furthermore, both the teacher network and the student network adopt a U-Net network structure, which includes three downsampling links and three upsampling links, and the intermediate layers are connected by convolutions; during the training process, the L2 distance is used to calculate the loss, that is, the average of the squared difference between the input image and the reconstructed image in each batch is calculated.
[0013] Furthermore, the detection network adopts the ResNet18 residual convolutional network structure, including four basic residual blocks, each residual block contains two convolutional layers and a batch normalization layer, and each residual block fuses the output with the input feature vector through an identity connection, and finally outputs a normalized anomaly score map.
[0014] Furthermore, before the teacher network is trained in step S1, the teacher network is pre-trained to obtain a general ability to reconstruct various images.
[0015] Furthermore, the abnormalization process includes adding random patterns to the image of the good chip to destroy the texture structure of the normal image of the chip.
[0016] Furthermore, in step S1, after the image is alienated, a defect image and a corresponding defect mask image are obtained. During the training process, the defect mask image is used to guide the teacher network to correctly reconstruct the defect area.
[0017] Furthermore, the detection network compares the high-dimensional semantic features of the two reconstructed images to generate an anomaly score map; wherein, the L2 loss is calculated using the normalized anomaly score map and the corresponding defect mask image to measure the difference between the two to guide the training of the detection network.
[0018] Furthermore, in step S3, the anomaly score map adopts a heat map, and the generation function of the heat map takes the output of the teacher network and the student network as input. The value of each pixel point on the output heat map reflects the degree of deviation of the point from the normal image feature, thereby indicating the possibility of anomaly.
[0019] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the unsupervised chip anomaly detection method based on distillation learning.
[0020] A computer program product includes a computer program, which, when executed by a processor, implements the unsupervised chip anomaly detection method based on distillation learning.
[0021] The present invention has the following beneficial effects:
[0022] The present invention proposes an unsupervised distillation learning method for chip anomaly detection, which reconstructs good and abnormalized chip images by training a teacher network, then freezes its weights, and uses these good images to train a student network. The student network learns to imitate the teacher network, focusing on reconstructing good images rather than processing defective images, thereby providing a stable good image reference benchmark for the detection network. In the anomaly detection stage, the detection network compares the reconstructed images of the teacher network and the student network, outputs an anomaly score map, and then determines whether the image has an anomaly and its location and degree. The present invention avoids the conventional practice of directly comparing with the original image, can greatly reduce false detections caused by reconstruction deviations, and improves the accuracy of detection.
[0023] The present invention effectively combines unsupervised learning and distillation learning techniques, reducing the reliance on manually labeled data, thereby reducing costs and improving the degree of automation of the detection process. Through the collaborative work of the teacher network and the student network, the present invention improves the accuracy of chip anomaly detection and reduces the false detection rate. The student network learns to imitate the teacher network's ability to reconstruct good product images, providing a stable reference benchmark for the detection network, allowing the detection network to more accurately identify true anomalies. In addition, the present invention has experimentally verified its effectiveness in reducing false detections and improving detection accuracy, demonstrating its potential and value in practical applications. The method of the present invention promotes the development of defect detection technology in a more intelligent and precise direction, and helps to improve the efficiency and effectiveness of product quality control.
[0024] Other beneficial effects of the embodiments of the present invention will be further described below. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] Figure 1 A conventional defect detection method based on image reconstruction
[0026] Figure 2 A schematic diagram of teacher network pre-training according to an embodiment of the present invention.
[0027] Figure 3 Schematic diagram of training a teacher network using chip images according to an embodiment of the present invention.
[0028] Figure 4 Schematic diagram of training a student network for distillation learning according to an embodiment of the present invention.
[0029] Figure 5 Schematic diagram of a training detection network according to an embodiment of the present invention.
[0030] Figure 6 This is a flow chart of an embodiment of the present invention using a trained network to perform chip anomaly detection.
[0031] Figure 7 This is a graph of detection results using a new distillation learning method according to an embodiment of the present invention.
[0032] Figure 8 This is the detection result diagram of the conventional image reconstruction method. DETAILED DESCRIPTION
[0033] The following is a detailed description of the embodiments of the present invention. It should be emphasized that the following description is only exemplary and is not intended to limit the scope and application of the present invention.
[0034] See also Figures 2 to 6 The embodiment of the present invention provides an unsupervised chip anomaly detection method based on distillation learning, comprising the following steps:
[0035] S1. Teacher network training: Use a data set containing good chip images and good chip images processed by abnormalization to train the teacher network, so that the teacher network can reconstruct good chip images and corresponding abnormal images;
[0036] S2. Distillation learning: Freeze the weights of the trained teacher network, and train the student network using good chip images through distillation learning, so that the student network can learn and imitate the teacher network's ability to reconstruct good chip images, and learn to reconstruct any input image (whether good or defective) into a good image;
[0037] S3. Anomaly detection: Input the image to be detected into the trained teacher network and student network to obtain their respective reconstructed images; analyze and compare the reconstructed images of the two through the trained detection network, and output an anomaly score map; determine whether the image to be detected contains anomalies based on the anomaly score map, and determine the location and degree of the abnormal area.
[0038] The unsupervised chip anomaly detection method of the present invention uses a distillation learning method, and does not directly compare the reconstructed image with the original image, but compares the images before and after distillation to detect defects in the image. In the present invention, the student network focuses on learning the ability of the teacher network to process and reconstruct good images through the distillation learning process, rather than learning how to process defective images. This learning strategy ensures that the student network can provide a stable reference point for the detection network that tends to output good images, thereby helping the detection network to more accurately identify true anomalies.
[0039] The following further describes an algorithm example and experimental verification of a specific embodiment of the present invention.
[0040] The present invention comprises three networks, namely a teacher network, a student network and a detection network. The teacher network and the student network both adopt U-Net network, and both have the same network structure, but the learned model weight parameters are different. The input and output of the teacher network and the student network are both images, and the task of both is to reconstruct the input image. The detection network adopts ResNet18 residual convolution network.
[0041] The present invention uses a standard U-Net network, which includes three downsampling and three upsampling links, and the middle layer uses convolutional connections. The teacher model and the student model are respectively t (Teacher Model) and M s (Student Model) indicates that the teacher model and the apprentice model use L2 distance when calculating the loss, and the formula is:
[0042]
[0043] n is the batch_size of each training, x i Represents the input image of each batch, y i Represents the model reconstructed image for each batch.
[0044] Detection Model M seg The input of F is the reconstructed graph output by the apprentice network and the teacher network, and the output is the normalized anomaly score. The backbone is the ResNet18 residual convolutional network, which contains four basic residual blocks, each of which consists of two convolutional layers and Batch Normalization. i (x,W) is the backbone path output of the residual block in each stage. The backbone network can be expressed as:
[0045] H i (x) = F i (x,W)+xfor i=1,2,3,4
[0046] Where x is the input residual block tensor, i is the stage number, H i (x) is the output feature vector.
[0047] The training process of the present invention is divided into the following four steps.
[0048] In the first step, the teacher network is trained with the Imagenet dataset so that it can reconstruct images that are exactly the same as the input images for various input images. This process is called pre-training. t1 Input the image I of the Imagenet dataset, and output the reconstructed image R of the teacher model. t1 , the expression formula is as follows:
[0049] R t1 =M t1 (I)
[0050] When training the model, Loss calculates the L2 distance between the teacher model's reconstructed image and the original image. The calculation formula is:
[0051] L t1 =L2(R t1 ,I)
[0052] Figure 2 The step one teacher network pre-training process is shown.
[0053] The second step is to train the teacher network with a dataset of chip images so that the teacher network can reconstruct the input chip images. There are two types of image data used in this training process: good product images and abnormal good product images. The abnormal method is to add random patterns to the good product images to destroy the texture structure of the normal chip images. This training process enables the teacher network to reconstruct good product images into good product images, and reconstruct the same defective images for abnormal defective images.
[0054] After the image is alienated, the defect image and the corresponding defect mask image I are obtained. mask When training the teacher model this time, first load the training parameters of the pre-trained model to speed up the model convergence speed and training accuracy. The training process is the same as the first step, the difference is that the training data set is changed to the good product images of the actual product to be tested. ok and the alienated image I ano , the teacher model training data set can be expressed as:
[0055] I all =I ok +I ano
[0056] The output of the model training process is R t2 , L t2 The L2 distance between the teacher model reconstruction image and the original image is expressed as follows:
[0057] R t2 =M t2 (I all ), L t2 =L2(R t2 ,I all )
[0058] Figure 3 The figure shows the process of training the teacher network with chip images in step 2.
[0059] The third step is to train the student network and let the student network learn from the teacher network so that it can reconstruct images that are completely consistent with the teacher network. The image data used in this training process is good-quality images. This learning is a distillation learning process. The weights of the teacher network are frozen, and the student network is allowed to learn from the teacher network and only learn its reconstruction of good-quality images, but not the ability to reconstruct defective patterns. Therefore, even if it is a defective image, the student network can only reconstruct it into a good one. The training loss function is the L2 distance between the teacher network and the student network reconstructed image, L s In this case, the teacher network is no longer trained but only inferred, R s is the feature vector output by the student model, which can be expressed as:
[0060] Rs =M s (I ok ), L s =L2(R s , M t2 (I ok ))
[0061] Figure 4 The process of training the student network through distillation learning in step three is shown.
[0062] The fourth step is to find abnormal areas through image comparison, that is, detect defects. This process is implemented by the detection network, and the input of the detection network is the output of the teacher network and the student network. The training process is as follows Figure 5 As shown in the figure, the chip quality alienation graph is reconstructed by the teacher network and the student network respectively, and then these two graphs (R s ,R t2 ) as the input of the detection network, which compares the two images in a high-dimensional semantic space and outputs the judgment result.
[0063] Detection network output anomaly score map D seg It can be expressed as
[0064] D seg =M seg (R s ,R t2 )
[0065] When calculating Loss, since the anomaly score map has been normalized, the anomaly score map and I can be directly calculated mask The L2 distance, L seg The calculation formula is as follows:
[0066] L seg =L2(D seg ,I mask ).
[0067] After completing the network training in the above four steps, it can be used for actual defect detection. Specifically, input a picture to be tested X, and output a heat map D representing the degree of abnormality. seg , as shown below:
[0068] D seg =M seg (M s (X),M t2 (X)
[0069] When the image to be tested is a good product, both the teacher network and the student network output the same good product image, and the detection network will not detect defects; when the image to be tested is a defective image, the output of the teacher network is a defective image, while the output of the student network is a repaired good product, and the detection network can detect defects based on these two images. Figure 6 The process of chip anomaly detection using the trained teacher network, student network, and detection network is shown.
[0070] Comparative experiment
[0071] Figure 7 This is a detection result diagram of the unsupervised chip anomaly detection method based on the distillation learning method in an embodiment of the present invention. Figure 8 The figure is the detection result of the conventional image reconstruction method. According to the comparison of the experimental results, it can be seen that according to the conventional image reconstruction method, in addition to the wear mark defect in the upper left corner, there will be more false detection areas in the lower right corner and other areas, while the method of the present invention does not have such false detection areas, which can effectively reduce false detection and improve the accuracy of chip abnormality detection.
[0072] In summary, the unsupervised chip anomaly detection method based on distillation learning provided by the present invention effectively combines unsupervised learning and distillation learning techniques, reduces the reliance on manually labeled data, thereby reducing costs and improving the degree of automation of the detection process. Through the collaborative work of the teacher network and the student network, the present invention improves the accuracy of chip anomaly detection and greatly reduces the false detection rate. The student network learns to imitate the teacher network's ability to reconstruct good product images, providing a stable reference benchmark for the detection network, so that the detection network can more accurately identify real anomalies. The present invention has verified its effectiveness in reducing false detections and improving detection accuracy through experiments, demonstrating its potential and value in practical applications. By this method, the present invention promotes the development of defect detection technology in a more intelligent and precise direction, which helps to improve the efficiency and effectiveness of product quality control.
[0073] An embodiment of the present invention further provides a storage medium for storing a computer program, which at least performs the above method when executed.
[0074] An embodiment of the present invention further provides a control device, comprising a processor and a storage medium for storing a computer program; wherein the processor is configured to execute at least the method described above when executing the computer program.
[0075] An embodiment of the present invention further provides a processor, wherein the processor executes a computer program and at least executes the method described above.
[0076] The storage medium can be implemented by any type of non-volatile storage device, or a combination thereof. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a ferromagnetic random access memory (Flash Memory), a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a disk memory or a tape memory. The storage medium described in the embodiments of the present invention is intended to include, but is not limited to, these and any other suitable types of memory.
[0077] In the several embodiments provided by the present invention, it should be understood that the disclosed systems and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.
[0078] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.
[0079] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.
[0080] A person skilled in the art can understand that: all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), disks or optical disks, etc. Various media that can store program codes.
[0081] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.
[0082] The methods disclosed in the several method embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new method embodiments.
[0083] The features disclosed in several product embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new product embodiments.
[0084] The features disclosed in several method or device embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.
[0085] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art of the present invention, several equivalent substitutions or obvious variations can be made without departing from the concept of the present invention, and the performance or use is the same, which should be regarded as belonging to the protection scope of the present invention.
Claims
1. An unsupervised chip anomaly detection method based on distillation learning, characterized in that: The following steps are involved: S1. Teacher network training: Use a data set containing good chip images and good chip images processed by abnormalization to train the teacher network, so that the teacher network can reconstruct good chip images and corresponding abnormal images; S2. Distillation learning: Freeze the weights of the trained teacher network, and train the student network using good chip images through distillation learning, so that the student network can learn and imitate the teacher network's ability to reconstruct good chip images, and learn to reconstruct any input image into a good image; S3. Anomaly detection: Input the image to be detected into the trained teacher network and student network to obtain their respective reconstructed images; analyze and compare the reconstructed images of the two through the trained detection network, and output an anomaly score map; determine whether the image to be detected contains anomalies based on the anomaly score map, and determine the location and degree of the abnormal area.
2. The unsupervised chip anomaly detection method based on distillation learning according to claim 1, characterized in that: Both the teacher network and the student network adopt a U-Net network structure, which includes three downsampling links and three upsampling links, and the intermediate layers are connected by convolutions; during the training process, the L2 distance is used to calculate the loss, that is, the average of the squared differences between the input image and the reconstructed image in each batch is calculated.
3. The unsupervised chip anomaly detection method based on distillation learning according to claim 1, characterized in that: The detection network adopts the ResNet18 residual convolutional network structure, including four basic residual blocks, each of which contains two convolutional layers and a batch normalization layer. Each residual block fuses the output with the input feature vector through an identity connection, and finally outputs a normalized anomaly score map.
4. The unsupervised chip anomaly detection method based on distillation learning according to any one of claims 1 to 3, characterized in that: Before the teacher network is trained in step S1, the teacher network is pre-trained to obtain a general ability to reconstruct various images.
5. The unsupervised chip anomaly detection method based on distillation learning according to any one of claims 1 to 4, characterized in that: The abnormalization process includes adding random patterns to the image of the good chip to destroy the texture structure of the normal image of the chip.
6. The unsupervised chip anomaly detection method based on distillation learning according to any one of claims 1 to 5, characterized in that: In step S1, after the image is alienated, a defect image and a corresponding defect mask image are obtained. During the training process, the defect mask image is used to guide the teacher network to correctly reconstruct the defect area.
7. The unsupervised chip anomaly detection method based on distillation learning according to claim 6, characterized in that: The detection network compares the high-dimensional semantic features of the two reconstructed images to generate an anomaly score map; wherein the L2 loss is calculated using the normalized anomaly score map and the corresponding defect mask image to measure the difference between the two to guide the training of the detection network.
8. The unsupervised chip anomaly detection method based on distillation learning according to any one of claims 1 to 7, characterized in that: In step S3, the anomaly score map adopts a heat map, and the generation function of the heat map takes the output of the teacher network and the student network as input. The value of each pixel point on the output heat map reflects the degree of deviation of the point from the normal image feature, thereby indicating the possibility of anomaly.
9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, it implements the unsupervised chip anomaly detection method based on distillation learning as described in any one of claims 1 to 8.
10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, it implements the unsupervised chip anomaly detection method based on distillation learning as described in any one of claims 1 to 8.