Chip defect detection method based on difference grey-scale map anomaly detection
By using deep learning anomaly detection methods based on differential grayscale maps in chip defect detection, the shortcomings of traditional detection methods in micro defect detection and non-defect error interference are solved, and higher detection accuracy and performance are achieved.
Patent Information
- Application Number
- CN202411491270.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-24
- Publication Date
- 2025-02-11
AI Technical Summary
Traditional chip defect detection methods are insufficient in detecting small defects and are susceptible to non-defect error factors such as color difference, solder feet, and random texture during chip manufacturing, resulting in a high rate of false detection and missed detection.
Deep learning abnormality detection method based on differential grayscale map is adopted. By building a chip abnormality detection model, the chip's grayscale differential map is used for training and detection, thereby eliminating the interference of complex feature of the chip itself and improving detection accuracy.
It significantly reduces the false detection and missed detection rates, improves the accuracy and performance of chip defect detection, and especially performs excellently in detecting small defects.
Smart Images
Figure CN119417790A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of industrial anomaly detection, and in particular to a chip defect detection method based on differential grayscale image anomaly detection. Background Art
[0002] Chip defect detection is one of the important categories of industrial anomaly detection. Usually, during the chip manufacturing or packaging process, various defects such as dust, dirt, broken edges, scratches, and component defects will appear. These defects may affect the performance and reliability of the chip, or cause the chip to fail. Therefore, defect detection is required during the chip production process to screen out unqualified products.
[0003] Traditional chip defect detection methods rely on machine vision. The more common detection method is the image difference method. A qualified chip image is used to generate a template. When performing the test, the chip image to be tested is first matched with the template. After the match is successful, the image to be tested is affine transformed to make it the same as the template in terms of scaling and rotation. The image to be tested and the template are then differentiated, and the part with abnormal grayscale value in the differentiated image is detected as a defect. This method can detect more obvious defects, but it has poor detection capabilities for minor defects. It is also easily affected by non-defect error factors such as color difference and minor deformation that occur during chip manufacturing, resulting in high false detection and missed detection rates.
[0004] In recent years, the rapid development of deep learning technology has brought new ideas to chip defect detection. However, model training methods such as classification and positioning based on neural networks have difficulties in obtaining various chip defect samples, resulting in sample imbalance and prone to overfitting, which leads to weak generalization ability of deep learning models and insufficient recognition accuracy. In response to such problems, some researchers have proposed anomaly detection methods that only need to use positive samples corresponding to qualified products to train the model, thereby solving the problem of difficulty in obtaining defect samples. However, in actual applications, anomaly detection models are prone to confuse defects in chips with the complex characteristics of the chips themselves, resulting in reduced defect detection capabilities. Summary of the invention
[0005] The technical problem to be solved by the present invention is to address the deficiencies of the above-mentioned prior art and to provide a chip defect detection method based on differential grayscale image anomaly detection. The chip defect detection method based on differential grayscale image anomaly detection uses chip differential grayscale images for deep learning anomaly detection, thereby solving the problem of false detection of traditional grayscale differential detection methods due to non-defect errors in the chip itself, such as color difference, solder feet, and random texture.
[0006] In order to solve the above technical problems, the technical solution adopted by the present invention is:
[0007] A chip defect detection method based on differential grayscale image anomaly detection includes the following steps.
[0008] Step 1. Build a chip anomaly detection model: The chip anomaly detection model is a deep learning model based on a neural network. The input of the chip anomaly detection model is the grayscale difference map of the chip, and the output of the chip anomaly detection model is the chip thermal map and the chip defect location.
[0009] Step 2: Obtain a grayscale difference base map: The grayscale difference base map is a grayscale image of a positive chip sample image of the same type as the chip to be detected.
[0010] Step 3, obtain a differential image positive sample training set: perform differential analysis on the grayscale images of N positive sample images of the same model chip and the grayscale differential base image obtained in step 2 to obtain a differential image positive sample training set; wherein N is not less than 100; the N chips of the same model include various types of non-defective error chips; wherein each type of non-defective error chips is not less than 10.
[0011] Step 4: Model training: Use the differential image positive sample training set obtained in step 3 to iteratively train the chip anomaly detection model constructed in step 1 to obtain a trained chip anomaly detection model.
[0012] Step 5, obtaining a grayscale differential image to be detected: Differentiate the grayscale image of the chip image to be detected from the grayscale differential base image obtained in step 2 to obtain a grayscale differential image to be detected.
[0013] Step 6, chip anomaly detection: The grayscale difference image to be detected obtained in step 5 is input into the chip anomaly detection model trained in step 4 to obtain the thermal map and defect location of the chip to be detected.
[0014] In step 3, non-defect error chips refer to chips whose chip images have characteristic changes but will not cause the chips to be unqualified; wherein, non-defect error chips include color difference change chips, solder foot change chips and random texture change chips.
[0015] In step 1, the chip anomaly detection model is one or a combination of the FastFlow model, the Padim model, the Patchcore model and the ReverseDistillation model.
[0016] In step 1, the loss function of the chip anomaly detection model includes a cross entropy function obtained by combining the softmax regression classification function and the negative log-likelihood function (NLLloss);
[0017] In step 4, the convergence condition of the loss function during chip anomaly detection model training is that the output loss function value must be reduced to 0.05.
[0018] In step 2, the grayscale difference base map is the average value of the grayscale images of multiple positive chip sample images of the same type as the chip to be tested; wherein the positive chip samples are qualified chip samples without dust, broken edges, scratches or other defects that can be visually identified and affect the normal function of the chip; each positive chip sample image grayscale image is obtained by grayscale processing the collected positive chip sample image.
[0019] Assume that the RGB values corresponding to the pixels in the mth row and nth column in the forward chip sample image are R m×n , G m×n and B m×n , then the gray value of the pixel in the mth row and nth column after conversion to grayscale image is Gray m×n The calculation formula is:
[0020] Gray m×n =0.299×R m×n +0.587×G m×n +0.114×B m×n
[0021] Suppose the gray value of the pixel in the mth row and nth column of the grayscale difference base map is The calculation formula is:
[0022]
[0023] In the formula, It is the gray value of the pixel in the mth row and nth column in the i-th chip grayscale image sample.
[0024] In step 3, the grayscale value of the pixel in the mth row and nth column in the jth grayscale difference image sample in the difference image positive sample training set is The calculation formula is:
[0025]
[0026] In the formula, is the grayscale value of the pixel in the mth row and nth column in the jth chip grayscale image sample.
[0027] In step 5, the gray value of the pixel in the mth row and nth column of the grayscale image of the chip to be detected is Gray m×n , then the gray value of the pixel in the mth row and nth column in the gray difference image to be detected is The calculation formula is:
[0028]
[0029] In step 1, the chip defect location refers to the area in the chip thermal map where the thermal value is greater than the set thermal threshold.
[0030] The present invention has the following beneficial effects: the present invention uses the chip's differential grayscale image instead of the chip image for deep learning anomaly detection, solving the problem of misdetection caused by the traditional grayscale differential detection method due to non-defect errors in the chip itself, such as color difference, solder feet, random texture, etc. At the same time, it eliminates the interference of the complex features of the chip itself on the anomaly detection model, and improves the detection ability of the anomaly detection model for chip defects. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] Figure 1 The flowchart of the chip defect detection method based on differential grayscale image anomaly detection of the present invention is shown.
[0032] Figure 2 The color difference change chip images collected in the present invention are shown; wherein (a) and (b) are two types of color difference change chip images respectively.
[0033] Figure 3 (a) and (b) are Figure 2 Grayscale difference images generated by (a) and (b).
[0034] Figure 4 The image of the chip to be detected in the embodiment of the present invention is shown.
[0035] Figure 5 The grayscale difference image to be detected in the embodiment of the present invention is shown.
[0036] Figure 6 The defect heat map of the chip differential image output by the anomaly detection model in the embodiment of the present invention is shown, wherein the red area is the abnormal area with high thermal value output by the detection.
[0037] Figure 7 The defect thermogram of the chip original image used as a control in the embodiment of the present invention is shown, wherein the red area is the abnormal area with high thermodynamic value output by the detection. DETAILED DESCRIPTION
[0038] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific preferred embodiments.
[0039] In the description of the present invention, it should be understood that the terms "left side", "right side", "upper part", "lower part" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. "First", "second" and the like do not indicate the importance of the components, and therefore cannot be understood as limiting the present invention. The specific dimensions used in this embodiment are only for illustrating the technical solution, and do not limit the scope of protection of the present invention.
[0040] like Figure 1 As shown, a chip defect detection method based on differential grayscale image anomaly detection includes the following steps.
[0041] Step 1: Build a chip anomaly detection model: The chip anomaly detection model is a deep learning model based on a neural network. The input of the chip anomaly detection model is the grayscale difference map of the chip, and the output of the chip anomaly detection model is the chip thermal map and the chip defect location. The chip defect location refers to the area in the chip thermal map where the thermal value is greater than the set thermal threshold.
[0042] The above chip anomaly detection model is preferably one or a combination of the FastFlow model, the Padim model, the Patchcore model and the ReverseDistillation model. Further, the loss function of the chip anomaly detection model includes a cross entropy function obtained by combining the softmax regression classification function and the negative log-likelihood function (NLLloss);
[0043] Step 2, obtaining a grayscale difference base map: The grayscale difference base map is preferably an average of grayscale images of multiple positive chip sample images of the same type as the chip to be detected. However, as an alternative, it can also be a selected base map.
[0044] The above-mentioned positive chip samples are qualified chip samples without dust, broken edges, scratches or other visually identifiable defects that affect the normal function of the chip; each positive chip sample image grayscale image is obtained by grayscale processing the collected positive chip sample image.
[0045] Assume that the RGB values corresponding to the pixels in the mth row and nth column in the forward chip sample image are R m×n , G m×n and B m×n , then the gray value of the pixel in the mth row and nth column after conversion to grayscale image is Gray m×n The calculation formula is:
[0046] Gray m×n =0.299×R m×n +0.587×G m×n +0.114×B m×n
[0047] Suppose the gray value of the pixel in the mth row and nth column of the grayscale difference base map is The calculation formula is:
[0048]
[0049] In the formula, It is the gray value of the pixel in the mth row and nth column in the i-th chip grayscale image sample.
[0050] Step 3, obtain a differential image positive sample training set: perform differential analysis on the grayscale images of N positive sample images of the same model chip and the grayscale differential base image obtained in step 2 to obtain a differential image positive sample training set; wherein N is not less than 100; the N chips of the same model include various types of non-defective error chips; wherein each type of non-defective error chips is not less than 10.
[0051] The above-mentioned non-defect error chips refer to chips whose chip images have characteristic changes but will not cause the chip to be unqualified; among them, non-defect error chips include color difference change chips, solder foot change chips and random texture change chips, etc.
[0052] The grayscale value of the pixel in the mth row and nth column in the jth grayscale difference image sample in the above difference image positive sample training set is The calculation formula is:
[0053]
[0054] In the formula, is the grayscale value of the pixel in the mth row and nth column in the jth chip grayscale image sample.
[0055] Figure 2 (a) and (b) show partial images of two chips of the same model produced in different batches. From the comparison of the two images, it can be seen that even qualified chips of the same model will inevitably have color differences. Figure 3 (a) and (b) show the use of Figure 2 (a) and (b) are grayscale difference images formed according to the above processing rules. As can be seen from the figure, non-defect errors such as color difference of the chip will cause a large gap in the grayscale difference image of two qualified chips. When processing grayscale difference images, traditional defect detection algorithms can easily misjudge these gaps as defects, resulting in false detection.
[0056] Step 4: Model training: Use the differential image positive sample training set obtained in step 3 to iteratively train the chip anomaly detection model constructed in step 1 to obtain a trained chip anomaly detection model.
[0057] In this embodiment, the loss function convergence condition during chip anomaly detection model training is that the output loss function value needs to be reduced to 0.05.
[0058] Step 5, obtaining a grayscale differential image to be detected: Differentiate the grayscale image of the chip image to be detected from the grayscale differential base image obtained in step 2 to obtain a grayscale differential image to be detected.
[0059] The gray value of the pixel in the mth row and nth column of the grayscale image of the chip to be tested is Gray m×n , then the gray value of the pixel in the mth row and nth column in the gray difference image to be detected is The calculation formula is:
[0060]
[0061] Step 6, chip anomaly detection: The grayscale difference image to be detected obtained in step 5 is input into the chip anomaly detection model trained in step 4 to obtain the thermal map and defect location of the chip to be detected.
[0062] Figure 4 and Figure 5 The image of a chip area with two defects and its corresponding grayscale difference image are given. Figure 4 It can be seen that the given area includes two areas (marked with red boxes): an obvious defect in the upper right position, and a less obvious tiny defect in the lower left position. The tiny defect is closer to some features of the chip itself. Figure 5 In the grayscale difference image shown, the tiny defect in the lower left corner is highlighted because the image features of the chip are eliminated and faded due to the difference step.
[0063] In order to better illustrate the detection performance of the present invention for minor defects, the detection results of the grayscale difference image of the abnormality detection model chip described in the present invention are compared with the detection results of the non-differential chip image of the abnormality detection model trained using non-differential chip image samples.
[0064] Anomaly detection model trained using non-differential chip image samples Figure 4 The defect detection results of the test samples of the non-differential chip diagram shown in Figure 7 Among them, the blue / green and other cold-toned areas are normal areas, and the yellow / red and other warm-toned areas are defective areas. As can be seen from the figure, when the non-differential chip image input anomaly detection model is used for training and detection, the more obvious defects in the upper right position of the image can be identified. However, for the tiny defects in the lower left position, the defects cannot be identified due to the interference of the chip's own characteristics.
[0065] The anomaly detection model trained using the grayscale difference image sample set described in the present invention is Figure 5 The defect detection results of the test sample of the grayscale difference image of the chip to be tested are shown in Figure 6 As can be seen from the figure, after the interference of chip features on abnormality detection is eliminated or reduced by the image grayscale difference described in the present invention, not only the obvious defects in the upper right position can be identified, but also the tiny defects in the lower left position can be successfully identified, which significantly improves the detection performance of the abnormality detection model.
[0066] The chip defect detection method based on differential grayscale image anomaly detection provided by the present invention adopts an anomaly detection model, which solves the problem that the traditional differential detection method is easily interfered by factors such as color difference, solder feet, and random textures that occur during the chip manufacturing process. At the same time, by performing anomaly detection training and testing on the grayscale differential image of the chip image instead of the original image, the interference of chip features on the detection of minor defects can be eliminated. Compared with traditional algorithms, the false detection and missed detection rates are significantly reduced, and the detection performance is improved.
[0067] In the chip defect detection method based on differential grayscale image anomaly detection provided by an embodiment of the present invention, the grayscale images of the positive sample images of multiple chips are first averaged to obtain a grayscale differential base image; secondly, the grayscale image of the positive sample image of the chip is differentiated from the grayscale differential base image to obtain a grayscale differential image sample; then the grayscale differential image sample is input into the anomaly detection model for iterative training; the grayscale differential image to be detected is input into the trained anomaly detection model to obtain an output chip defect detection result.
[0068] The present invention utilizes the formed processing rules to solve the problem of misjudgment caused by chip color difference and calibration error in the traditional grayscale image difference method. At the same time, it can eliminate the interference of the complex characteristics of the chip itself on abnormal detection, improve the detection performance and accuracy of tiny defects of the chip, and reduce the missed detection rate.
[0069] The preferred embodiments of the present invention are described in detail above; however, the present invention is not limited to the specific details in the above embodiments. Within the technical concept of the present invention, various equivalent transformations can be made to the technical solutions of the present invention, and these equivalent transformations all belong to the protection scope of the present invention.
Claims
1. A chip defect detection method based on differential grayscale image anomaly detection, characterized in that: The steps include: Step 1: Build a chip anomaly detection model: The chip anomaly detection model is a deep learning model based on a neural network. The input of the chip anomaly detection model is the grayscale difference map of the chip, and the output of the chip anomaly detection model is the chip thermal map and the chip defect location. Step 2, obtaining a grayscale difference base map: the grayscale difference base map is a grayscale image of a positive chip sample image of the same type as the chip to be detected; Step 3, obtaining a differential image positive sample training set: performing differential analysis on the grayscale images of N positive sample images of the same type of chip and the grayscale differential base image obtained in step 2, respectively, to obtain a differential image positive sample training set; wherein N is not less than 100; the N chips of the same type include various types of non-defective error chips; wherein each type of non-defective error chips has no less than 10; Step 4: Model training: Use the differential image positive sample training set obtained in step 3, input the chip anomaly detection model constructed in step 1, perform iterative training, and obtain the trained chip anomaly detection model; Step 5, obtaining a grayscale differential image to be detected: performing differential analysis on the grayscale image of the chip image to be detected and the grayscale differential base image obtained in step 2 to obtain a grayscale differential image to be detected; Step 6, chip anomaly detection: The grayscale difference image to be detected obtained in step 5 is input into the chip anomaly detection model trained in step 4 to obtain the thermal map and defect location of the chip to be detected.
2. The chip defect detection method based on differential grayscale image anomaly detection according to claim 1, characterized in that: In step 3, non-defect error chips refer to chips whose chip images have characteristic changes but will not cause the chips to be unqualified; wherein, non-defect error chips include color difference change chips, solder foot change chips and random texture change chips.
3. The chip defect detection method based on differential grayscale image anomaly detection according to claim 1, characterized in that: In step 1, the chip anomaly detection model is one or a combination of the FastFlow model, the Padim model, the Patchcore model and the ReverseDistillation model.
4. The chip defect detection method based on differential grayscale image anomaly detection according to claim 3 is characterized in that: In step 1, the loss function of the chip anomaly detection model includes a cross entropy function obtained by combining the softmax regression classification function and the negative log-likelihood function (NLLloss).
5. The chip defect detection method based on differential grayscale image anomaly detection according to claim 4 is characterized in that: In step 4, the convergence condition of the loss function during chip anomaly detection model training is that the output loss function value must be reduced to 0.
05.
6. The chip defect detection method based on differential grayscale image anomaly detection according to claim 1, characterized in that: In step 2, the grayscale difference base map is the average value of the grayscale images of multiple positive chip sample images of the same type as the chip to be tested; wherein the positive chip samples are qualified chip samples without dust, broken edges, scratches or other defects that can be visually identified and affect the normal function of the chip; each positive chip sample image grayscale image is obtained by grayscale processing the collected positive chip sample image.
7. The chip defect detection method based on differential grayscale image anomaly detection according to claim 6 is characterized in that: Assume that the RGB values corresponding to the pixels in the mth row and nth column in the forward chip sample image are R m×n , G m×n and B m×n , then the gray value of the pixel in the mth row and nth column after conversion to grayscale image is Gray m×n The calculation formula is: Gray m×n =0.299×R m×n +0.587×G m×n +0.114×B m×n Suppose the gray value of the pixel in the mth row and nth column of the grayscale difference base map is The calculation formula is: In the formula, It is the gray value of the pixel in the mth row and nth column in the i-th chip grayscale image sample.
8. The chip defect detection method based on differential grayscale image anomaly detection according to claim 7 is characterized in that: In step 3, the grayscale value of the pixel in the mth row and nth column in the jth grayscale difference image sample in the difference image positive sample training set is The calculation formula is: In the formula, is the grayscale value of the pixel in the mth row and nth column in the jth chip grayscale image sample.
9. The chip defect detection method based on differential grayscale image anomaly detection according to claim 7, characterized in that: In step 5, the gray value of the pixel in the mth row and nth column of the grayscale image of the chip to be detected is Gray m×n , then the gray value of the pixel in the mth row and nth column in the gray difference image to be detected is The calculation formula is:
10. The chip defect detection method based on differential grayscale image anomaly detection according to claim 1, characterized in that: In step 1, the chip defect location refers to the area in the chip thermal map where the thermal value is greater than the set thermal threshold.
Citation Information
Cited By
Chip carrier band defect detection and sorting method and system, terminal and storage medium
CN121729057A