Chip defect detection method and device

Abstract

The invention provides a chip defect detection method and device. According to the chip defect detection method, characteristic values of sample chip images are obtained to represent color difference attributes of the sample chip images, various color difference intervals are subdivided according to the color difference attributes, and color difference classification is carried out on the sample chip images. And utilizing a machine learning model to automatically train and generate a standard feature image of each color difference interval to serve as a reference image of the to-be-detected chip image. On the basis, according to the characteristic value of the to-be-detected chip image, the color difference interval to which the to-be-detected chip image belongs can be judged, and then the standard characteristic image of the color difference interval to which the to-be-detected chip image belongs is selected to be subjected to defect comparison with the to-be-detected chip image. Compared with the prior art, the chip defect detection method provided by the invention can detect the defects of the to-be-detected chip under the same chromatic aberration attribute, is high in detection efficiency, can effectively avoid the interference of background chromatic aberration on defect detection, alleviates the false detection problem caused by chromatic aberration, and improves the accuracy of defect detection.

Description

Technical Field

[0001] The present invention relates to the field of semiconductor technology, and in particular to a chip defect detection method and device. Background Art

[0002] With the development of semiconductor technology, the market has higher and higher requirements for the accuracy of chip surface defect detection. At present, chip surface defect detection is mainly based on visual inspection methods, which use cameras to take pictures of each die on the wafer, and then compare and process the images to detect the defect location and defect type of the chip. However, in the existing process, the manufacture of special chips will cause obvious differences in brightness and darkness of dies at different positions on the same wafer. Such chips are generally called color difference process chips. Figure 1a to Figure 1c As shown, the average grayscale values ​​of the three Die100 images at different positions on the same wafer are significantly different. Since the background signal cannot be eliminated after comparing images with different degrees of color difference, it is easy to cause a large number of false detections, affecting the accuracy of the detection results. In this regard, the prior art is to adjust the image color difference by manually setting color difference related parameters to achieve the correction of the image background. However, this correction method is inefficient, has low color difference classification accuracy, and cannot be flexibly configured according to user scenarios, resulting in the defect detection accuracy of color difference process sheets is still low.

[0003] Therefore, a new chip defect detection method is urgently needed to solve the above technical problems. Summary of the invention

[0004] The object of the present invention is to provide a chip defect detection method and device to solve the problem of how to improve the defect detection accuracy of color difference process sheets.

[0005] In order to solve the above technical problems, the present invention provides a chip defect detection method, comprising:

[0006] Providing a plurality of sample chip images, and obtaining a feature value of each of the sample chip images; wherein at least some of the sample chip images have color differences;

[0007] determining a plurality of color difference intervals based on at least some of the characteristic values;

[0008] Determining the type of the color difference interval corresponding to each of the sample chip images according to the feature value thereof;

[0009] According to the sample chip images or the corresponding feature values ​​in the same color difference interval, training and generating a standard feature image of the corresponding color difference interval;

[0010] Acquire a characteristic value of the chip image to be tested, and determine the color difference interval corresponding to the chip image to be tested at least according to the characteristic value of the chip image to be tested;

[0011] The image of the chip to be inspected is compared with the standard feature image of the color difference interval corresponding to the image of the chip to be inspected to obtain a defect detection result of the chip to be inspected.

[0012] Optionally, in the chip defect detection method, the eigenvalue includes a eigenvector of an image, and the process of determining a plurality of color difference intervals at least based on all the eigenvalues, and determining the type of the color difference interval corresponding to each of the sample chip images includes:

[0013] Calculating the number of types of the color difference intervals according to all the characteristic values ​​and a preset clustering algorithm;

[0014] According to all the characteristic values ​​and the number of types of the color difference intervals, obtaining the color difference center of each color difference interval;

[0015] According to all the characteristic values ​​and the color difference center of each color difference interval, the type of each color difference interval and the color difference interval corresponding to each sample chip image are determined.

[0016] Optionally, in the chip defect detection method, the process of obtaining the color difference center of each color difference interval according to all the characteristic values ​​and the number of types of the color difference intervals includes:

[0017] Step 1: taking all the characteristic values ​​as a data point set, and selecting any data point as a color difference center;

[0018] Step 2: Calculate the distance between each data point in the data point set and the color difference center;

[0019] Step 3: Select the data point with the largest distance as another color difference center;

[0020] Step 4: Repeat step 2 to step 3 until the number of the acquired color difference centers reaches the number of types of the color difference intervals.

[0021] Optionally, in the chip defect detection method, the process of determining the type of each color difference interval and the color difference interval corresponding to each sample chip image according to all the characteristic values ​​and the color difference center of each color difference interval includes:

[0022] Calculating the distance between each of the data points and each of the color difference centers;

[0023] Select the color difference center with the smallest distance from the data point as the color difference center corresponding to the data point;

[0024] The data points corresponding to the same color difference center constitute a color difference interval.

[0025] Optionally, in the chip defect detection method, the process of training and generating a standard feature image of the color difference interval of the corresponding type according to the sample chip image or the corresponding feature value in the same color difference interval includes:

[0026] Inputting the characteristic values ​​of the sample chip image in each color difference interval into the machine learning model for iterative training to generate a central vector of each color difference interval that meets preset requirements;

[0027] The central vector of each color difference interval is reversely encoded to obtain the standard feature image of each color difference interval.

[0028] Optionally, in the chip defect detection method, the process of comparing the image of the chip to be inspected with the standard feature image of the color difference interval corresponding to the image of the chip to be inspected to obtain the defect detection result of the chip to be inspected includes:

[0029] Step 1: Calculate the correlation between the chip image to be tested and the standard feature image;

[0030] Step 2: Determine whether the correlation is greater than a threshold range. If so, the image of the chip to be tested and the standard feature image are in a linear relationship, and step 3 is executed; if not, the image of the chip to be tested and the standard feature image are in a nonlinear relationship, and step 4 is executed;

[0031] Step 3: performing linear transformation processing on the image of the chip to be inspected according to the standard feature image;

[0032] Step 4: performing nonlinear transformation processing on the image of the chip to be inspected according to the standard feature image;

[0033] Step 5: Compare the image of the chip to be inspected after linear transformation or nonlinear transformation with the standard feature image to obtain a defect detection result of the chip to be inspected.

[0034] Optionally, in the chip defect detection method, the linear transformation processing process includes: extracting key points of the chip image to be inspected and the standard feature image respectively, and performing linear regression fitting on the key points of the chip image to be inspected and the key points of the standard feature image to generate the chip image to be inspected after linear transformation; and

[0035] The process of the nonlinear transformation processing includes: respectively acquiring the histograms of the chip image to be tested and the standard feature image, and performing histogram normalization processing on the histogram of the chip image to be tested according to the histogram of the standard feature image to generate the chip image to be tested after nonlinear transformation.

[0036] Optionally, in the chip defect detection method, the characteristic value includes a grayscale average value of the image, and the process of determining a plurality of color difference intervals based on at least all of the characteristic values ​​includes:

[0037] According to the extreme difference range of the grayscale average values ​​of all the sample chip images and the preset grayscale color difference tolerance threshold, a plurality of color difference intervals are divided.

[0038] Optionally, in the chip defect detection method, the process of training and generating a standard feature image of the color difference interval of the corresponding type based on the sample chip image of the same color difference interval or the corresponding feature value includes:

[0039] A machine learning model is used to extract image features of the sample chip images within the same color difference interval, and a standard feature image of the color difference interval of the corresponding type is generated based on the image feature training.

[0040] Based on the same inventive concept, the present invention also provides a chip defect detection device, comprising: a data acquisition unit, a color difference classification unit, a feature training unit and a defect detection unit;

[0041] The data acquisition unit is used to acquire multiple sample chip images and extract feature values ​​of each sample chip image; and acquire a chip image to be tested and extract feature values ​​of the chip image to be tested; wherein at least some of the sample chip images have color difference;

[0042] The color difference classification unit is used to determine a plurality of color difference intervals according to at least part of the characteristic values, and to determine the type of the color difference interval corresponding to each of the sample chip images; and to determine the color difference interval corresponding to the chip image to be tested according to at least the characteristic values ​​of the chip image to be tested;

[0043] The feature training unit is used to train and generate a standard feature image of the color difference interval of the corresponding type according to the sample chip image or the corresponding feature value in the same color difference interval;

[0044] The defect detection unit is used to compare the image of the chip to be inspected with the standard feature image of the color difference interval corresponding to the image of the chip to be inspected, so as to obtain a defect detection result of the chip to be inspected.

[0045] In summary, the present invention provides a chip defect detection method and device. Among them, the chip defect detection method is to obtain the characteristic value of the sample chip image to characterize the color difference attribute of each sample chip image, and thereby subdivide a plurality of color difference intervals, and classify the color difference of each sample chip image. And, a machine learning model is used to automatically train and generate a standard characteristic image of each color difference interval as a reference image of the chip image to be inspected. Based on this, according to the characteristic value of the chip image to be inspected, the color difference interval to which the chip image to be inspected belongs can be determined, and then the standard characteristic image of the color difference interval to which the chip image to be inspected belongs is selected to compare defects with the chip image to be inspected. Compared with the prior art, the chip defect detection method provided by the present invention can detect defects of the chip to be inspected under the same color difference attribute, has high detection efficiency, and can effectively avoid the interference of background color difference on defect detection, alleviate the problem of false detection caused by color difference, and improve the accuracy of defect detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Those skilled in the art will appreciate that the drawings are provided for a better understanding of the present invention, but do not constitute any limitation on the scope of the present invention.

[0047] Figure 1a is a schematic diagram of Die having a first average grayscale value in the present invention.

[0048] Figure 1b Schematic diagram of Die with a second average gray value in the present invention.

[0049] Figure 1c is a schematic diagram of Die having a third average grayscale value in the present invention.

[0050] Figure 2 It is a flow chart of the chip defect detection method in the first embodiment of the present invention.

[0051] Figure 3 It is a flow chart of the chip defect detection method in the first embodiment of the present invention.

[0052] Figure 4 Schematic diagram of a standard characteristic image in the first color difference interval in the first embodiment of the present invention.

[0053] Figure 5 Schematic diagram of a standard characteristic image in the second color difference range in the first embodiment of the present invention.

[0054] Figure 6 Schematic diagram of a standard characteristic image in the third color difference interval in the first embodiment of the present invention.

[0055] Figure 7 Schematic diagram of a standard characteristic image in the fourth color difference interval in the first embodiment of the present invention.

[0056] Figure 8 It is a structural schematic diagram of a chip defect detection device in Embodiment 1 of the present invention.

[0057] Fig. 9 It is a flow chart of the chip defect detection method in the second embodiment of the present invention.

[0058] Fig.10 This is a flow chart of obtaining the number of color difference interval types in the second embodiment of the present invention.

[0059] Fig.11 This is a flow chart of reducing the color difference of the image of the chip to be inspected in the second embodiment of the present invention.

[0060] Fig.12 It is a schematic diagram of the image of the chip to be inspected in the second embodiment of the present invention.

[0061] Fig.13 It is the grayscale histogram of the image of the chip to be inspected in the second embodiment of the present invention.

[0062] Fig.14 It is a schematic diagram of a standard feature image in the second embodiment of the present invention.

[0063] Fig.15 It is the grayscale histogram of the standard feature image in the second embodiment of the present invention.

[0064] Fig.16 Schematic diagram of the image of the chip to be inspected after linear processing in the second embodiment of the present invention.

[0065] Fig.17 It is the grayscale histogram of the image of the chip to be inspected after linear processing in the second embodiment of the present invention.

[0066] Fig.18 Schematic diagram of the image of the chip to be inspected after nonlinear processing in the second embodiment of the present invention.

[0067] Fig.19 It is the grayscale histogram of the image of the chip to be inspected after nonlinear processing in the second embodiment of the present invention.

[0068] Fig. 20 It is a structural diagram of a chip defect detection device in Embodiment 2 of the present invention.

[0069] And, in the attached drawings:

[0070] 100-Die;

[0071] Embodiment 1: 201-standard characteristic image of the first color difference interval; 202-standard characteristic image of the second color difference interval; 203-standard characteristic image of the third color difference interval; 204-standard characteristic image of the fourth color difference interval; 301-data acquisition unit; 302-color difference classification unit; 303-feature training unit; 304-defect detection unit;

[0072] Embodiment 2: 401 - image of the chip to be inspected; 401a - image of the chip to be inspected after linear processing; 401b - image of the chip to be inspected after nonlinear processing; 402 - standard feature image; 501 - data acquisition unit; 502 - color difference classification unit; 503 - feature training unit; 504 - defect detection unit. DETAILED DESCRIPTION

[0073] In order to make the purpose, advantages and features of the present invention clearer, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the drawings are all in a very simplified form and are not drawn to scale, and are only used to conveniently and clearly assist in explaining the purpose of the embodiments of the present invention. In addition, the structure shown in the drawings is often a part of the actual structure. In particular, the emphasis of each drawing is different, and sometimes different proportions are used. It should also be understood that, unless otherwise specified or indicated, the terms "first", "second", "third", etc. in the specification are only used to distinguish between the various components, elements, steps, etc. in the specification, and are not used to represent the logical relationship or sequential relationship between the various components, elements, steps, etc.

[0074] <Example 1>

[0075] See also Figure 2 and Figure 3 , this embodiment provides a chip defect detection method, including:

[0076] Step 1 S10: providing a plurality of sample chip images, and obtaining a feature value of each of the sample chip images; wherein at least some of the sample chip images have color differences.

[0077] It should be noted that the sample chip image referred to in this embodiment can be an image of a die at a different position on the same wafer, or an image of a die on different wafers in the same batch, and an integrated circuit is formed on each die. In addition, each of the sample chip images is a defect-free pattern, or a pattern that meets the defect standard range. However, in view of the process, there is a certain color difference between the multiple sample chip images. That is, the average grayscale value of at least some of the sample chip images is different. Based on this, the feature value in this embodiment is the average grayscale value of all pixels on the sample chip image.

[0078] Step 2 S20: Determine a plurality of color difference intervals according to at least part of the characteristic values; and determine the type of the color difference interval corresponding to the characteristic value of each sample chip image.

[0079] Specifically, according to the range of the grayscale average values ​​of all the sample chip images and the preset grayscale color difference tolerance threshold, a plurality of color difference intervals are divided. For example, in step 1 S10, 10 sample chip images are provided, and the characteristic values ​​of the sample chip images 1 to 10 are 202, 232, 231, 210, 211, 201, 203, 202, 230, and 209, respectively. The maximum value of the grayscale average values ​​of all the sample chip images is 232, the minimum value is 201, and the range is 31. Assuming that the grayscale color difference tolerance threshold is 5, the number of types of the color difference intervals is 31 / 5≈6 (rounded off). Therefore, the color difference intervals are: the first color difference interval [201, 206], the second color difference interval [207, 212], the third color difference interval [213, 218], the fourth color difference interval [219, 224], the fifth color difference interval [225, 230] and the sixth color difference interval [231, 236]. Based on this, it can be determined that the 1st, 6th, 7th and 8th sample chip images belong to the first color difference interval; the 4th, 5th and 10th sample chip images belong to the second color difference interval; the 9th sample chip image belongs to the fifth color difference interval; and the 2nd and 3rd sample chip images belong to the sixth color difference interval. Alternatively, assuming that the grayscale color difference tolerance threshold is 10, the number of types of the color difference intervals is 31 / 10≈3, and the color difference intervals are: the first color difference interval [201, 211], the second color difference interval [212, 222], and the third color difference interval [223, 233]. Based on this, the 1st, 4th to 8th, and 10th sample chip images belong to the first color difference interval; the 2nd, 3rd, and 9th sample chip images belong to the third color difference interval.

[0080] It should be noted that the smaller the grayscale color difference tolerance threshold is, the higher the accuracy of the color difference interval division is, and it can be flexibly configured according to the accuracy requirements of the user scenario. And, when the number of the sample chip images is large enough, each color difference interval corresponds to at least one sample chip image. Among them, this embodiment does not limit the specific number of the sample chip images.

[0081] Step three S30: training and generating a standard feature image of the color difference interval of the same type according to the sample chip images or the corresponding feature values ​​within the color difference interval.

[0082] Exemplarily, the number of sample chip images is 10, and the 1st to 3rd sample chip images belong to the first color difference interval; the 4th to 6th sample chip images belong to the second color difference interval; the 7th and 8th sample chip images belong to the third color difference interval; and the 9th and 10th sample chip images belong to the fourth color difference interval. Therefore, in order to obtain an image that can characterize the grayscale state of each color difference interval, a machine learning model is used to extract image features of the sample chip images in the same color difference interval, and a standard feature image of the corresponding color difference interval is generated based on the image feature training. That is, the 1st to 3rd sample chip images are first formed into a first image group, the 4th to 6th sample chip images are formed into a second image group, the 7th and 8th sample chip images are formed into a third image group, and the 9th and 10th sample chip images are formed into a fourth image group. Then, the data of the first image group is input into the machine learning model, and the feature images of the first sample chip image, the second sample chip image and the third sample chip image are extracted and trained by the machine learning model to generate the following combination: Figure 4 Similarly, the data of the second image group, the third image group and the fourth image group are respectively input into the machine learning model to generate the standard feature image 201 of the first color difference interval as shown. Figure 5 The standard characteristic image 202 of the second color difference interval shown in FIG. Figure 6 The standard characteristic image 203 of the third color difference interval shown in FIG. Figure 7 The standard feature image 204 of the fourth color difference interval is shown. In this embodiment, the specific type of the machine learning model for implementing feature extraction is not limited, and it can be a convolutional neural network or the like.

[0083] Step 4 S40: Acquire the characteristic value of the image of the chip to be tested, and determine the color difference interval corresponding to the image of the chip to be tested at least according to the characteristic value of the image of the chip to be tested.

[0084] Exemplarily, the average grayscale value of the chip image to be tested is 205, and the average grayscale value range of the first color difference interval determined in step 2 S20 is [201, 206], then the color difference state of the chip image to be tested corresponds to the first color difference interval.

[0085] Step five S50: Compare the image of the chip to be inspected with the standard feature image of the color difference interval corresponding to the image of the chip to be inspected to obtain a defect detection result of the chip to be inspected.

[0086] It can be understood that the standard feature image can characterize the color difference background characteristics of the color difference interval to which it belongs, and the standard feature image is generated based on the defect-free sample chip image, so the standard feature image can be used as a reference image for chip defect detection within the same color difference interval. Therefore, after obtaining the color difference interval to which the image of the chip to be inspected belongs, the standard feature image corresponding to the color difference interval can be retrieved as a reference image for the image of the chip to be inspected. Based on this, under the same color difference attribute, the defect comparison between the image of the chip to be inspected and the standard feature image can effectively avoid the interference of background color difference on defect detection, alleviate the problem of false detection caused by color difference, and improve the accuracy of defect detection.

[0087] Based on the same inventive concept, please refer to Figure 8 , this embodiment also provides a chip defect detection device, including: a data acquisition unit 301, a color difference classification unit 302, a feature training unit 303 and a defect detection unit 304. Specifically, the data acquisition unit 301 is used to acquire multiple sample chip images and extract the feature value of each sample chip image; and acquire the chip image to be inspected and extract the feature value of the chip image to be inspected; wherein at least some of the sample chip images have color difference. In other words, the data acquisition unit 301 is used to acquire images and acquire the average gray value of each pixel unit in the image as a feature value. The color difference classification unit 302 is used to determine multiple color difference intervals according to at least some of the feature values, and determine the type of the color difference interval corresponding to each sample chip image; and determine the color difference interval corresponding to the chip image to be inspected according to at least the feature value of the chip image to be inspected. The feature training unit 303 is used to train and generate a standard feature image of the color difference interval of the corresponding type according to the sample chip images or the corresponding feature values ​​in the same color difference interval. The defect detection unit 304 is used to compare the image of the chip to be inspected with the standard feature image of the color difference interval corresponding to the image of the chip to be inspected, so as to obtain a defect detection result of the chip to be inspected.

[0088] In summary, the chip defect detection method and device provided in this embodiment uses the grayscale average value of the image as the color difference feature value, subdivides into a variety of color difference intervals, and uses machine learning model training to generate standard feature images for each color difference interval as the reference image of the chip image to be inspected. Based on this, according to the grayscale average value of the chip image to be inspected and the various color difference intervals, the color difference attribute of the chip image to be inspected can be judged, and then the defects of the chip to be inspected can be detected under the same color difference attribute, without the need to correct the chip image to be inspected with color difference to a color-free state before detection. Not only is the detection efficiency high, but it also effectively avoids the interference of background color difference on defect detection, alleviates the problem of false detection caused by color difference, and improves the accuracy of defect detection.

[0089] <Example 2>

[0090] In order to further improve the detection accuracy of the chip defect detection method, this embodiment also provides a chip defect detection method. Fig. 9 , the chip defect detection method comprises:

[0091] Step 1 S10: providing a plurality of sample chip images, and obtaining a feature value of each of the sample chip images; wherein at least some of the sample chip images have color differences.

[0092] To facilitate subsequent data processing, the characteristic value in this embodiment includes a characteristic vector of the image, that is, the data of the sample chip image is vector-encoded. Exemplarily, the coordinate position of each pixel unit in the sample chip image and the grayscale value corresponding to each pixel unit are constructed into a characteristic vector of the sample chip image. Alternatively, a logical operation or other mixed operation is performed on the grayscale values ​​corresponding to all pixel units in the sample chip image, and the operation result and the number of the sample chip image are used to form the characteristic vector of the sample chip image. It should be noted that this embodiment does not limit the specific vector encoding method, and it is sufficient to convert the image format into a vector format.

[0093] Step 2 S20: Determine a plurality of color difference intervals according to at least part of the characteristic values; and determine the type of the color difference interval corresponding to the characteristic value of each sample chip image.

[0094] In order to improve the classification accuracy of the color difference interval and improve the accuracy of defect detection, this embodiment adopts an automatic optimization method to obtain the color difference interval. Specifically, the number of types of the color difference interval is calculated based on all the eigenvalues ​​and the preset clustering algorithm. Exemplarily, the number of color difference interval types is automatically obtained by using the elbow method based on K-means. That is, the objective function is set to minimize the square error:

[0095]

[0096] Wherein, k is the total number of types of color difference intervals, Ci is the i-th color difference interval, and μ_i is the mean vector of Ci. It can be understood that, compared with manually setting the color difference tolerance threshold, using a clustering algorithm to automatically determine the optimal number of types of the color difference interval has high computational efficiency and high accuracy.

[0097] Further, after obtaining the number k of the color difference interval types, it is necessary to determine the center vector of each color difference interval. That is, according to all the eigenvalues ​​and the number of the color difference interval types, the color difference center of each color difference interval is obtained. For example, Fig.10 As shown, the process includes:

[0098] Step 1 S201: All the characteristic values ​​are taken as a data point set, and any data point is selected as a color difference center.

[0099] Exemplarily, the serial number of the sample chip image and the corresponding characteristic value constitute a data point, and all the data points constitute the data point set. To obtain the central vector of each color difference interval, that is, the color difference center, one of the data points can be arbitrarily selected as one of the color difference centers.

[0100] Step 2 S202: Calculate the distance between each data point in the data point set and the color difference center.

[0101] Step three S203: Select the data point with the largest distance as another color difference center.

[0102] It should be noted that each color difference interval is equivalent to a cluster, and the greater the distance between any two data points, the greater the possibility that the two are in different clusters. Therefore, the data point with the largest distance from the color difference center has the greatest possibility of becoming the center vector of another color difference interval, and this data point is used as the second color difference center.

[0103] Step 4 S204: Repeat step 2 to step 3 until the number of the acquired color difference centers reaches the number of types of the color difference intervals.

[0104] Similarly, according to the maximum distance selection principle, k color difference centers can be obtained. It can be understood that the accuracy of the color difference centers obtained here is limited, but the approximate range of each color difference interval can be clearly distinguished.

[0105] Furthermore, after obtaining the color difference center of each color difference interval, the type of the color difference interval corresponding to each color difference interval and each sample chip image is determined based on all the characteristic values ​​and the color difference center of each color difference interval. Specifically, based on the clustering principle, the distance between each data point in the data point set and each color difference center can be calculated, and then the color difference center with the smallest distance to the data point is selected as the color difference center corresponding to the data point, and finally the data points corresponding to the same color difference center constitute a color difference interval. In this way, the sample chip image corresponding to each color difference interval can be determined.

[0106] Step three S30: training and generating a standard feature image of the color difference interval of the same type according to the sample chip images or the corresponding feature values ​​within the color difference interval.

[0107] Specifically, the characteristic values ​​of the sample chip image in each color difference interval are input into the machine learning model for iterative training to generate the center vector of each color difference interval that meets the preset requirements. Exemplarily, the preset requirement for controlling the training to stop may be the number of iterations, or the minimum error change is less than a threshold. Furthermore, in this embodiment, the data trained by the machine learning model is vector data, and there is no need to perform image-level training processing on the entire image data. Based on this, the output result of the machine learning model is the center vector of each color difference interval, and compared with the aforementioned color difference center, the center vector can more accurately characterize the characteristic conditions of the corresponding color difference interval. After obtaining the center vector of each color difference interval, each center vector is reversely encoded to obtain the standard feature image of each color difference interval. Among them, this embodiment does not limit the specific machine learning model.

[0108] Step 4 S40: Acquire the characteristic value of the image of the chip to be tested, and determine the color difference interval corresponding to the image of the chip to be tested at least according to the characteristic value of the image of the chip to be tested.

[0109] In view of the color difference interval obtained in the above step, the type of the color difference interval to which the chip image to be tested belongs can be found according to the characteristic value, that is, the characteristic vector, of the chip image to be tested.

[0110] Step five S50: Compare the image of the chip to be inspected with the standard feature image of the color difference interval corresponding to the image of the chip to be inspected to obtain a defect detection result of the chip to be inspected.

[0111] It should be noted that, when the detection accuracy requirement is not high, the defect detection can be directly performed by referring to the comparison method in Example 1. That is, the standard feature image corresponding to the color difference interval to which the image of the chip to be tested belongs is retrieved, and the image of the chip to be tested is compared with the standard feature image to obtain the defects of the chip to be tested. However, it is understandable that there is still a certain degree of color difference between the standard feature image corresponding to the color difference interval to which the image of the chip to be tested belongs and the image of the chip to be tested. Therefore, please refer to Fig.11 In order to further avoid the influence of color difference on defect detection, this embodiment also needs to process the chip to be inspected during the comparison process, as follows:

[0112] Step 1 S501: Calculate the correlation between the image of the chip to be inspected and the standard feature image.

[0113] Specifically, the correlation r between the image of the chip to be inspected and the standard feature image is calculated as follows:

[0114]

[0115] Among them, R i is the grayscale value of the i-th pixel unit in the image of the chip to be inspected, is the average gray value of the image of the chip to be tested; T i is the gray value of the i-th pixel unit in the standard feature image, is the average gray value of the standard feature image.

[0116] Step 2 S502: Determine whether the correlation is greater than a threshold range. If so, the image of the chip to be tested is linearly related to the standard feature image, and step 3 is executed; if not, the image of the chip to be tested is nonlinearly related to the standard feature image, and step 4 is executed.

[0117] Step three S503: performing linear transformation processing on the image of the chip to be inspected according to the standard feature image.

[0118] Optionally, the image of the chip to be tested is processed by linear fitting to reduce the color difference between the image of the chip to be tested and the standard feature image. Specifically, the ORB algorithm is first used to extract the key points of the image of the chip to be tested and the standard feature image respectively. Among them, the ORB algorithm can also resist the influence of noise and image transformation to a certain extent. Then, the key points of the image of the chip to be tested and the key points of the standard feature image are linearly regressed and fitted. That is, the linear fitting model Test(i,j)=k*Ref(i,j)+b is satisfied. Among them, i is a key point of the image of the chip to be tested, and j is a key point of the standard feature image. All key points are brought into the linear fitting model, and the least squares method is used to optimize the straight line fitting, and the constants k and b in the linear fitting model are calculated. Based on this, the linear fitting model is simplified as: y=kx+bI. Further let: X=[x,I], c=(k,b)T, then the linear fitting model can be further expressed as: y=Xc. Thus, the solution of the linear fitting model is: c=(XTX)-1XTy. Based on the linear fitting model, the linear transformation of the chip image to be tested can be realized to generate the chip image to be tested after linear transformation, thereby further reducing the color difference between the chip image to be tested and the standard feature image.

[0119] Step 4 S504: Perform nonlinear transformation processing on the image of the chip to be inspected according to the standard feature image.

[0120] Optionally, the image of the chip to be tested is processed by histogram regulation to reduce the color difference between the image of the chip to be tested and the standard feature image. The histogram reflects the relationship between each gray level in the digital image and its frequency of occurrence. Based on this, the histograms of the image of the chip to be tested and the standard feature image are first obtained respectively, and then the histogram of the image of the chip to be tested is processed by histogram regulation according to the histogram of the standard feature image to generate the image of the chip to be tested after nonlinear transformation. That is, the transformation function of the histogram regulation of the image of the chip to be tested is solved according to the probability density function of the standard feature image. Then, the gray level s obtained in the image of the chip to be tested is used instead of v to solve the inverse transformation z=G-1(s), thereby obtaining the transformed image of the image of the chip to be tested, so as to further reduce the color difference between the image of the chip to be tested and the standard feature image. In this regard, this embodiment does not limit the use of histogram regulation to perform nonlinear data processing.

[0121] Step 5 S505: Compare the image of the chip to be inspected after linear transformation or nonlinear transformation with the standard feature image to obtain a defect detection result of the chip to be inspected.

[0122] Based on this, Figures 12 to 19As shown, before the linear transformation or nonlinear transformation, the image of the chip to be inspected 401 and the standard feature image 402 have obvious color difference, which easily affects the accuracy of chip defect detection. However, after the linear transformation, the color difference between the image of the chip to be inspected 401a and the standard feature image 402 is significantly reduced, and after the nonlinear transformation, the color difference between the image of the chip to be inspected 401a and the standard feature image 402 is also significantly reduced, which is conducive to eliminating the interference of color difference factors on defect detection.

[0123] Based on the same inventive concept, please refer to Fig. 20 , this embodiment also provides a chip defect detection device, including: a data acquisition unit 501, a color difference classification unit 502, a feature training unit 503 and a defect detection unit 504. Specifically, the data acquisition unit 501 is used to acquire multiple sample chip images and extract the feature value of each sample chip image; and acquire the chip image to be inspected and extract the feature value of the chip image to be inspected; wherein at least some of the sample chip images have color difference. In other words, the data acquisition unit 501 is used to acquire images and vector encode the images to serve as feature values ​​of the images. The color difference classification unit 502 is used to determine multiple color difference intervals according to at least some of the feature values, and determine the type of the color difference interval corresponding to each sample chip image; and determine the color difference interval corresponding to the chip image to be inspected according to at least the feature value of the chip image to be inspected. Among them, this embodiment adopts an automatic optimization method to obtain the color difference interval and determine the color difference interval corresponding to each image. Compared with the method of manually setting the color difference tolerance threshold to obtain each color difference interval in the first embodiment, the classification accuracy is higher and the calculation efficiency is faster. The feature training unit 503 is used to train and generate a standard feature image of the color difference interval of the corresponding type according to the sample chip image or the corresponding feature value in the same color difference interval. The defect detection unit 504 is used to compare the image of the chip to be inspected with the standard feature image of the color difference interval corresponding to the image of the chip to be inspected to obtain the defect detection result of the chip to be inspected. Furthermore, before the comparison, this embodiment also performs linear or nonlinear processing on the image of the chip to be inspected by determining the correlation between the image of the chip to be inspected and the standard feature image, so as to further reduce the color difference between the image of the chip to be inspected and the standard feature image, alleviate the problem of false detection or inaccurate detection due to color difference, and improve the accuracy of defect detection.

[0124] It should be noted that for the definitions of terms or implementation conditions not fully described in this embodiment, please refer to the description in Embodiment 1.

[0125] In summary, the chip defect detection method and device provided in this embodiment performs vector encoding on the sample chip image as its eigenvalue, and uses a clustering algorithm based on each eigenvalue to automatically optimize and obtain each color difference interval, which is beneficial to improving the color difference classification accuracy and improving the classification efficiency. In addition, the chip defect detection method and device can flexibly adapt to different detection accuracy requirements. And when the detection accuracy requirement is low, the standard feature image of the color difference interval corresponding to the chip image to be inspected can be directly obtained as a reference image, and the defects of the chip to be inspected can be detected under the same color difference attribute, with high detection efficiency. And, when the detection accuracy requirement is high, the chip image to be inspected is further corrected so that the color difference between the chip image to be inspected and the standard feature image is reduced to a minimum, which is beneficial to eliminate the influence of the color difference factor on the defect detection accuracy, and can obtain more accurate detection results.

[0126] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other. In addition, the different parts between the various embodiments can also be used in combination with each other, and the present invention is not limited to this.

[0127] In addition, it should be recognized that although the present invention has been disclosed as a preferred embodiment, the above embodiment is not intended to limit the present invention. For any technician familiar with the art, without departing from the scope of the technical solution of the present invention, the technical content disclosed above can be used to make many possible changes and modifications to the technical solution of the present invention, or modified into equivalent embodiments of equivalent changes. Therefore, any simple modification, equivalent change and modification made to the above embodiment according to the technical essence of the present invention without departing from the content of the technical solution of the present invention still belongs to the scope of protection of the technical solution of the present invention.

Claims

1. A chip defect detection method, characterized in that: include: Providing a plurality of sample chip images, and obtaining a feature value of each of the sample chip images; wherein at least some of the sample chip images have color differences; determining a plurality of color difference intervals based on at least some of the characteristic values; Determining the type of the color difference interval corresponding to each of the sample chip images according to the feature value thereof; According to the sample chip images or the corresponding feature values ​​in the same color difference interval, training and generating a standard feature image of the corresponding color difference interval; Acquire a characteristic value of the chip image to be tested, and determine the color difference interval corresponding to the chip image to be tested at least according to the characteristic value of the chip image to be tested; The image of the chip to be inspected is compared with the standard feature image of the color difference interval corresponding to the image of the chip to be inspected to obtain a defect detection result of the chip to be inspected.

2. The chip defect detection method according to claim 1, characterized in that: The eigenvalues ​​include eigenvectors of the image, and the process of determining a plurality of color difference intervals at least according to all the eigenvalues, and determining the type of the color difference interval corresponding to each sample chip image includes: Calculating the number of types of the color difference intervals according to all the characteristic values ​​and a preset clustering algorithm; According to all the characteristic values ​​and the number of types of the color difference intervals, obtaining the color difference center of each color difference interval; According to all the characteristic values ​​and the color difference center of each color difference interval, the type of each color difference interval and the color difference interval corresponding to each sample chip image are determined.

3. The chip defect detection method according to claim 2, characterized in that: The process of obtaining the color difference center of each color difference interval according to all the characteristic values ​​and the number of types of the color difference intervals includes: Step 1: taking all the characteristic values ​​as a data point set, and selecting any data point as a color difference center; Step 2: Calculate the distance between each data point in the data point set and the color difference center; Step 3: Select the data point with the largest distance as another color difference center; Step 4: Repeat step 2 to step 3 until the number of the acquired color difference centers reaches the number of types of the color difference intervals.

4. The chip defect detection method according to claim 3, characterized in that: The process of determining the type of each color difference interval and the color difference interval corresponding to each sample chip image according to all the characteristic values ​​and the color difference center of each color difference interval includes: Calculating the distance between each of the data points and each of the color difference centers; Select the color difference center with the smallest distance from the data point as the color difference center corresponding to the data point; The data points corresponding to the same color difference center constitute a color difference interval.

5. The chip defect detection method according to claim 2, characterized in that: The process of training and generating a standard feature image of a corresponding type of color difference interval according to the sample chip image or the corresponding feature value in the same type of color difference interval includes: Inputting the characteristic values ​​of the sample chip image in each color difference interval into the machine learning model for iterative training to generate a central vector of each color difference interval that meets preset requirements; The central vector of each color difference interval is reversely encoded to obtain the standard feature image of each color difference interval.

6. The chip defect detection method according to claim 2, characterized in that: The process of comparing the image of the chip to be inspected with the standard feature image of the color difference interval corresponding to the image of the chip to be inspected to obtain the defect detection result of the chip to be inspected includes: Step 1: Calculate the correlation between the chip image to be tested and the standard feature image; Step 2: Determine whether the correlation is greater than a threshold range. If so, the image of the chip to be tested and the standard feature image are in a linear relationship, and step 3 is executed; if not, the image of the chip to be tested and the standard feature image are in a nonlinear relationship, and step 4 is executed; Step 3: performing linear transformation processing on the image of the chip to be inspected according to the standard feature image; Step 4: performing nonlinear transformation processing on the image of the chip to be inspected according to the standard feature image; Step 5: Compare the image of the chip to be inspected after linear transformation or nonlinear transformation with the standard feature image to obtain a defect detection result of the chip to be inspected.

7. The chip defect detection method according to claim 6, characterized in that: The linear transformation process includes: extracting key points of the chip image to be tested and the standard feature image respectively, and performing linear regression fitting on the key points of the chip image to be tested and the key points of the standard feature image to generate the chip image to be tested after linear transformation; and The process of the nonlinear transformation processing includes: respectively acquiring the histograms of the chip image to be tested and the standard feature image, and performing histogram normalization processing on the histogram of the chip image to be tested according to the histogram of the standard feature image to generate the chip image to be tested after nonlinear transformation.

8. The chip defect detection method according to claim 1, characterized in that: The characteristic value includes a grayscale average value of the image, and the process of determining multiple color difference intervals at least based on all the characteristic values ​​includes: According to the extreme difference range of the grayscale average values ​​of all the sample chip images and the preset grayscale color difference tolerance threshold, a plurality of color difference intervals are divided.

9. The chip defect detection method according to claim 8, characterized in that: The process of training and generating a standard feature image of a corresponding type of color difference interval according to the sample chip image of the same type of color difference interval or the corresponding feature value comprises: A machine learning model is used to extract image features of the sample chip images within the same color difference interval, and a standard feature image of the color difference interval of the corresponding type is generated based on the image feature training.

10. A chip defect detection device, characterized in that: include: Data acquisition unit, color difference classification unit, feature training unit and defect detection unit; The data acquisition unit is used to acquire multiple sample chip images and extract feature values ​​of each sample chip image; and acquire a chip image to be tested and extract feature values ​​of the chip image to be tested; wherein at least some of the sample chip images have color difference; The color difference classification unit is used to determine a plurality of color difference intervals according to at least part of the characteristic values, and to determine the type of the color difference interval corresponding to each of the sample chip images; and to determine the color difference interval corresponding to the chip image to be tested according to at least the characteristic values ​​of the chip image to be tested; The feature training unit is used to train and generate a standard feature image of the color difference interval of the corresponding type according to the sample chip image or the corresponding feature value in the same color difference interval; The defect detection unit is used to compare the image of the chip to be inspected with the standard feature image of the color difference interval corresponding to the image of the chip to be inspected, so as to obtain a defect detection result of the chip to be inspected.

Images