A semiconductor chip surface defect detection method, device and medium

Abstract

The application discloses a kind of semiconductor chip surface defect detection method, device and medium, it is related to artificial intelligence technical field, the method includes: obtaining the original image of semiconductor chip;Original image is compared with the template image of semiconductor chip pixel by pixel, and the binary mask image of gray super difference is obtained;Wherein, the template image of semiconductor chip is standard and the reference figure without defect;Connected domain analysis is carried out to binary mask image, and at least one suspected defect area is obtained;For each suspected defect area, the image of suspected defect area in the original image, template image and the corresponding position area of binary mask image is cut out, and is merged into three channel image;Each three channel image is detected by confidence degree by defect detection model based on deep learning, and the detection result of each suspected defect area is determined, and the detection result is defective or defectless. Utilize the method to carry out defect detection to semiconductor chip, improve defect detection accuracy.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and specifically to a method, apparatus, and medium for detecting surface defects in semiconductor chips. Background Technology

[0002] Machine vision-based product surface defect detection is a technical means in industrial quality inspection. Traditional machine vision-based product surface defect detection methods typically first use a specific light source system and a high-speed industrial camera to photograph the product surface. Then, based on the product image, traditional image processing algorithms (such as histogram equalization, filtering and denoising, and grayscale binarization) are used to obtain a simplified image with foreground and background separation. Finally, mathematical morphology, Fourier transform, Gabor transform, and other algorithms, as well as machine learning models, are used to complete the labeling and detection of defects.

[0003] While image processing algorithms for product surface defect detection have achieved good results in certain specific applications, they still have many shortcomings. For example, image preprocessing steps are numerous and highly targeted. When facing the detection needs of new products, the algorithm iteration speed is slow and its versatility is poor. It requires highly specialized R&D personnel to manually extract features of specific defects, resulting in high R&D costs. Furthermore, it has high requirements for image acquisition conditions, such as changes in lighting conditions, image size, resolution, and whether the image is rotated or deformed.

[0004] Deep learning is a new research direction in the field of machine learning, motivated by the goal of building and simulating neural networks that allow the human brain to analyze and learn. Deep learning can fit labeled data to massive datasets, resulting in models with excellent robustness and accuracy. It exhibits remarkable adaptability to variations in lighting, data resolution, rotation, and scaling of the sample data. Solutions utilizing deep neural networks such as convolutional neural networks can gradually replace traditional algorithm-based image processing methods. Currently, deep learning-based image processing methods far surpass traditional rule-based methods in stability and robustness. By automatically learning common features from large datasets and updating model parameters, deep learning avoids the need for manually designing complex algorithms, significantly reducing development difficulties and meeting real-time industrial requirements.

[0005] For semiconductor chips, if a single raw image of the chip is directly used as input to train a neural network model, the accuracy of surface defect detection using the trained model is low. This is because during model training, only the label information of the input image is available. When the neural network extracts image features through convolution, it does not know the specific location of the defect. The trained model cannot confirm whether it has effectively extracted features at the defect location, but the key factor determining the model's prediction accuracy is often the model's feature extraction of key defect locations. In addition, when detecting defects in images, interference from complex textures often prevents the model from truly focusing on the core defect features to determine the authenticity of the defect. As a result, defect detection models based on single raw images perform poorly in both the training and prediction phases. Summary of the Invention

[0006] The technical problem this invention aims to solve is that directly using a single raw image of a semiconductor chip as input to train a neural network model results in low accuracy for detecting surface defects on semiconductor chips. To address this problem, this invention provides a method, apparatus, and medium for detecting surface defects on semiconductor chips.

[0007] The technical solution of the present invention to solve the above-mentioned technical problems is as follows:

[0008] A method for detecting surface defects in a semiconductor chip includes the following steps:

[0009] Step 1: Obtain the raw image of the semiconductor chip;

[0010] Step 2: Compare the original image with the template image of the semiconductor chip pixel by pixel to obtain a binarized mask image with grayscale discrepancies; wherein, the template image of the semiconductor chip is a standard reference image without defects;

[0011] Step 3: Perform connected component analysis on the binarized mask image to obtain at least one suspected defect region;

[0012] Step 4: For each suspected defect region, extract the image of the suspected defect region at the corresponding position in the original image, the template image, and the binarized mask image, and merge them into a three-channel image;

[0013] Step 5: Perform confidence detection on each of the three-channel images using a deep learning-based defect detection model to determine the detection result of each of the suspected defect regions, wherein the detection result is either defective or not defective.

[0014] To address the aforementioned technical problems, the present invention also provides a semiconductor chip surface defect detection device, comprising:

[0015] Image acquisition module, used to acquire raw images of semiconductor chips;

[0016] The first image processing module is used to compare the original image with the template image of the semiconductor chip pixel by pixel to obtain a binarized mask image with grayscale discrepancies; wherein the template image of the semiconductor chip is a standard reference image without defects;

[0017] The second image processing module is used to perform connected component analysis on the binarized mask image to obtain at least one suspected defect region.

[0018] The third image processing module is used to extract the image of the suspected defect region in the corresponding position area of ​​the original image, the template image and the binarized mask image for each suspected defect region, and merge them into a three-channel image;

[0019] The defect detection module is used to perform confidence detection on each of the three-channel images using a deep learning-based defect detection model, and to determine the detection result of each of the suspected defect regions, wherein the detection result is either defective or defect-free.

[0020] To address the aforementioned technical problems, the present invention also provides a semiconductor chip surface defect detection device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the semiconductor chip surface defect detection method as described above.

[0021] To address the aforementioned technical problems, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the semiconductor chip surface defect detection method as described above.

[0022] The beneficial effects of this invention are as follows: For each suspected defect region, images of the corresponding locations of the suspected defect region in the original image, template image, and binarized mask image are extracted and merged into a three-channel image. After inputting the three-channel image into a deep learning-based defect detection model, feature extraction is performed on the corresponding locations of the suspected defect region in the three channels of the original image, template image, and binarized mask image using a convolution parameter sharing method. This feature extraction method allows the model to learn the differences between the corresponding locations of the suspected defect region in the original image and the corresponding locations of the suspected defect region in the template image. In addition, by adding a binarized mask image, the model can more effectively extract features of the defect location. This method, by adding a template image and a binarized mask image to the model, is equivalent to adding an "attention mechanism" to the model, which allows the model to pay more attention to the features of the defect region. This method enables the model to complete the extraction of defect features based on the suspected defect region (which is contained in the binarized mask image), the original image, and the template image, thereby improving the accuracy of the model in defect detection and achieving high detection efficiency. Attached Figure Description

[0023] Figure 1 This is a schematic flowchart A of the semiconductor chip surface defect detection method of the present invention;

[0024] Figure 2 This is a flowchart (B) illustrating the semiconductor chip surface defect detection method of the present invention.

[0025] Figure 3 This is a schematic diagram of the framework of the defect detection model in this invention;

[0026] Figure 4 This is a schematic diagram of the semiconductor chip surface defect detection system of the present invention. Detailed Implementation

[0027] The principles and features of the present invention are described below. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0028] Example 1

[0029] like Figure 1 and Figure 2 As shown, this embodiment provides a method for detecting surface defects in a semiconductor chip, including the following steps:

[0030] Step 1: Obtain the raw image of the semiconductor chip;

[0031] Step 2: Compare the original image with the template image of the semiconductor chip pixel by pixel to obtain a binarized mask image with grayscale discrepancies; wherein, the template image of the semiconductor chip is a standard reference image without defects;

[0032] Step 3: Perform connected component analysis on the binarized mask image to obtain at least one suspected defect region;

[0033] Step 4: For each suspected defect region, extract the image of the suspected defect region at the corresponding position in the original image, the template image, and the binarized mask image, and merge them into a three-channel image;

[0034] Step 5: Perform confidence detection on each of the three-channel images using a deep learning-based defect detection model to determine the detection result of each of the suspected defect regions, wherein the detection result is either defective or not defective.

[0035] In this embodiment of the invention, the suspected defect area meets the detection specifications of the semiconductor chip industry, and the parameters corresponding to the detection specifications are such as defect width, defect height, and defect area.

[0036] In this embodiment of the invention, for each suspected defect region, images of the corresponding locations of the suspected defect region in the original image, template image, and binarized mask image are extracted and merged into a three-channel image. This three-channel image is then input into a deep learning-based defect detection model. Feature extraction is performed on the corresponding locations of the suspected defect region in the three channels of the original image, template image, and binarized mask image using convolution parameter sharing. This feature extraction method allows the model to learn the differences between the corresponding locations of the suspected defect region in the original image and the corresponding locations of the suspected defect region in the template image. Furthermore, by adding a binarized mask image, the model's feature extraction of defect locations becomes more targeted. This method, by adding template and binarized mask images to the model, is equivalent to adding an "attention mechanism," enabling the model to more effectively focus on the features of the defect region. This method uses the suspected defect region (which is contained in the binarized mask image), the original image, and the template image to enable the model to extract defect features, improving the model's accuracy in defect detection and achieving high detection efficiency.

[0037] In this embodiment of the invention, each pixel in the template image has a corresponding preset grayscale deviation threshold; step 2 includes the following steps:

[0038] Step 2.1: For each pixel in the original image, determine whether the pixel is a pixel with grayscale out-of-range based on the preset grayscale out-of-range threshold corresponding to the pixel value of the pixel and the corresponding pixel in the template image.

[0039] Step 2.2: Set the pixel value of the corresponding pixel in the pre-constructed grayscale image to 255 for each grayscale out-of-range pixel, and set the pixel value of the pixel in the grayscale image that does not correspond to a grayscale out-of-range pixel to 0, to obtain the binarized mask image.

[0040] The embodiments of the present invention generate a binary mask image based on the pixel values ​​of each pixel in the original image and the preset grayscale deviation threshold, which can remove noise in the original image and highlight the contour of the target.

[0041] Specifically, the preset grayscale deviation threshold includes a first grayscale deviation threshold and a second grayscale deviation threshold, wherein the first grayscale deviation threshold is greater than the second grayscale deviation threshold.

[0042] Step 2.1 includes the following steps: For each pixel in the original image, if the pixel value of the pixel is greater than the first grayscale out-of-range threshold corresponding to the pixel at the corresponding position in the template image or less than the second grayscale out-of-range threshold corresponding to the pixel at the corresponding position in the template image, the pixel is a grayscale out-of-range pixel.

[0043] In this embodiment of the invention, step 4 includes the following steps: for each suspected defect region, extract the image of the corresponding position region of the suspected defect region in the original image and use it as the upper layer image; extract the image of the corresponding position region of the suspected defect region in the template image and use it as the middle layer image; extract the image of the corresponding position region of the suspected defect region in the binarized mask image and use it as the lower layer image; and fuse the upper layer image, the middle layer image, and the lower layer image in a three-channel manner from top to bottom to obtain the three-channel image.

[0044] like Figure 2 As shown, for each suspected defect region, the upper, middle and lower layer images corresponding to the suspected defect region are merged in order from top to bottom to obtain a three-channel image corresponding to the suspected defect region. After the three-channel image is input into the defect detection model, the detection result of the suspected defect region is determined by the defect detection model, that is, whether the suspected defect region has a defect.

[0045] This invention improves the accuracy of defect detection by extracting images of the corresponding locations of suspected defect areas in the original image, template image, and binarized mask image, merging them into a three-channel image, and then using a deep learning-based defect detection model to perform confidence detection on the three-channel image.

[0046] The defect detection model is obtained through the following steps:

[0047] Obtain the original training set of the semiconductor chip, the original training set including multiple original training images;

[0048] For each of the original training images, the original training image is compared pixel by pixel with the template image of the semiconductor chip to obtain the binarized mask image corresponding to the original training image;

[0049] For each of the original training images, connected component analysis is performed on the binarized mask image corresponding to the original training image to obtain at least one suspected defect region corresponding to the original training image.

[0050] For each of the original training images, at least one target training image corresponding to the original training image is obtained based on each suspected defect region corresponding to the original training image, the template image, and the corresponding binarized mask image.

[0051] Based on the target training images corresponding to each of the original training images, a target training set is obtained;

[0052] The defect detection model is obtained by training a deep neural network using the target training set.

[0053] The template image is a standard, flawless reference image. During the neural network convolution process, for each suspected defect region, images of the corresponding locations in the original training image, template image, and binarized mask image are extracted and merged into a three-channel image (i.e., the target training image). This three-channel image is then input into a deep learning-based defect detection model. Feature extraction is performed on the corresponding locations of the suspected defect regions in the three channels of the original training image, template image, and binarized mask image using convolution parameter sharing. This method allows the model to learn the differences between the suspected defect region and the template image. Often, the complex textures of the template image and the suspected defect region are the same, with differences only in the defect location. This allows the model to better focus on the differing regions between the two images during training, thereby improving the model's accuracy and robustness. Furthermore, adding the corresponding binarized mask image provides more targeted assistance to the model in extracting features at the defect location. This method, by adding the template image and the binarized mask image to the model, is equivalent to adding an "attention mechanism," enabling the model to more effectively focus on the features of the defect region.

[0054] In this embodiment of the invention, the defect detection model includes an input layer, a first convolutional layer, a first pooling layer, multiple second convolutional layers, multiple third convolutional layers, a first fully connected layer, a second fully connected layer, and an output layer connected in sequence.

[0055] For each suspected defect region, the detection result of the suspected defect region is obtained through the following steps:

[0056] The first feature map is obtained by extracting local features from the three-channel image through the first convolutional layer.

[0057] The first feature map is downsampled using the first pooling layer to obtain the second feature map;

[0058] Based on the second feature map, a third feature map is obtained through each of the second convolutional layers;

[0059] The third feature map is downsampled using the second pooling layer to obtain the fourth feature map;

[0060] Based on the fourth feature map, the fifth feature map is obtained through each of the third convolutional layers;

[0061] The fifth feature map is reduced in dimensionality using the first fully connected layer to obtain the first feature vector;

[0062] Based on the first feature vector, a second feature vector is obtained through the second fully connected layer;

[0063] Based on the second feature vector, the output layer outputs the category to obtain the detection result corresponding to the three-channel image.

[0064] In embodiments of the present invention, such as Figure 3 As shown, the defect detection model has a total of 11 layers, which are connected in sequence as follows: Input layer, first convolutional layer C1, first pooling layer S1, second convolutional layer C2, second convolutional layer C3, second pooling layer S2, third convolutional layer C4, third convolutional layer C5, first fully connected layer FC1, second fully connected layer FC2, and output layer Output layer. Taking an input image of size 128×128 as an example, the relevant parameters of each convolutional layer and each pooling layer in the defect detection model, the size of the input image / feature map, and the size of the output feature map are shown in Table 1 below.

[0065] Table 1 - Relevant information for each convolutional layer and each pooling layer

[0066]

[0067]

[0068] Specifically, for a single-channel input image of size 128×128, the first convolutional layer C1(32,3,3) performs a 3×3 convolution operation on the input image to extract local feature information, resulting in a feature map of size 128×128 with 32 channels. The output vector of this layer is 32×128×128. The first pooling layer S1(2,2,2) reduces the width and height of its input to 0.5 times their original values. This reduction in width and height also represents a reduction in the number of parameters, which is a downsampling process to prevent the model from overfitting due to an excessive number of parameters. The output vector of this layer is 32×64×64. The second convolutional layer C2(64,3,3) further reduces the input... The first convolutional layer C3(64,3,3) performs a 3×3 convolution operation on its input to extract local feature information, while increasing the number of channels to 64. This increase in channels means that richer local feature information, such as color, texture, and contour, can be extracted through convolution. The output vector of this layer is 64×64×64. The second convolutional layer C3(64,3,3) performs a 3×3 convolution operation on its input to extract local feature information. The width, height, and number of channels of the output remain unchanged, aiming to improve the non-linear mapping capability of the model. The output vector of this layer is also 64×64×64. The second pooling layer S2(2,2,2) performs pooling downsampling on its input, further reducing the width and height of the input to 0. The first convolutional layer increases the number of channels by 5 times while keeping the number of channels constant and further reducing the number of parameters. At this point, the width and height of the feature map are both 32, and the number of channels is 64. The output vector of this layer is 64×32×32. The third convolutional layer C4(128,3,3) performs a 3×3 convolution operation on its input to extract local feature information. The width and height of the output remain unchanged, but the number of channels is increased to 128, resulting in richer extracted local feature information. The output vector of this layer is 128×32×32. The third convolutional layer C5(128,3,3) performs a 3×3 convolution operation on its input to extract local feature information. The width, height, and number of channels of the output remain unchanged. The purpose is to improve the nonlinear mapping capability of the model. The output of this layer... The vector is 128×32×32; the first fully connected layer FC1 includes 1024 neurons. The first fully connected layer FC1 maps its input (a three-dimensional feature map) into a one-dimensional vector of length 1024. All neurons in the first fully connected layer FC1 and the second fully connected layer FC2 are connected to all neurons in the output layer Output to quickly reduce the number of parameters in the feature map; the second fully connected layer FC2 has the same dimension as the first fully connected layer FC1, and its purpose is to increase the non-linear mapping capability of the fully connected layer. The output layer Output is used to map the 1024 one-dimensional vector into a two-dimensional vector.

[0069] In this embodiment of the invention, the output layer includes two neurons (i.e., the input image has two categories: defective and non-defective, with each neuron corresponding to one category). The output layer maps its input to a one-dimensional vector of length 2 (i.e., the values ​​of the two neurons in the output layer). The values ​​of the two one-dimensional vectors are determined by a Softmax regression function. For each one-dimensional vector, the value of the one-dimensional vector represents the probability estimate of the corresponding category by the Softmax regression function. The estimate is the confidence score of the input image under that category, and its value is between 0 and 1. For each input image, the detection result of the image is the category corresponding to the one-dimensional vector with the larger value among the two one-dimensional vectors.

[0070] In this embodiment of the invention, in order to reduce the computational cost of the model, the image input to the model is first bilinearly interpolated to a size of 128×128 before being fed into the model. Since convolution operations cannot be performed at the boundaries of the image, the size of the image after convolution is smaller than the size of the image before convolution. Therefore, this method adopts a padding strategy, that is, padding the boundaries of the feature map after each convolution operation with 0, so that the size of the feature map before and after convolution remains unchanged. In addition, each pooling layer uses max pooling to make the model more robust to small distortions in the input image.

[0071] In this embodiment of the invention, the cross-entropy function is selected as the loss function of the model. The cross-entropy function is used at the end of the model to evaluate the closeness between the model's output and the category label. The category label is divided into defective and non-defective. The category label of each image input to the model is known.

[0072] In this embodiment of the invention, the weight parameters are updated in batches of 50 samples (i.e., the target training images). In addition, this method introduces L2 regularization and adds a weight decay term to the loss function to penalize large weights during training and avoid overfitting.

[0073] During training, this method employs stochastic gradient descent to address the parameter training problem. Stochastic gradient descent is an optimization algorithm where the unoptimized loss function differs from the optimal loss. Gradient descent optimizes the loss function along the gradient descent direction to find the optimal parameters that minimize the loss function. Furthermore, this method incorporates momentum and learning rate decay into the stochastic gradient descent optimizer to address the instability caused by the consistent step size and direction of each descent in stochastic gradient descent without momentum and learning rate decay. This method incorporates momentum to simulate the inertia of a moving object, retaining a portion of the previously updated direction with each parameter update. This provides some ability to escape local optima, guiding the loss function optimization towards the global optimum. Learning rate decay is included because as the gradient optimizes, the gradient step size should decrease, rather than continuously optimizing with the same step size. Learning rate decay helps optimize the loss towards the global optimum. By automatically updating model parameters during training to continuously learn more features from the input three-channel images, this method effectively distinguishes real defects, ultimately improving the model's detection accuracy.

[0074] Optionally, during the training phase, this method employs a dropout strategy with a probability of 0.5 for the first and second fully connected layers. Dropout randomly disables some neurons in the fully connected layer with a certain probability, rendering them ineffective. A dropout of 0.5 means that 50% of the neurons are active, while the remaining neurons do not contribute to the results, helping to avoid overfitting. To accelerate the training process, this method also employs batch normalization after each convolutional layer, normalizing the output of the convolutional layer to a normal distribution with a mean of 0 and a variance of 1, thus addressing the issue of internal convolutional parameter distribution shift.

[0075] Based on a validation set containing 1000 defective images and a validation set containing 1000 defect-free images, the detection performance of our method was compared with that of two existing methods. The comparison results are shown in Table 2.

[0076] Table 2 - Comparison of detection performance between this method and existing methods

[0077]

[0078] Existing Method 1 involves directly training the defect detection model using a single original image to obtain the target model. The original image to be detected is then directly input into the target model for defect detection. The difference between Existing Method 1 and this method lies in the input to the model. Existing Method 2 uses SIFT features combined with an artificial neural network for defect detection. As shown in Table 2, this method achieves an overall detection accuracy of 99.8%, outperforming both Existing Method 1 and Existing Method 2. It is suitable for detecting defects in semiconductor chips, where background textures are complex and defect patterns are diverse and irregular.

[0079] Preferably, the method further includes the following steps:

[0080] For each of the original training images, data augmentation processing is performed on the original training images to obtain new training images;

[0081] Determine the target training image corresponding to each of the new training images;

[0082] The target training set is obtained based on the target training image corresponding to each of the original training images and the target training image corresponding to each of the new training images.

[0083] In this embodiment of the invention, the method for determining the target training image corresponding to the new training image is similar to the method for determining the target training image corresponding to the original training image, and will not be described in detail here.

[0084] In this embodiment of the invention, the original training set includes 20,000 images with real defects (i.e., defective samples) and 80,000 images without defects but with interference patterns (i.e., defect-free samples). Each original training image in the original training set is obtained by capturing images from a camera and then cutting out the defect locations. To prevent overfitting, this method employs a data augmentation strategy on the original training set. Specifically, for the original training set, data augmentation methods are used to make the number of target training images in the target training set N times the number of original training images in the original training set. The data augmentation methods include one or more combinations of image rotation (90 degrees, 180 degrees, and 270 degrees), image mirroring (horizontal and vertical mirroring), increasing image brightness, decreasing image brightness, and adding Gaussian noise to the image. Simultaneously, to alleviate the imbalance in the number of defective and non-defective samples, training sampling probabilities are set for defective samples and different types of defective samples. The total probability of all types of samples is 1. The training sampling probability of samples with a larger number of samples is lower than that of samples with a smaller number of samples, thereby increasing the weight of samples with a smaller number of samples and improving the recognition performance of these samples.

[0085] This method provides a high-efficiency and high-accuracy approach for detecting surface defects in semiconductor chips. It is applicable to a wide variety of semiconductor chips with complex background textures and diverse defect morphologies, meeting the demand for high-efficiency surface defect detection. Based on deep learning, this method trains the model using three-channel images, resulting in a robust defect detection model with high detection efficiency and accuracy. Furthermore, the model in this method does not require extensive threshold parameter control and periodic adjustments, and it exhibits good adaptability to changes in camera exposure and white balance over time, reducing the complexity of algorithm development.

[0086] Example 2

[0087] Based on the same principle as the semiconductor chip surface defect detection method described in Embodiment 1 above, this embodiment provides a semiconductor chip surface defect detection device, such as... Figure 4 As shown, it includes:

[0088] Image acquisition module, used to acquire raw images of semiconductor chips;

[0089] The first image processing module is used to compare the original image with the template image of the semiconductor chip pixel by pixel to obtain a binarized mask image with grayscale discrepancies; wherein the template image of the semiconductor chip is a standard reference image without defects;

[0090] The second image processing module is used to perform connected component analysis on the binarized mask image to obtain at least one suspected defect region.

[0091] The third image processing module is used to extract the image of the suspected defect region in the corresponding position area of ​​the original image, the template image and the binarized mask image for each suspected defect region, and merge them into a three-channel image;

[0092] The defect detection module is used to perform confidence detection on each of the three-channel images using a deep learning-based defect detection model, and to determine the detection result of each of the suspected defect regions, wherein the detection result is either defective or defect-free.

[0093] Each pixel in the template image has a corresponding preset grayscale deviation threshold;

[0094] The first image processing module includes:

[0095] The judgment unit is used to determine whether a pixel is a pixel with grayscale out-of-range based on the pixel value of the pixel and the preset grayscale out-of-range threshold corresponding to the pixel at the corresponding position in the template image for each pixel in the original image.

[0096] The adjustment unit is used to set the pixel value of the corresponding pixel in the pre-constructed grayscale image to 255 for each grayscale out-of-range pixel, and to set the pixel value of the pixel in the grayscale image that does not correspond to a grayscale out-of-range pixel to 0, so as to obtain the binarized mask image.

[0097] The preset grayscale deviation threshold includes a first grayscale deviation threshold and a second grayscale deviation threshold, wherein the first grayscale deviation threshold is greater than the second grayscale deviation threshold.

[0098] The judgment unit is specifically used for:

[0099] For each pixel in the original image, if the pixel value of the pixel is greater than the first grayscale out-of-range threshold corresponding to the corresponding pixel in the template image or less than the second grayscale out-of-range threshold corresponding to the corresponding pixel in the template image, the pixel is a grayscale out-of-range pixel.

[0100] The system further includes a model determination module, which is used to determine the defect detection model;

[0101] The model determination module is specifically used for:

[0102] Obtain the original training set of the semiconductor chip, the original training set including multiple original training images;

[0103] For each of the original training images, the original training image is compared pixel by pixel with the template image of the semiconductor chip to obtain the binarized mask image corresponding to the original training image;

[0104] For each of the original training images, connected component analysis is performed on the binarized mask image corresponding to the original training image to obtain at least one suspected defect region corresponding to the original training image.

[0105] For each of the original training images, at least one target training image corresponding to the original training image is obtained based on each suspected defect region corresponding to the original training image, the template image, and the corresponding binarized mask image.

[0106] Based on the target training images corresponding to each of the original training images, a target training set is obtained;

[0107] The defect detection model is obtained by training a deep neural network using the target training set.

[0108] Preferably, the system further includes a training data augmentation module, which is used for:

[0109] For each of the original training images, data augmentation processing is performed on the original training images to obtain new training images;

[0110] Determine the target training image corresponding to each of the new training images;

[0111] The target training set is obtained based on the target training image corresponding to each of the original training images and the target training image corresponding to each of the new training images.

[0112] Example 3

[0113] To address the aforementioned technical problems, this embodiment provides a semiconductor chip surface defect detection device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the semiconductor chip surface defect detection method as described in Embodiment 1.

[0114] Example 4

[0115] To address the aforementioned technical problems, this embodiment provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the semiconductor chip surface defect detection method as described in Embodiment 1.

[0116] In the description of this invention, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0117] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," and "some examples" indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0118] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method of detecting surface defects of a semiconductor chip, characterized by, The method comprises the following steps: Step 1, obtaining an original image of a semiconductor chip; Step 2, performing pixel-by-pixel comparison between the original image and a template image of the semiconductor chip to obtain a binary mask image of gray level difference; wherein the template image of the semiconductor chip is a standard and defect-free reference image; Step 3, performing connected domain analysis on the binary mask image to obtain at least one suspected defect region; Step 4, for each suspected defect region, cutting out an image of a region corresponding to the suspected defect region in the original image as an upper layer image, cutting out an image of a region corresponding to the suspected defect region in the template image as a middle layer image, and cutting out an image of a region corresponding to the suspected defect region in the binary mask image as a lower layer image, and performing three-channel fusion on the upper layer image, the middle layer image and the lower layer image in a top-to-bottom order to obtain a three-channel image; Step 5, performing confidence detection on each three-channel image by a defect detection model based on deep learning to determine a detection result of each suspected defect region, wherein the detection result is defective or non-defective.

2. The method of claim 1, wherein, The step 2 comprises the following steps: Step 2.1, for each pixel point in the original image, determining whether the pixel point is a gray level difference pixel point according to a preset gray level difference threshold corresponding to a pixel value of the pixel point and a pixel value of a pixel point at a corresponding position in the template image; Step 2.2, setting a pixel value of a pixel point at a corresponding position in a gray level image pre-constructed as 255 for each gray level difference pixel point, and setting a pixel value of a pixel point in the gray level image not corresponding to a gray level difference pixel point as 0 to obtain the binary mask image.

3. The method of claim 2, wherein, The preset gray level difference threshold comprises a first gray level difference threshold and a second gray level difference threshold, and the first gray level difference threshold is greater than the second gray level difference threshold; The step 2.1 comprises the following steps: For each pixel point in the original image, if a pixel value of the pixel point is greater than a first gray level difference threshold corresponding to a pixel point at a corresponding position in the template image or less than a second gray level difference threshold corresponding to the pixel point at the corresponding position in the template image, the pixel point is a gray level difference pixel point.

4. The method of claim 1, wherein, The defect detection model is obtained by the following steps: Obtaining an original training set of the semiconductor chip, wherein the original training set comprises a plurality of original training images; For each original training image, performing pixel-by-pixel comparison between the original training image and a template image of the semiconductor chip to obtain a binary mask image corresponding to the original training image; For each original training image, performing connected domain analysis on the binary mask image corresponding to the original training image to obtain at least one suspected defect region corresponding to the original training image; For each original training image, obtaining at least one target training image corresponding to the original training image according to each suspected defect region corresponding to the original training image, the template image and the corresponding binary mask image; According to the target training image corresponding to each of the original training images, a target training set is obtained; The deep neural network is trained through the target training set, and the defect detection model is obtained.

5. The method of claim 4, wherein, The method further includes the following steps: For each of the original training images, data enhancement processing is performed on the original training image to obtain a new training image; Determine the target training image corresponding to each of the new training images; According to the target training image corresponding to each of the original training images, the target training image corresponding to each of the new training images, a target training set is obtained.

6. A semiconductor chip surface defect inspection apparatus characterized by comprising: It includes: An image acquisition module is configured to acquire an original image of a semiconductor chip; A first image processing module is configured to compare the original image with a template image of the semiconductor chip pixel by pixel to obtain a binary mask image of gray level out-of-tolerance; wherein the template image of the semiconductor chip is a standard and defect-free reference image; A second image processing module is configured to perform connected component analysis on the binary mask image to obtain at least one suspected defect region; A third image processing module is configured to, for each of the suspected defect regions, extract an image of a region corresponding to the suspected defect region in the original image as an upper image, extract an image of a region corresponding to the suspected defect region in the template image as a middle image, and extract an image of a region corresponding to the suspected defect region in the binary mask image as a lower image; and perform three-channel fusion on the upper image, the middle image, and the lower image in a top-to-bottom order to obtain a three-channel image; A defect detection module is configured to perform confidence detection on each of the three-channel images through a defect detection model based on deep learning to determine a detection result of each of the suspected defect regions, wherein the detection result is either defective or non-defective.

7. The apparatus of claim 6, wherein, The first image processing module includes: A judgment unit is configured to, for each pixel point in the original image, determine whether the pixel point is a gray level out-of-tolerance pixel point based on a pixel value of the pixel point and a preset gray level out-of-tolerance threshold corresponding to a pixel point at a corresponding position in the template image; An adjustment unit is configured to set a pixel value of a pixel point at a corresponding position in a pre-constructed gray image to 255 for each of the gray level out-of-tolerance pixel points, and set a pixel value of a pixel point in the gray image that does not correspond to a gray level out-of-tolerance pixel point to 0 to obtain the binary mask image.

8. The apparatus of claim 7, wherein, The preset gray level out-of-tolerance threshold includes a first gray level out-of-tolerance threshold and a second gray level out-of-tolerance threshold, and the first gray level out-of-tolerance threshold is greater than the second gray level out-of-tolerance threshold; The judgment unit is specifically configured to: For each pixel point in the original image, if a pixel value of the pixel point is greater than a first gray level out-of-tolerance threshold corresponding to a pixel point at a corresponding position in the template image or less than a second gray level out-of-tolerance threshold corresponding to a pixel point at a corresponding position in the template image, the pixel point is a gray level out-of-tolerance pixel point.

9. A semiconductor chip surface defect detection device, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the semiconductor chip surface defect detection method in any one of claims 1 to 5.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the semiconductor chip surface defect detection method in any one of claims 1 to 5.

Images