A one-shot based wafer defect detection method

Abstract

The present invention discloses a one-shot based wafer defect detection method, belonging to the technical field of object detection, which includes the following steps: input data, and use the preprocessed data as the network input; construct a twin network double-branch structure; train the model with a meta-learning strategy; and perform detection using a dynamic threshold. The present invention only needs one reference sample and a small number of defect samples, combined with data augmentation technology, to quickly extract sample features and complete model training, which reduces the dependence of the algorithm on large-scale labeled data while significantly increasing the detection efficiency; by dynamically adjusting the similarity threshold through the image sharpness value, compared with the traditional method of manually setting the threshold, the automation degree is higher, and the false detection rate and missed detection rate can be effectively reduced; and through the meta-learning strategy, when encountering a new task, only one internal loop is needed to fine-tune the model to quickly adapt to the new task.

Description

Technical Field

[0001] The present invention belongs to the technical field of object detection, and particularly relates to a wafer defect detection method based on one-shot (one-time learning). Background Art

[0002] Wafer defect detection is a crucial step in the chip manufacturing process, directly affecting the yield and reliability of chips. In the actual production process, wafer defect detection algorithms face multiple technical challenges. First, the accuracy and robustness of the detection algorithm are crucial. Since the sizes of wafer defects are tiny, a high-precision algorithm is required to ensure a high recall rate and a low false alarm rate. Second, there are a wide variety of wafer defects with complex manifestations and are greatly affected by factors such as process and materials. Traditional image processing methods are difficult to adapt to complex defects. Finally, due to the particularity and complexity of wafer defect detection, defect samples are scarce and difficult to obtain. Therefore, it is particularly important to design a defect detection algorithm with good adaptability under the condition of scarce samples.

[0003] The existing mainstream wafer surface defect detection methods can be roughly divided into traditional optical image processing methods and object detection algorithms based on deep learning. Traditional image processing methods include threshold segmentation, morphological processing, etc. Most of these methods rely on manually setting threshold parameters and are sensitive to environmental changes such as illumination. These problems result in their inability to adapt to complex background environments and complex defects and their inability to perform efficient automation. Among the object detection algorithms based on deep learning, the yolo series of algorithms are the most popular. These algorithms usually require a large amount of data training to optimize the model performance. However, due to the particularity and complexity of wafer defect detection, the same type of publicly available datasets are very scarce, and it is also very difficult to obtain a large number of defect samples for training in the actual production process. Summary of the Invention

[0004] To make up for the deficiencies of the existing technology, the present invention aims to provide a wafer defect detection method based on one-shot (one-time learning) to reduce the dependence of the algorithm on large-scale labeled data and complete the extraction of features with only a small number of samples; and combines dynamic threshold optimization to enhance the model's ability to capture tiny defects and reduce the false detection rate of noise. In addition, a meta-learning method is introduced to enhance a small number of samples through synthetic data and improve the generalization ability of the model.

[0005] The technical problems solved by the present invention can be realized through the following specific technical solutions:

[0006] The described one-shot based wafer defect detection method includes the following steps:

[0007] Step 1, preprocess the data and use the preprocessed data as the network input;

[0008] Step 2: Construct a twin network dual-branch structure;

[0009] Step 3: Meta-learning strategy training model;

[0010] Step 4: Use dynamic threshold for detection.

[0011] Furthermore, in step 1, the data includes the following three categories:

[0012] The first category is the reference sample, which contains a single defect-free wafer image, which is an RGB (red, green and blue three-channel) image with a resolution of 2048×2048 pixels;

[0013] The second category is the sample to be tested, which includes a single image of the wafer to be tested from the same batch or the same process conditions as the reference sample;

[0014] The third category is meta-training task data, which contains a small number of historical defect samples and is used for meta-learning training.

[0015] Furthermore, in step 1, the preprocessing includes standardization, image quality enhancement and data enhancement. The standardization adopts grayscale normalization. The purpose of image quality enhancement is to improve the image signal-to-noise ratio. The image sharpness is calculated to determine whether to perform filtering and noise reduction on the image. The sharpness calculation formula is as follows:

[0016] (1)

[0017] In the formula, is the Laplace operator, N Refers to the total number of pixels, x and y is the pixel horizontal and vertical coordinates, I ( x , y ) represents the pixel gray value, S Indicates the sharpness value, S The smaller the value, the lower the sharpness of the image and the blurrier the image. S <0.2, use the following filter function to process the image:

[0018] (2)

[0019] In the formula, I Represents the original image, I deblur represents the denoised image, F and F -1 denote Fourier transform and inverse transform respectively, F(I) Indicates Fourier transform of the original image; K is the bias constant, K =0.01; H(u, v)is the point spread function, directly generated by the mathematical model, is H(u, v) the complex conjugate of

[0020] Furthermore, a data augmentation method that combines traditional image transformation and physical simulation of defects is adopted. On the one hand, by performing image transformation, the diversity of data is enhanced; on the other hand, based on the reference samples, physical simulation defects based on morphological operations are generated.

[0021] Furthermore, in step 2, the ResNet-18 network is selected as the default backbone network for the dual-branch structure. The input image is processed as a grayscale image in the preprocessing stage, and the number of input channels is changed to a single channel; the output is a feature vector for calculating the cosine similarity. The classification head of the original ResNet-18 is removed and replaced with global average pooling, and a fully connected layer outputs a 256-dimensional feature vector; a dropout layer is set before the fully connected layer and the dropout rate is set to 0.3. The cosine similarity calculation formula is:

[0022] (3)

[0023] where, f a 、 f b respectively represent the feature vectors of the reference image and the image to be detected, ||fa|| and ||fb|| respectively represent the L2 norms of the two vectors; the output range of the cosine similarity S is [-1, 1], and it is mapped to [0, 1] through Sigmoid (an activation function) as the final similarity score.

[0024] Furthermore, in step 3, the specific content of training the model with the meta-learning strategy is as follows:

[0025] ① Construct tasks. The support set of each task is an image without defects and an image with defects, and the query set is 5 images;

[0026] ② Randomly initialize the parameters of the backbone network ResNet-18. Sample 32 tasks in each batch, set the meta-learning rate to 0.001, and the internal learning rate of the task to 0.01;

[0027] ③ Perform internal loop training within each task to update the network parameters, and perform external loop training between different tasks;

[0028] ④ After each round of training, select difficult samples with similarity between [0.4, 0.6], increase their sampling weights in the next round of training, and terminate the training if the loss value does not decrease for 5 consecutive rounds.

[0029] Furthermore, the loss function adopted by the model trained with the meta-learning strategy is as follows:

[0030] (4)

[0031] Wherein, L contrastive and L triplet are the contrast loss and the triplet loss respectively. The specific expression of the contrast loss L contrastive is as follows:

[0032] (5)

[0033] Wherein, x a and x b are the input sample pairs, y i is the sample pair label, d ( x a , x b ) represents the feature distance of the sample pair, reflecting the similarity of the sample pair; m is the preset boundary value, and N is the total number of sample pairs; The expression of the triplet loss L triplet is as follows:

[0034] (6)

[0035] Wherein, x a represents the anchor sample, x p represents the positive sample, x n represents the negative sample, d ( a , b ) represents a and b the feature distance between; The function of the triplet loss L triplet is to make the distance between the reference sample and the positive sample less than the distance between the negative sample by at least one boundary value m , usually set between 0.2 and 1.0.

[0036] Furthermore, in the said step 4, the specific content of using the dynamic threshold for detection is as follows:

[0037] Input the image to be detected and the reference image into the network to extract feature vectors, and then calculate the cosine similarity. If the cosine similarity value is lower than the threshold, it indicates that a defect has been detected. The similarity threshold changes dynamically with the image sharpness value. The calculation formula for the sharpness value S is shown in Equation (1), and the similarity threshold T is determined by the following formula:

[0038] (7)

[0039] In the formula, T is the similarity threshold, T base is the basic threshold, generally taking T base = 0.3, k is the adjustment coefficient, generally taking k = 0.1, S norm refers to the sharpness value after normalization. When the image sharpness value is low, the image is blurred, and the similarity threshold should be set to a higher value to reduce noise interference; when the image sharpness value is high, the feature discrimination is strong, and the threshold can be reduced to reduce false detection.

[0040] Compared with the prior art, the present invention has the following advantages:

[0041] (1) Reducing the dependence of the algorithm on large-scale labeled data: The present invention only needs one reference sample and a small number of defect samples to cooperate with the data augmentation technology to quickly extract sample features and complete model training, significantly increasing the detection efficiency while reducing the dependence of the algorithm on large-scale labeled data.

[0042] (2) Adaptive similarity threshold: The present invention dynamically adjusts the similarity threshold through the image sharpness value. Compared with the traditional method of manually setting the threshold, it has a higher degree of automation and can effectively reduce the false detection rate and missed detection rate.

[0043] (3) Good adaptability to new tasks: Through the meta-learning strategy, when encountering a new task, the present invention only needs to perform an internal loop once to fine-tune the model to quickly adapt to the new task. Brief Description of the Drawings

[0044] Figure 1 is the flow chart of the steps of the detection method of the present invention;

[0045] Figure 2 is the schematic diagram of physically simulating different types of defects under the background of the same wafer of the present invention;

[0046] Figure 3 is the schematic diagram of the dual-branch structure of the twin network of the present invention. Detailed Embodiment

[0047] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0048] In the field of semiconductor manufacturing, improving the accuracy and efficiency of wafer defect detection algorithms is one of the key challenges that need to be addressed urgently. The application background of the present invention focuses on achieving precise and efficient detection of wafer defects under the conditions of scarce samples, or even single samples.

[0049] As Figure 1 shown, a wafer defect detection method based on one-shot (one-time learning) includes data input and preprocessing, siamese network construction, model training, and dynamic threshold inference. The present invention combines a siamese network with meta-learning to achieve rapid adaptation of small samples, and introduces a one-time learning method in wafer defect detection. The specific content is as follows:

[0050] (I) Data input and preprocessing.

[0051] The input data can be divided into the following three categories: The first category is reference samples. Ten defect-free wafer images of the same process batch provided by semiconductor manufacturers are selected, with an image resolution of 2048×2048. After grayscale conversion, they are used as a reference library. During actual detection, any one of them is selected as the reference image for the current task. The second category is samples to be detected, including 100 wafer images of the same batch, of which 50 are defective images with different types of defects and 50 are defect-free images. The third category is meta-training task data, including 30 defect samples of 5 categories collected historically, and the defect types cover scratches, particles, concavities and convexities, line width anomalies, and edge breakages.

[0052] Preprocessing includes three aspects: normalization processing, image quality enhancement, and data augmentation. The goal of normalization processing is to reduce the influence of equipment differences and shooting conditions and unify the input feature space. In the present invention, simple grayscale normalization processing is adopted. The purpose of image quality enhancement is to improve the signal-to-noise ratio of the image. In the present invention, it is judged whether to perform filtering and noise reduction processing on the image by calculating the image sharpness. The sharpness calculation formula is as follows:

[0053] (1)

[0054] In the formula, is the Laplacian operator, N refers to the total number of pixels, x and y are the horizontal and vertical coordinates of the pixel, I ( x , y ) represents the pixel gray value, S represents the sharpness value,S The smaller the value, the lower the sharpness of the picture and the blurrier the picture. If S < 0.2, the image is processed using the following filtering function:

[0055] (2)

[0056] In the formula, I represents the original picture, I deblur represents the picture after denoising, F and F -1 represent Fourier transform and inverse transform respectively, F(I) represents performing Fourier transform on the original picture; K is a bias constant, taking K = 0.01; H(u, v) is the point spread function, directly generated by a mathematical model, is H(u, v) the complex conjugate of.

[0057] To meet the model generalization requirements under the small sample condition, the present invention adopts a data augmentation method combining traditional image transformation and physical simulation defects. On the one hand, through image transformations such as randomly rotating (plus or minus 15 degrees), horizontally / vertically flipping, and scaling (0.8 - 1.2 times) the existing defect pictures; on the other hand, based on the reference samples, physical simulation defects based on morphological operations are generated, including scratches with random lengths (5 - 50 pixels) and particle contaminations with random diameters (3 - 20 pixels). The effects of the physical simulation defects are as Figure 2 shown, and 500 enhanced samples are generated.

[0058] (II) Hardware environment

[0059] The hardware configuration for the deep learning network training in the present invention example is: NVIDIA 4070SUPER GPU (12GB video memory), AMD 9700X CPU (6 cores and 12 threads), 32GB memory; the deep learning framework used is pytorch 2.6.0, CUDA 12.6.

[0060] (III) Siamese network training.

[0061] The Siamese network in the present invention is the most core feature extraction module, aiming to extract the feature vectors of the reference image and the image to be detected through a dual-branch structure, and then using the similarity comparison between the feature vectors and combining meta-learning to achieve small sample defect detection. The following is a detailed description:

[0062] As Figure 3The overall network structure is shown. Different from traditional deep learning networks, the network in the present invention has two input branches. Among them, the reference image branch inputs the defect-free wafer image, and the branch to be detected inputs the wafer image to be detected. It should be noted that the two branches must use the same convolutional neural network structure and share all parameters to ensure the consistency of feature extraction.

[0063] In this application, ResNet-18 (a residual network) is selected as the default backbone network for the dual-branch structure. The 18-layer structure has an appropriate depth, balancing the feature expression ability and computational efficiency, and is suitable for processing high-resolution wafer images; the residual connection alleviates the problem of gradient disappearance and improves the training stability, which is suitable for the problem of scarce samples. It is also necessary to adjust the input and output formats of the ResNet-18 network to adapt to the requirements of the wafer defect detection task. Specifically, the input image is processed as a grayscale image in the preprocessing stage, so the number of input channels is changed to a single channel; the output should be a feature vector convenient for calculating the cosine similarity, so the classification head of the original ResNet-18 is removed, replaced by global average pooling, and a fully connected layer is used to output a 256-dimensional feature vector. To improve the generalization ability of the model, a dropout layer is set before the fully connected layer and the dropout rate is set to 0.3.

[0064] The cosine similarity calculation formula of the present invention is:

[0065] (3)

[0066] In the formula, f a and f b represent the feature vectors of the reference image and the image to be detected respectively, ||fa|| and ||fb|| represent the L2 norms of the two vectors respectively; the output range of the cosine similarity S is [-1, 1], and it is mapped to [0, 1] through Sigmoid (an activation function) as the final similarity score.

[0067] (4) The training process of the meta-learning strategy.

[0068] The model training of the present invention combines the siamese network and the meta-learning method to achieve high-precision defect detection under small sample conditions through multi-task optimization. The following is a detailed step-by-step description of the training process:

[0069] (1) Construct tasks. The support set of each task is 1 defect-free image (positive sample) and 1 defective image (negative sample), and the query set is 5 images (3 positive samples + 2 negative samples) for evaluating the task generalization ability.

[0070] (2) The parameters of the backbone network (ResNet-18) were randomly initialized, 32 tasks were sampled in each batch, the meta-learning rate was set to 0.001, the intra-task learning rate was set to 0.01, and the training rounds were 200.

[0071] (3) Internal loop training is performed within each task to update network parameters, aiming to improve the model's ability to distinguish image similarities and to force the feature similarity values ​​of positive sample pairs within the same task to be high and the feature similarity of negative sample pairs to be low; external loop training is performed between different tasks to improve the model's ability to quickly adapt to new tasks and reduce the number of samples required for adaptation to new tasks.

[0072] (4) After each round of training, difficult samples with a similarity between [0.4, 0.6] are selected, and the sampling weight is increased by 50% in the next round of training; if the loss value does not decrease for five consecutive rounds, the training is terminated.

[0073] The loss function is a key factor in network training. The loss function used in this invention is for:

[0074] (4)

[0075] Where, L contrastive , L triplet They are contrast loss and triplet loss respectively. The specific expression of contrast loss is as follows:

[0076] (5)

[0077] In the formula, x a , x b is an input sample pair, y i is the sample pair label, d ( x a , x b ) represents the feature distance of the sample pair, reflecting the similarity of the sample pair; m is the preset boundary value, N is the total number of sample pairs; the role of contrast loss is to shorten the distance between similar sample pairs (positive sample pairs) and to extend the distance between dissimilar samples (negative sample pairs). L triplet The expression is as follows:

[0078] (6)

[0079] In the formula, x a represents the anchor point sample, x pIndicates a positive sample, x n Indicates a negative sample, d ( a , b ) indicates a the feature distance between b ; The triplet loss L triplet is to make the distance between the reference sample and the positive sample less than the distance to the negative sample by at least a margin value m , set to 0.5.

[0080] (V) Detection using a dynamic threshold.

[0081] After completing the network training, inference on a new task can be performed. In the inference stage, first adapt to the new task: input the defect-free reference image of the new process wafer and a defect sample, and then perform an internal update to fine-tune the model. After the new task adaptation is completed, the image to be detected and the reference image can be input into the network to extract feature vectors, and then the cosine similarity is calculated. If the cosine similarity value is lower than the threshold, it indicates that a defect has been detected. The similarity threshold changes dynamically with the image sharpness value. The calculation formula of the sharpness value S is shown in Equation (1), and the similarity threshold T is determined by the following formula:

[0082] (7)

[0083] In the formula, is the similarity threshold, is the base threshold, taking , is the adjustment coefficient, taking , refers to the normalized sharpness value. When the image sharpness value is low, the image is blurred, and the similarity threshold will be set to a higher value by formula (7) to reduce noise interference; when the image sharpness value is high, the feature discrimination is strong, and the threshold can be reduced to reduce false detection.

[0084] Compare the performance of the present invention with the traditional fixed threshold method (threshold 0.5) and YoloV5 (1000 defect samples) on 100 test samples, and the results are as follows:

[0085]

[0086] As can be seen from the above data, the present invention achieves an accuracy of 92.3 with only 30 original defect samples combined with data augmentation, reducing the labeled data by 97% compared to YoloV5. The dynamic threshold results in a false detection rate as low as 7.7% and a missed detection rate as low as 12.4%. In summary, through specific sample configuration, data augmentation parameters, training process and quantization experiments, the present invention verifies the feasibility and effectiveness of the technical solution.

[0087] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A one-shot wafer defect detection method, characterized in that: The following steps are involved: Step 1: preprocess the data and use the preprocessed data as network input; Step 2: Construct a twin network dual-branch structure; The ResNet-18 network is selected as the default backbone network of the dual-branch structure. The input image is processed into a grayscale image in the preprocessing stage, and the number of input channels is changed to a single channel. The output is the feature vector for calculating cosine similarity. The classification head of the original ResNet-18 is removed and replaced with global average pooling, and a 256-dimensional feature vector is output using a fully connected layer. A dropout layer is set before the fully connected layer and the dropout rate is set to 0.

3. The formula for calculating cosine similarity is: Among them, f a 、f b Represent the feature vectors of the reference image and the image to be detected, respectively. ||fa|| and ||fb|| represent the L2 norms of the two vectors respectively. The output range of cosine similarity S is [-1,1], which is mapped to [0,1] as the final similarity score through the activation function Sigmoid. Step 3: Meta-learning strategy training model; the specific contents are as follows: ① Construct tasks. The support set of each task is one defect-free image and one defective image, and the query set is 5 images. ② Randomly initialize the parameters of the backbone network ResNet-18, sample 32 tasks per batch, set the meta-learning rate to 0.001, and set the task internal learning rate to 0.01; ③ Perform internal loop training within each task to update network parameters, and perform external loop training between different tasks; ④ After each round of training, select difficult samples with similarity between [0.4, 0.6] and increase their sampling weight in the next round of training. If the loss value does not decrease for 5 consecutive rounds, terminate the training; Step 4: Use dynamic threshold for detection; input the image to be detected and the reference image into the network to extract the feature vector, and then calculate the cosine similarity. If the cosine similarity value is lower than the threshold, it means that a defect has been detected, where the similarity threshold changes dynamically with the image sharpness value.

2. The one-shot wafer defect detection method according to claim 1, characterized in that: In step 1, the data includes the following three categories: The first category is the reference sample, which contains a single defect-free wafer image, which is an RGB image with a resolution of 2048×2048 pixels; The second category is the sample to be tested, which includes a single image of the wafer to be tested from the same batch or the same process conditions as the reference sample; The third category is meta-training task data, which contains a small number of historical defect samples and is used for meta-learning training.

3. The one-shot wafer defect detection method according to claim 1, characterized in that: In step 1, the preprocessing includes standardization, image quality enhancement and data enhancement. The standardization adopts grayscale normalization. The purpose of image quality enhancement is to improve the image signal-to-noise ratio. The image sharpness is calculated to determine whether to perform filtering and noise reduction on the image. The sharpness calculation formula is as follows: In the formula, is the Laplacian operator, N refers to the total number of pixels, x and y are the horizontal and vertical coordinates of the pixels, I(x,y) represents the pixel grayscale value, and S represents the sharpness value; if S<0.2, the following filter function is used to process the image: In the formula, I represents the original image, I deblur Denotes the denoised image, F and F -1 They represent Fourier transform and inverse transform respectively, F(I) means Fourier transform of the original image; K is the bias constant, K=0.01; H(u,v) is the point spread function, which is directly generated by the mathematical model, and H*(u,v) is the complex conjugate of H(u,v).

4. The one-shot wafer defect detection method according to claim 3, characterized in that: A data enhancement method combining traditional image transformation and physical simulation defects is adopted. On the one hand, the diversity of data is enhanced by image transformation; on the other hand, physical simulation defects based on morphological operations are generated based on reference samples.

5. The one-shot wafer defect detection method according to claim 1, characterized in that: The loss function L used by the meta-learning strategy training model is: <h2 style=";text-align:left;direction:ltr">L=0.7L<h2 style=";text-align:left;direction:ltr"> contrastive <h2 style=";text-align:left;direction:ltr"> +0.3Lt<h2 style=";text-align:left;direction:ltr"> riplet <h2 style=";text-align:left;direction:ltr"> (4) Among them, L contrastive , L triplet They are contrast loss and triplet loss, contrast loss L contrastive The specific expression is as follows: In the formula, x a 、x b is the input sample pair, y i is the sample pair label, d(x a , x b ) represents the feature distance of the sample pair, reflecting the similarity of the sample pair; m is the preset boundary value, N is the total number of sample pairs; the triple loss L triplet The expression is as follows: In the formula, x a represents the anchor point sample, x p represents a positive sample, x n represents a negative sample, d(a,b) represents the feature distance between a and b; triple loss L triplet The role of is to make the distance between the reference sample and the positive sample smaller than the distance between the negative sample by at least a boundary value m, which is usually set between 0.2 and 1.

0.

6. The one-shot wafer defect detection method according to claim 3, characterized in that: In step 4, the calculation formula of the sharpness value S is shown in formula (1), and the similarity threshold T is determined by the following formula: T=T base -k·(1-S norm ) (7) In the formula, T is the similarity threshold, T base As the basic threshold, take T base =0.3; k is the adjustment coefficient, k=0.1, S norm Refers to the normalized sharpness value.

Images