Wafer defect classification method, device and system, storage medium and program product
By using labeled sample sets and pseudo-label sample sets for semi-supervised training in wafer defect classification, adjusting the weight ratio of the loss function, the problem of inaccurate classification in optical detection equipment is solved, and the generalization ability and accuracy of the model are improved.
Patent Information
- Application Number
- CN202411900201.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-19
- Publication Date
- 2025-05-09
AI Technical Summary
In the wafer defect classification, existing optical detection equipment has inaccurate classification due to the difference between the sampling sample distribution space and the entire sample distribution space.
The labeled sample set and the pseudo-label sample set are used as semi-supervised training data to adjust the initial wafer defect classification model. By adjusting the weight ratio of the loss function, the impact of labeled samples and pseudo-label samples in model training is balanced.
The generalization ability and accuracy of the wafer defect classification model is improved, the utilization of unsampled samples is enhanced, the dependence on labeled samples is reduced, and the classification accuracy is improved.
Smart Images
Figure CN119963877A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of semiconductor manufacturing technology, and in particular to a wafer defect classification method, device, system, storage medium and program product. Background Art
[0002] After completing the wafer scan, the optical inspection equipment will obtain a large amount of defect data. Due to the limitations of optical imaging limits and detection algorithms, in addition to real defects, there will also be a certain number of false defects in the scan results. The defects need to be classified later to control the false defects in the scan results to a certain level.
[0003] The first step in defect classification for current optical inspection equipment is to sample a certain number of samples from a large number of samples for inspection, label the defect types of the samples, and then train a supervised classification algorithm based on the labeled data. Therefore, unsampled samples are not used, resulting in inaccurate classification of the trained model due to certain differences between the distribution space of the sampled samples and the distribution space of all samples. Summary of the invention
[0004] The present application provides a wafer defect classification method, device, system, storage medium and program product, which are used to provide a technical solution for combining labeled samples and unlabeled samples to perform semi-supervised classification model training to improve the accuracy of wafer defect classification.
[0005] In a first aspect, the present application provides a wafer defect classification method, comprising:
[0006] determining at least one defect image of a wafer to be classified;
[0007] Performing defect classification on the at least one defect image based on a preset wafer defect classification model to determine the defect type of the wafer to be classified;
[0008] The training process of the preset wafer defect classification model includes:
[0009] The labeled sample set and the pseudo-label sample set are used as semi-supervised training data to adjust the initial wafer defect classification model to obtain the preset wafer defect classification model; wherein the ratio of the weight of the loss function of the labeled samples in the preset wafer defect classification model to the weight of the loss function of the pseudo-label samples is determined according to the ratio of the number of pseudo-label samples in the pseudo-label sample set to the number of labeled samples in the labeled sample set.
[0010] In an optional implementation, the training process of the preset wafer defect classification model further includes:
[0011] Performing sample feature clustering on the wafer defect sample set, and sampling and labeling the wafer defect sample set after feature clustering to obtain a labeled sample set;
[0012] Based on the clustering information, label propagation is performed on the unlabeled wafer defect sample set after the feature clustering to generate a pseudo-label sample set.
[0013] In an optional implementation, the step of clustering the wafer defect sample set by sample features, and sampling and labeling the wafer defect sample set after the sample feature clustering, to obtain a labeled sample set includes:
[0014] Performing feature extraction on the wafer defect sample set, and converting each of the wafer defect samples into a corresponding N-dimensional feature vector;
[0015] Determine the number of cluster centers, and perform clustering and dimensionality reduction processing on the N-dimensional feature vector;
[0016] The wafer defect samples after clustering and dimensionality reduction are sampled and labeled to obtain the labeled sample set.
[0017] In an optional implementation, sampling and labeling the wafer defect samples after clustering and dimensionality reduction processing includes:
[0018] Sampling the wafer defect samples after the clustering and dimensionality reduction processing to obtain sampled samples;
[0019] Sending the sampled sample to a scanning electron microscope, and setting the sample category of the sampled sample according to the sample category determined by the scanning electron microscope;
[0020] The sampled samples are labeled according to the defect types of the sampled samples.
[0021] In an optional implementation, after extracting samples of the high-dimensional popular space surface from the wafer defect samples after clustering and dimensionality reduction processing to obtain sampling samples, the method further includes:
[0022] Reducing the sample dimensions to a visible dimension;
[0023] In the visible dimension, the distribution of the sampled samples in the clustering category is determined, and when the distribution does not meet the preset distribution, the wafer defect samples after the clustering and dimensionality reduction processing are resampled.
[0024] In an optional implementation, the training process of the initial wafer defect classification model includes:
[0025] The labeled sample set is used as supervised training data to train the deep learning pre-training model to obtain the initial wafer defect classification model.
[0026] In an optional implementation, the using the labeled sample set as supervised training data to train the deep learning pre-training model includes:
[0027] Determine the ratio of the training set and the validation set corresponding to each cluster category according to the distribution of the labeled sample set in each cluster category;
[0028] According to the ratio, the labeled sample set is divided into a training set and a validation set according to the clustering category;
[0029] Based on the training set and the validation set, the deep learning pre-training model is trained.
[0030] In an optional implementation, using the labeled sample set and the pseudo-labeled sample set as semi-supervised training data to adjust the initial wafer defect classification model includes:
[0031] Determine a loss function according to the weight of the loss function of the labeled sample and the weight of the loss function of the pseudo-labeled sample;
[0032] The initial wafer defect classification model is adjusted according to the loss function.
[0033] In an optional implementation, in the wafer defect sample set, the wafer defect sample is an image obtained by performing defect detection on a wafer image acquired by a wafer imaging device.
[0034] In a second aspect, the present application also provides a wafer defect classification device, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the program implements any one of the methods in the first aspect when executed by the processor.
[0035] In a third aspect, the present application further provides a wafer inspection system, comprising: a wafer imaging device and the wafer defect classification device in the second aspect;
[0036] The wafer imaging device is used to obtain wafer images of wafers to be classified;
[0037] The wafer defect classification device is specifically used for:
[0038] Performing defect detection on the wafer image to obtain at least one defect image of the wafer to be classified;
[0039] Defect classification is performed on the at least one defect image based on a preset wafer defect classification model to determine the defect type of the wafer to be classified.
[0040] In a possible implementation, the system further includes a scanning electron microscope, and the scanning electron microscope is connected to the wafer defect classification device;
[0041] The scanning electron microscope is used to receive the sampling sample and scan the wafer corresponding to the sampling sample to determine the defect category of the sampling sample.
[0042] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method described in any one of the first aspects is implemented.
[0043] In a fifth aspect, the present application further provides a computer program product, comprising a computer program, which implements any one of the methods in the first aspect when executed by a processor.
[0044] The technical solution provided by the present application uses a labeled sample set and a pseudo-labeled sample set as semi-supervised training data to adjust the initial wafer defect classification model. Compared with training the classification model using only a labeled sample set, more effective training data is added, which can improve the generalization ability and accuracy of the obtained preset wafer defect classification model. Therefore, the defect classification of wafers based on the preset wafer defect classification model can improve the accuracy of wafer classification.
[0045] Furthermore, the ratio of the weight of the loss function of labeled samples in the preset wafer defect classification model of the embodiment of the present application to the weight of the loss function of pseudo-label samples is determined according to the ratio of the number of pseudo-label samples in the pseudo-label sample set to the number of labeled samples in the labeled sample set. Based on this, the influence of labeled samples and pseudo-label samples in model training can be balanced, and the model can be effectively prevented from being overly dependent on a certain type of samples, thereby improving the accuracy of the classification model. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.
[0047] Figure 1 A flowchart of a wafer defect classification method provided in an embodiment of the present application;
[0048] Figure 2 A schematic diagram of sample distribution after clustering provided in an embodiment of the present application;
[0049] Figure 3 A schematic diagram of the distribution of a sampling sample in the overall sample provided in an embodiment of the present application;
[0050] Figure 4 A schematic diagram of the distribution of another sampling sample in the overall sample provided in an embodiment of the present application;
[0051] Figure 5 A schematic diagram of the distribution of all samples in corresponding clustering labels after clustering provided in an embodiment of the present application;
[0052] Figure 6 A schematic diagram of the distribution of samples in cluster labels after all samples after clustering are labeled provided in an embodiment of the present application;
[0053] Figure 7 A schematic diagram of the distribution of samples in cluster labels after cluster label propagation is performed on unlabeled samples provided in an embodiment of the present application;
[0054] Figure 8 A structural diagram of a wafer inspection system is provided for an embodiment of the present application;
[0055] Fig. 9 A structural diagram of a wafer defect classification device provided in an embodiment of the present application.
[0056] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION
[0057] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.
[0058] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those generally understood by those skilled in the art to which the present application belongs. The terms used herein in the specification of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. It is understood that the terms "first", "second", etc. used in the present application can be used to describe various information or data in this article, but these elements are not limited by these terms. These terms are only used to distinguish the first information from another information. For example, without departing from the scope of the present application, the first action information can be referred to as the second action information, and similarly, the second action information can be referred to as the first action information. Both the first action information and the second action information are action information, but they are not the same action information.
[0059] After completing the wafer scan, the optical inspection equipment will obtain a large amount of defect data. Due to the limitations of optical imaging limits and detection algorithms, in addition to real defects, there will also be a certain number of false defects in the scan results. The defects need to be classified later to control the false defects in the scan results to a certain level.
[0060] The first step in defect classification for current optical inspection equipment is to sample a certain number of samples from a large number of samples for inspection, label the defect types of the samples, and then train a supervised classification algorithm based on the labeled data. Therefore, unsampled samples are not used, resulting in inaccurate classification of the trained model due to certain differences between the distribution space of the sampled samples and the distribution space of all samples.
[0061] In response to the above problems, the present application proposes the following technical concept: using labeled sample sets and pseudo-labeled sample sets as semi-supervised training data to adjust the initial wafer defect classification model, thereby providing a technical solution that combines labeled samples and unlabeled samples to train a semi-supervised classification model to improve the accuracy of wafer defect classification.
[0062] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.
[0063] First, refer to Figure 1 , the embodiment of the present application provides a wafer defect classification method, which may include the following steps:
[0064] S101, determining at least one defect image of a wafer to be classified.
[0065] Optionally, the embodiment of the present application may use a wafer imaging device to acquire images of the wafers to be classified, wherein the wafer imaging device may include a bright field imaging device and a dark field imaging device.
[0066] In some examples, a wafer imaging device may include an image sensor and a light source, wherein the image sensor is used to acquire images of wafers to be classified; and the light source is used to provide light of a suitable wavelength to optimize the imaging effect of the image sensor.
[0067] Furthermore, the wafer imaging device may further include a collecting device, which is used to receive light and transmit the received light to the image sensor.
[0068] Afterwards, a defect detection algorithm may be used to process the image of the wafer to be classified to generate at least one defect image of the wafer to be classified.
[0069] Optionally, the above-mentioned defect detection algorithm may adopt any image processing algorithm that can perform wafer defect detection, and the embodiments of the present application do not impose any special limitation on this.
[0070] S102: Perform defect classification on the at least one defect image based on a preset wafer defect classification model to determine the defect type of the wafer to be classified.
[0071] The training process of the preset wafer defect classification model may include:
[0072] The labeled sample set and the pseudo-label sample set are used as semi-supervised training data to adjust the initial wafer defect classification model to obtain the preset wafer defect classification model; wherein the ratio of the weight of the loss function of the labeled samples in the preset wafer defect classification model to the weight of the loss function of the pseudo-label samples is determined according to the ratio of the number of pseudo-label samples in the pseudo-label sample set to the number of labeled samples in the labeled sample set.
[0073] It should be understood that in practical applications, obtaining labeled samples usually requires a lot of manpower and time, while pseudo-label samples can be automatically generated from unlabeled data. Therefore, by making reasonable use of pseudo-label samples, the generalization ability of the model can be enhanced under limited labeled sample conditions.
[0074] It should be understood that since the number of pseudo-label samples is usually much larger than that of labeled samples, by adjusting the weight ratio of the loss function, a large number of pseudo-label samples can be fully utilized, thereby accelerating the training process of the preset wafer defect classification model. Based on this, not only the training efficiency is improved, but also the dependence on labeled samples is reduced.
[0075] In different application scenarios, the ratio of pseudo-label samples to labeled samples may be different. The embodiment of the present application can dynamically adjust the weight ratio of the loss function to adapt to different sample distributions, so that the preset wafer defect classification model can maintain high performance in various scenarios.
[0076] Based on the above description, the embodiment of the present application uses a labeled sample set and a pseudo-labeled sample set as semi-supervised training data to adjust the initial wafer defect classification model. Compared with training the classification model using only a labeled sample set, more effective training data is added, which can improve the generalization ability and accuracy of the preset wafer defect classification model. Therefore, the defect classification of wafers based on the preset wafer defect classification model can improve the accuracy of wafer classification.
[0077] Furthermore, the ratio of the weight of the loss function of labeled samples in the preset wafer defect classification model of the embodiment of the present application to the weight of the loss function of pseudo-label samples is determined according to the ratio of the number of pseudo-label samples in the pseudo-label sample set to the number of labeled samples in the labeled sample set. Based on this, the influence of labeled samples and pseudo-label samples in model training can be balanced, and the model can be effectively prevented from being overly dependent on a certain type of samples, thereby improving the accuracy of the classification model.
[0078] In a possible implementation, the training process of the preset wafer defect classification model further includes the following steps:
[0079] Step A1, performing sample feature clustering on a wafer defect sample set, and sampling and labeling the wafer defect sample set after feature clustering to obtain a labeled sample set.
[0080] Among them, the above-mentioned wafer defect sample set is an image sample set obtained by collecting original image data from wafer defect detection equipment and through a defect detection algorithm. The wafer defect sample set includes real defect images of the wafer and false defect images of the wafer.
[0081] It should be understood that different wafer defect samples will usually differ depending on the equipment and lens configuration. For example, for a single-lens device, a wafer defect sample usually contains only one original image (test image) and one or two reference images (reference images). If it is a three-lens device, a wafer defect sample will have three original images (test images) and 3 or 6 reference images (reference images). Alternatively, when each sample is tested online according to the defect detection algorithm, a series of sample attribute values will be calculated (the number of attributes ranges from dozens to hundreds).
[0082] Therefore, the above wafer defect sample set includes the original image of the wafer defect + the reference image, or the original image of the wafer defect + the reference image + the sample attribute value.
[0083] In the embodiment of the present application, the data of three lenses shot by a dark field imaging device is taken as an example to illustrate that a wafer defect sample includes 3 original images + 3 reference images + 120 attributes, and the resolution of the image can be 32*32.
[0084] The above sample feature clustering is used to cluster all samples in the wafer defect sample set. After the feature clustering, the wafer defect sample set after the feature clustering is sampled and labeled to obtain a labeled sample set.
[0085] In an optional embodiment, the above step A1 may include the following sub-steps:
[0086] Sub-step 11: extract features from the wafer defect sample set according to the deep learning pre-training model, and convert each of the wafer defect samples into a corresponding N-dimensional feature vector.
[0087] Optionally, a deep learning pre-trained model can be used to extract features from all samples in the wafer defect sample set, and each wafer defect sample can be converted into an N-dimensional high-dimensional vector.
[0088] Specifically, the deep learning pre-training model can be a classification model obtained by supervised training of a multi-classification model based on a large-scale image data set (millions of images). The deep learning pre-training model is used to perform convolution on each sample, and the features of each sample are extracted respectively, so as to convert each wafer defect sample into a corresponding N-dimensional feature vector.
[0089] For example, each sample includes 3 original images, 3 reference images and 120 attributes. Feature extraction is performed on the 3 original images and 3 reference images respectively, and the extracted feature dimension is 512 dimensions. The features of the 3 original images and 3 reference images are stacked in one-dimensional direction to form a 3072-dimensional feature vector, and the 120 attributes are also stacked into the feature vector, and finally the feature vector dimension is 3192 dimensions.
[0090] In the embodiment of the present application, feature extraction based on deep learning is carried out by learning a large number of image features in a wafer defect sample set. The features extracted by the deep learning algorithm can effectively overcome the noise introduced by inaccurate alignment of multiple channels, brightness changes, etc., making the extracted features more robust.
[0091] Sub-step 12, determining the number of cluster centers, and performing clustering and dimensionality reduction processing on the N-dimensional feature vector.
[0092] Optionally, the number of cluster centers can be set to k, and each N-dimensional feature vector can be clustered using a deep clustering technique to reduce each N-dimensional feature vector to an M-dimensional space. It should be understood that the embodiment of the present application can also use other clustering methods to cluster N-dimensional feature vectors, and there is no special limitation on this.
[0093] Deep clustering technology combines the advantages of deep learning and traditional clustering methods. It can automatically learn useful features in high-dimensional data and perform effective clustering, thereby improving the accuracy of clustering and classification. Therefore, the embodiment of the present application uses deep clustering technology to cluster the N-dimensional feature vector corresponding to each sample, which can improve the accuracy of sample classification.
[0094] As a specific embodiment, clustering the above N-dimensional feature vector using deep clustering technology may include: setting the number of cluster centers to k, and reducing the feature vector of each sample from N-dimensional to M-dimensional space. Exemplarily, when the N-dimensional feature vector is a 3192-dimensional feature vector, the number of cluster centers k=200 may be set, and the N-dimensional feature vector is mapped and transformed through a convolutional neural network architecture for image segmentation tasks, and the 3192-dimensional feature vector is converted to a 512-dimensional space.
[0095] Optionally, the number of cluster centers k can be set to 10 to 50 times the expected labeled categories to ensure that the samples in each cluster are sufficiently concentrated. For example, the k value is not less than 100.
[0096] Sub-step 13, sampling and labeling the wafer defect samples after clustering and dimensionality reduction processing to obtain the labeled sample set.
[0097] The embodiment of the present application samples wafer defect samples in the space after dimensionality reduction, and extracts samples where the wafer defect samples are concentrated on the surface of the high-dimensional popular space, so that the final sampled samples can be more focused on the interface, thereby improving the sampling coverage and sample separability, and ensuring that the final classification result has better noise resistance.
[0098] In the embodiment of the present application, a clustering method based on deep learning is adopted. Through deep clustering technology, effective dimensionality reduction of the sample space is achieved, effective sampling of different types of samples is achieved, effective coverage of the feature space by samples is ensured, and all types of defect samples are sampled.
[0099] The above sub-step 13 may include:
[0100] Sub-step 131 , sampling the wafer defect samples after the clustering and dimensionality reduction processing to obtain sampled samples.
[0101] For example, the sampling method may include geometric sampling, Monte Carlo, Markov Chain Monte Carlo, Particle-based Variational Inference, etc. The sampling basis may be distance, gradient, distribution, etc.
[0102] As a specific example, assume that the space of wafer defect samples after clustering and dimensionality reduction is an M-dimensional space. In the M-dimensional space, each wafer defect sample can be regarded as a point, and in the M-dimensional space, geometric sampling is used to sample samples in the wafer defect sample set, and the sampling basis can be distance.
[0103] Here, we take the geometric characteristic curvature of the point cloud as an example for sampling. There is a surface at any point in the point cloud. The greater the curvature of the surface, the greater the curvature of the arc, which means that the feature change in the area is greater, so more sampling points are required. The specific sampling process may include:
[0104] 1) First, the K domain of each point is calculated, and then the normal angle between the point and the domain point is calculated, so as to approximate the curvature effect and improve the calculation efficiency, because the greater the curvature, the greater the angle. It is worth noting that the points described here can be the above samples, and the sampling points can be sampling samples.
[0105] 2) Set an angle threshold. When the angle value of the point area is greater than the threshold, it is considered to be an area with obvious features, and the rest of the area is an inconspicuous area.
[0106] 3) Sample the areas with obvious features according to the curvature from large to small and the points that satisfy the requirement that the interval between two adjacent sampling points is greater than a given minimum interval.
[0107] For example, the number of cluster centers is 4, Figure 2 It shows the sample distribution after clustering.
[0108] If the sampling method is not appropriate, such as uniform sampling and small curvature, the distribution of possible sampling points in the overall sample is as follows: Figure 3 As shown, the sampling points are circled. It can be seen that the sampling points are concentrated in the cluster center rather than the edge of the cluster category.
[0109] If the sampling method is chosen appropriately, the sampling points will be distributed at the boundaries of samples in different clusters (where the curvature is larger), such as Figure 4 shown.
[0110] In this embodiment, considering the time consumption of manual sample annotation using SEM and the sample quantity requirement of training data, the number of samples can be set to be no less than 1000 and no more than 3000.
[0111] The embodiments of the present application can ensure that the sampled samples fully cover the entire sample space during the sampling phase, thereby solving the problem of poor model reasoning generalization performance of the classification model due to the inconsistency between the distribution of the training data itself and the actual sample space distribution.
[0112] Sub-step 132, sending the sample to a scanning electron microscope, and marking the sample according to the defect type determined by the scanning electron microscope.
[0113] In this sub-step, while sending the sampling sample to the scanning electron microscope, the wafer corresponding to the sampling sample also needs to be placed on the scanning electron microscope platform, and the scanning electron microscope is used to check the defect types at the corresponding positions of the wafer and the sampling sample, and the sample category of the sampling sample is set based on the defect type.
[0114] In this embodiment, according to the sampled samples set in the previous sub-step, for example, 2000 samples, the defect types of these samples are determined in turn using a scanning electron microscope, and the corresponding sampled samples are marked according to the defect types.
[0115] Step A2: Based on the clustering information, label propagation is performed on the unlabeled wafer defect sample set after the feature clustering to generate a pseudo-label sample set.
[0116] In an embodiment of the present application, for a set of wafer defect samples that are not labeled after feature clustering, a pseudo label is generated based on clustering information.
[0117] Clustering information is used for label propagation of unlabeled wafer defect samples to quickly generate a large number of pseudo labels for unlabeled samples. Different clusters may be merged into the same labeled class.
[0118] Exemplarily, the cluster labels are first updated to the labels of the samples in the corresponding cluster. Figure 5 As shown in , if the number of clusters at the beginning is 4 (the samples encircled by circles represent sampled samples, and the others are not sampled samples); Figure 6 As shown, the sampled samples are labeled to obtain the classification label G, where clusters C2 and C4 are merged into the same class G2; Figure 7 As shown, finally, the unlabeled samples of the same class are propagated using the labeled samples of this class to obtain pseudo labels.
[0119] The above content describes how to divide the wafer defect sample set into a labeled sample set and a pseudo-labeled sample set. In order to improve the accuracy of model training, the training process of the preset wafer defect classification model in the embodiment of the present application may also include the following steps:
[0120] Step B1, reducing the dimension of the sampled samples to a visible dimension.
[0121] This step can be implemented after extracting samples on the surface of the high-dimensional popular space from the wafer defect samples after clustering and dimensionality reduction processing to obtain sampling samples.
[0122] The above-mentioned dimension can be 2D or 3D, that is, the dimension observable by human eyes. The sampling samples are reduced to the visible dimension to realize the visualization of the distribution of the sampled samples and clustered samples, so as to more intuitively understand the distribution of the sampled samples and clustered samples.
[0123] In this embodiment, it is assumed that the M-dimensional space of the sampled samples is 512-dimensional. At this time, visualization cannot be achieved and dimensionality reduction processing is required. Optionally, principal component analysis (PCA), uniform manifold approximation and projection (Uniform Manifold Approximation and Projection for Dimension Reduction, UMAP) and other methods can be used for dimensionality reduction. For example, UMAP with better nonlinear mapping effect is used as a dimensionality reduction algorithm to reduce the sample from 512 dimensions to 3-dimensional space.
[0124] Step B2, within the visible dimension, determining the distribution of the sampled samples in the clustering category, and when the distribution does not meet the preset distribution, re-sampling the wafer defect samples after the clustering and dimensionality reduction processing.
[0125] Among them, the above-mentioned preset distribution situation can be that most of the sampled samples are distributed at the edge of the cluster category. If most of the sampled samples are distributed inside the cluster category, it means that the previously set sampling parameter settings are unreasonable and need to be iteratively modified. At this time, jump to sub-step 131 to continue to modify the sampling settings. If most of the sampled samples are distributed at the edge of the cluster category, it means that the sampled samples are key supporting sample points and are suitable sampling points.
[0126] The following section is used to describe the training process of the initial wafer defect classification model in the embodiment of the present application.
[0127] Optionally, the training process of the initial wafer defect classification model may include:
[0128] The labeled sample set is used as supervised training data to train the deep learning pre-training model to obtain the initial wafer defect classification model.
[0129] In this embodiment, the labeled sample set can be sampled in a certain ratio to obtain a training set and a validation set, and the learning rate, the maximum number of iterations, the convergence conditions for early termination, etc. can be set.
[0130] This embodiment can also perform image enhancement on the images in the labeled sample set, which may specifically include brightness enhancement, rotation, translation, blurring, stretching, salt and pepper noise, etc.
[0131] When the image in this embodiment is a three-channel dark field image, the image of each channel is enhanced by translation to simulate the possible alignment errors of different channels through inconsistency. The learning rate can be set to a dynamic learning rate, with an initial rate of 0.05 and a decrease of 10% every 10 iterations. The maximum number of iterations can be set to 50,000 times.
[0132] Resnet50 can be used as the backbone network for feature extraction of the above model. The extracted feature vector is 1024-dimensional. The 1024-dimensional feature vector is aggregated through a 3-layer fully connected network and finally an 8-dimensional feature vector is obtained through softmax. The category of the sample is determined according to the category corresponding to the dimension of the maximum normalized value.
[0133] It should be understood that the feature extraction backbone network can also be set to other suitable networks. For example, when speed is prioritized, a small network such as mobileNet v3 can be considered; if you want to further improve the classification accuracy, you can consider a network based on the vision transformer (ViT) structure.
[0134] In this embodiment, the loss function of the above model can use a cross entropy loss function.
[0135] Cross entropy loss definition:
[0136]
[0137] Among them, N represents the total number of labeled samples, G represents the type of label (the number of classification categories), and M represents the number of samples in each category.
[0138] In this embodiment, the training model can be checked for the classification F1-score, recall, accuracy and other indicators of the training set and the validation set to determine whether the trained model has converged, whether there are problems such as overfitting or underfitting. If there are related problems, the training model can be improved by adjusting the learning rate, number of iterations, image enhancement ratio, data imbalance categories, imbalance ratio and other parameters to obtain a better classification model.
[0139] Among them, F1-score is the harmonic mean of precision and recall. Precision refers to the ratio of correctly classified positive samples to the number of predicted positive samples. Recall refers to the ratio of correctly classified positive samples to the actual number of positive samples. Accuracy refers to the accuracy, which refers to the ratio of the number of samples correctly predicted by the model to the total number of samples.
[0140] More specifically, the training process of the initial wafer defect classification model may include the following sub-steps:
[0141] Sub-step C1, according to the distribution of the labeled sample set in each cluster category, determining the ratio of the training set and the validation set corresponding to each cluster category.
[0142] Optionally, for each cluster category, the ratio of the training set to the validation set can be set to 8:2. For cluster categories with a small number of labeled samples, the ratio of the training set to the validation set can be set to 5:5.
[0143] It should be understood that according to the distribution of the labeled sample set in each cluster category, the proportion of the training set and the validation set corresponding to each cluster category can be reasonably adjusted to ensure that the training set and the validation set are representative as a whole.
[0144] Sub-step C2: dividing the labeled sample set into a training set and a validation set according to the ratio and the clustering category.
[0145] For example, for category C1, if N1=100 and the ratio is 8:2, the training set contains 80 samples and the validation set contains 20 samples.
[0146] Sub-step C3: training the deep learning pre-training model based on the training set and the validation set.
[0147] The deep learning pre-trained model is trained using the training set. After each training cycle, the model performance is evaluated using the validation set and the model parameters are adjusted to improve accuracy.
[0148] The following section is used to describe how to use the labeled sample set and the pseudo-labeled sample set as semi-supervised training data to adjust the initial wafer defect classification model in the embodiment of the present application. Optionally, the following sub-steps may be included:
[0149] Sub-step D1, determining a loss function according to the weight of the loss function of the labeled sample set and the weight of the loss function of the pseudo-labeled sample set.
[0150] Prior to this, for samples in the labeled sample set and samples in the pseudo-label sample set, the ratios of the training set and the validation set were set according to the clustering categories.
[0151] For example, the labeled sample set includes 2000 samples and the clustering categories are 8. The ratio of the training set to the validation set in the labeled sample set is set to 8:2.
[0152] The pseudo-label sample set includes 18,000 samples, with 8 clustering categories. For the pseudo-label sample set, the ratio of the training set to the validation set can be considered based on the amount of wafer defect sample set data and the training time. The ratio can be from 8:2 to 5:5. Due to the large amount of pseudo-label data, the ratio of the training set to the validation set can be set to 5:5.
[0153] In this embodiment, the ratio of the weight of the loss function of the labeled samples in the preset wafer defect classification model to the weight of the loss function of the pseudo-label samples is determined according to the ratio of the number of pseudo-label samples in the pseudo-label sample set to the number of labeled samples in the labeled sample set.
[0154] For example, the ratio of the weight of the loss function of the labeled sample to the weight of the loss function of the pseudo-labeled sample is equal to the ratio of the number of pseudo-labeled samples in the pseudo-labeled sample set to the number of labeled samples in the labeled sample set.
[0155] Based on this, the loss function in this embodiment is:
[0156] L=9*L ce-label +L ce-unLabel
[0157] Among them, L ce-label is the loss function of labeled samples, L ce-unLabel is the loss function of the pseudo-label samples.
[0158] Sub-step D2: adjusting the initial wafer defect classification model according to the loss function.
[0159] Afterwards, you can also check the classification F1-score, recall, accuracy and other parameter labels of the training set and validation set to determine whether the trained model has converged, whether there are problems such as overfitting or underfitting. If the model has the above problems, you can readjust the learning rate, number of iterations, image enhancement ratio, data imbalance categories, imbalance ratio and other parameters of the training model to improve the training model and obtain a better classification model.
[0160] Based on the above description of the training process of the preset wafer defect classification model, in the sampling stage, the embodiment of the present application supports iterative setting of sampling parameters through feature clustering and dimensionality reduction visualization to achieve coverage of the full sample space by sampling samples.
[0161] Furthermore, the clustering method of the embodiment of the present application, through the feature extraction method of deep learning, can not only better extract high-dimensional features but also is offline calculation, which has higher online calculation efficiency than the semi-supervised training stage; through offline clustering label propagation, it can avoid the introduction of erroneous pseudo-labels by online calculation noise, thereby improving label quality and final classification accuracy.
[0162] Finally, the model training process of the embodiment of the present application, compared to supervised training with only labeled data, uses more real samples on the same amount of artificial labeled data, fully explores the distribution of data, and improves model accuracy and model generalization ability. And the embodiment of the present application first conducts supervised training and then semi-supervised training, which can make the convergence of the semi-supervised training model easier and improve the success rate of model training convergence. And the present application also generates pseudo-labels by combining unsupervised clustering, compared to using only valid labeled data for classification training, adding more valid training data, fully exploring data potential, and improving model generalization ability; compared to directly using a small number of label training model prediction results as pseudo-label methods, it can effectively reduce sample noise interference and improve pseudo-label quality.
[0163] In a second aspect, the present application also provides a wafer defect classification device, Fig. 9 This is a schematic diagram of the structure of a wafer defect classification device provided in an embodiment of the present application. Fig. 9 As shown, the wafer defect classification device may include: a transceiver 121 , a processor 122 , and a memory 123 .
[0164] The processor 122 executes the computer execution instructions stored in the memory, so that the processor 122 executes the scheme in the above embodiment. The processor 122 can be a general-purpose processor, including a central processing unit CPU, a network processor (network processor, NP), etc.; it can also be a digital signal processor DSP, an application-specific integrated circuit ASIC, a field programmable gate array FPGA or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.
[0165] The memory 123 is connected to the processor 122 via a system bus and completes communication between them. The memory 123 is used to store computer program instructions.
[0166] The transceiver 121 may be used to obtain tasks to be executed and configuration information of the tasks to be executed.
[0167] The system bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The system bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. The transceiver is used to realize the communication between the database access device and other computers (such as clients, read-write libraries, and read-only libraries). The memory may include random access memory (RAM) and may also include non-volatile memory.
[0168] An embodiment of the present application also provides a chip for running instructions, which is used to execute the technical solution of the wafer defect classification method in the above embodiment.
[0169] Thirdly, refer to Figure 8 The present application also provides a wafer inspection system, including: a wafer imaging device and the wafer defect classification device in the second aspect.
[0170] The wafer imaging device is used to obtain wafer images of wafers to be classified.
[0171] Among them, the wafer imaging equipment may include a bright field imaging equipment and a dark field imaging equipment.
[0172] In some examples, a wafer imaging device may include an image sensor and a light source, wherein the image sensor is used to acquire images of wafers to be classified; and the light source is used to provide light of a suitable wavelength to optimize the imaging effect of the image sensor.
[0173] Furthermore, the wafer imaging device may further include a collecting device, which is used to receive light and transmit the received light to the image sensor.
[0174] The wafer defect classification device is specifically used for:
[0175] Defect detection is performed on the wafer image to obtain at least one defect image of the wafer to be classified; defect classification is performed on the at least one defect image based on a preset wafer defect classification model to determine the defect type of the wafer to be classified.
[0176] In an embodiment of the present application, the wafer defect classification device may use a defect detection algorithm to process the image of the wafer to be classified to generate at least one defect image of the wafer to be classified. Then, a preset wafer defect classification model is used to classify the at least one defect image to determine the defect type of the wafer to be classified.
[0177] Optionally, the above-mentioned defect detection algorithm may adopt any image processing algorithm that can perform wafer defect detection, and the embodiments of the present application do not impose any special limitation on this.
[0178] It should be understood that the training process of the above-mentioned preset wafer defect classification model has been described in detail in the first aspect and will not be repeated here.
[0179] Furthermore, the system further comprises a scanning electron microscope, and the scanning electron microscope is connected to the wafer defect classification device;
[0180] The scanning electron microscope is used to receive the sampling sample and scan the wafer corresponding to the sampling sample to determine the defect category of the sampling sample.
[0181] When using a scanning electron microscope to scan the wafer corresponding to the sampling sample, it is also necessary to place the wafer corresponding to the sampling sample on the table of the scanning electron microscope, use the scanning electron microscope to check the defect types at the corresponding positions of the wafer and the sampling sample, and set the sample category of the sampling sample based on the defect type.
[0182] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, in which computer instructions are stored. When the computer instructions are executed on a computer, the computer executes the technical solution of the wafer defect classification method of the above embodiment.
[0183] In a fifth aspect, an embodiment of the present application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium. When at least one processor executes the computer program, it can implement the technical solution of the wafer defect classification method in the above embodiment.
[0184] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules is only a logical function division, and there may be other division methods in actual implementation, such as multiple modules can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.
[0185] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to implement the solution of this embodiment.
[0186] In addition, each functional module in each embodiment of the present application can be integrated into one processing unit, or each module can exist physically separately, or two or more modules can be integrated into one unit. The above-mentioned module-composed unit can be implemented in the form of hardware or in the form of hardware plus software functional units.
[0187] The above-mentioned integrated module implemented in the form of a software function module can be stored in a computer-readable storage medium. The above-mentioned software function module is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform some steps of the methods of various embodiments of the present application.
[0188] It should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the invention may be directly implemented as being executed by a hardware processor, or may be implemented by a combination of hardware and software modules in the processor.
[0189] The memory may include a high-speed RAM memory, and may also include a non-volatile storage NVM, such as at least one disk memory, and may also be a USB flash drive, a mobile hard disk, a read-only memory, a magnetic disk or an optical disk, etc.
[0190] The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of the present application is not limited to only one bus or one type of bus.
[0191] The above storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The storage medium can be any available medium that can be accessed by a general or special purpose computer.
[0192] An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the storage medium can also exist as discrete components in an electronic control unit or a main control device.
[0193] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.
[0194] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.
Claims
1. A wafer defect classification method, characterized in that: The method comprises: determining at least one defect image of a wafer to be classified; Performing defect classification on the at least one defect image based on a preset wafer defect classification model to determine the defect type of the wafer to be classified; The training process of the preset wafer defect classification model includes: The labeled sample set and the pseudo-label sample set are used as semi-supervised training data to adjust the initial wafer defect classification model to obtain the preset wafer defect classification model; wherein, the ratio of the weight of the loss function of the labeled samples in the preset wafer defect classification model to the weight of the loss function of the pseudo-label samples can be determined according to the ratio of the number of pseudo-label samples in the pseudo-label sample set to the number of labeled samples in the labeled sample set.
2. The method according to claim 1, characterized in that: The training process of the preset wafer defect classification model also includes: Performing sample feature clustering on the wafer defect sample set, and sampling and labeling the wafer defect sample set after feature clustering to obtain a labeled sample set; Based on the clustering information, label propagation is performed on the unlabeled wafer defect sample set after the feature clustering to generate a pseudo-label sample set.
3. The method according to claim 2, characterized in that The wafer defect sample set is clustered by sample features, and the wafer defect sample set after the sample feature clustering is sampled and labeled to obtain a labeled sample set including: Extract features from the wafer defect sample set according to the deep learning pre-training model, and convert each of the wafer defect samples into a corresponding N-dimensional feature vector; Determine the number of cluster centers, and perform clustering and dimensionality reduction processing on the N-dimensional feature vector; The samples in the wafer defect sample set after clustering and dimensionality reduction are sampled and labeled to obtain the labeled sample set.
4. The method according to claim 3, characterized in that: Sampling and labeling of wafer defect samples after clustering and dimensionality reduction include: Sampling the wafer defect samples after the clustering and dimensionality reduction processing to obtain sampled samples; The sample is sent to a scanning electron microscope, and the sample is labeled according to the defect type determined by the scanning electron microscope.
5. The method according to claim 4, characterized in that: After extracting samples on the surface of the high-dimensional popular space from the wafer defect samples after clustering and dimensionality reduction processing to obtain sampling samples, the method further includes: Reducing the sample dimensions to a visible dimension; In the visible dimension, the distribution of the sampled samples in the clustering category is determined, and when the distribution does not meet the preset distribution, the wafer defect samples after the clustering and dimensionality reduction processing are resampled.
6. The method according to any one of claims 1 to 5, characterized in that: The training process of the initial wafer defect classification model includes: The labeled sample set is used as supervised training data to train the deep learning pre-training model to obtain the initial wafer defect classification model.
7. The method according to claim 6, characterized in that Using the labeled sample set as supervised training data to train the deep learning pre-training model includes: Determine the ratio of the training set and the validation set corresponding to each cluster category according to the distribution of the labeled sample set in each cluster category; According to the ratio, the labeled sample set is divided into a training set and a validation set according to the clustering category; Based on the training set and the validation set, the deep learning pre-training model is trained.
8. The method according to any one of claims 1 to 5, characterized in that: Using the labeled sample set and the pseudo-label sample set as semi-supervised training data, adjusting the initial wafer defect classification model includes: Determine a loss function according to the weight of the loss function of the labeled sample set and the weight of the loss function of the pseudo-labeled sample set; The initial wafer defect classification model is adjusted according to the loss function.
9. The method according to any one of claims 1 to 5, characterized in that: In the wafer defect sample set, the wafer defect sample is an image obtained by performing defect detection on a wafer image acquired by a wafer imaging device.
10. A wafer defect classification device, characterized in that: The invention comprises a processor, a memory and a computer program stored in the memory and executable on the processor, wherein the program implements the method according to any one of claims 1 to 9 when executed by the processor.
11. A wafer inspection system, characterized in that: include: Wafer imaging equipment and wafer defect classification equipment according to claim 10; The wafer imaging device is used to obtain wafer images of wafers to be classified; The wafer defect classification device is specifically used for: Performing defect detection on the wafer image to obtain at least one defect image of the wafer to be classified; Defect classification is performed on the at least one defect image based on a preset wafer defect classification model to determine the defect type of the wafer to be classified.
12. The wafer inspection system according to claim 11, characterized in that: The system further comprises a scanning electron microscope, wherein the scanning electron microscope is connected to the wafer defect classification device; The scanning electron microscope is used to receive the sampling samples sent by the wafer defect classification device, and scan the wafer corresponding to the sampling samples to determine the defect category of the sampling samples.
13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.
14. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 9 when being executed by a processor.