Methods, systems, electronic equipment and storage media for detecting defects of semiconductor wafers
By performing feature extraction and similarity calculation on semiconductor wafer images, combined with deep learning feature nearest neighbor technology, the problem of data imbalance in semiconductor wafer defect category detection is solved, and the detection accuracy is improved.
Patent Information
- Application Number
- CN202411987838.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-30
- Publication Date
- 2025-04-08
Smart Images
Figure CN119784738A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data processing technology, and in particular to a method, system, electronic equipment and storage medium for semiconductor wafer defect detection. Background Art
[0002] The production process of semiconductor wafers is very complicated, and the resulting semiconductor wafers have many defect categories, and the characteristics of different defect categories are not very clear. This leads to poor results when quality inspectors conduct manual re-evaluation. Therefore, deep learning-based methods are usually used to automatically identify defect categories.
[0003] However, the defects generated in the actual production process of semiconductor wafers are often concentrated in a limited number of defect categories. If the deep learning model is trained according to the data generated in the actual production process of semiconductor wafers, the learning effect of the trained model for different defect categories will vary greatly, which will affect the model's detection results for semiconductor wafer defect categories. Summary of the invention
[0004] In view of this, an embodiment of the present invention provides a method, system, electronic device and storage medium for semiconductor wafer defect detection to improve the detection accuracy of the model for the defect category of the semiconductor wafer.
[0005] To achieve the above objectives, the embodiments of the present invention provide the following technical solutions:
[0006] A first aspect of an embodiment of the present invention discloses a method for detecting semiconductor wafer defects, the method comprising:
[0007] Acquire an image of a semiconductor wafer to be inspected;
[0008] Inputting the semiconductor wafer image into a defect detection model for feature extraction to obtain a feature vector of the semiconductor wafer image, wherein the defect detection model is obtained by training a deep learning model based on a training data stream determined by the probability of each defect category being sampled;
[0009] Determining the similarity between the feature vector of the semiconductor wafer image and the normalized average feature vector corresponding to each of the defect categories, the normalized average feature vector corresponding to each of the defect categories being calculated during the training process of the defect detection model;
[0010] Determine that the defect category corresponding to the highest similarity is the defect category detection result of the semiconductor wafer image.
[0011] Preferably, the process of training a defect detection model includes:
[0012] Determine a first probability vector based on a training data set, wherein the training data set includes wafer defect images of multiple defect categories, and the first probability vector includes the probability of each defect category being sampled;
[0013] Determining a first training data stream from the training data set according to the first probability vector;
[0014] The deep learning model is trained according to the first training data stream and a preset number of training rounds to obtain a defect detection model.
[0015] Preferably, the process of training a defect detection model includes:
[0016] Determine a second probability vector and a third probability vector based on a training data set, wherein the training data set includes wafer defect images of multiple defect categories, the second probability vector and the third probability vector include probabilities of each defect category being sampled, the probabilities of at least two defect categories being sampled in the second probability vector are different, and the probabilities of each defect category being sampled in the third probability vector are the same;
[0017] In the process of performing the tth round of iterative training on the deep learning model, based on the second probability vector, the third probability vector, the number of iterative training rounds t, and the preset number of training rounds, a fourth probability vector is determined, wherein the fourth probability vector includes the probability of each of the defect categories being sampled;
[0018] determining a second training data stream from the training data set according to the fourth probability vector;
[0019] The deep learning model is iteratively trained according to the second training data stream until the number of iterative training rounds t is greater than the preset number of training rounds to obtain a defect detection model.
[0020] Preferably, the process of training a defect detection model includes:
[0021] Determine a fifth probability vector based on a training data set and a root mean square calculation method, wherein the training data set includes wafer defect images of multiple defect categories, and the fifth probability vector includes the probability of each defect category being sampled;
[0022] Determining a third training data stream from the training data set according to the fifth probability vector;
[0023] The deep learning model is trained using the third training data stream and a preset number of training rounds to obtain a defect detection model.
[0024] Preferably, the process of calculating and obtaining the normalized average feature vector corresponding to each defect category includes:
[0025] In the process of training the defect detection model, dynamically calculating and updating the average feature vector corresponding to each defect category;
[0026] After the defect detection model is trained, the average feature vectors corresponding to the defect categories are normalized to obtain the normalized average feature vectors corresponding to the defect categories.
[0027] Preferably, determining the similarity between the feature vector of the semiconductor wafer image and the normalized average feature vector corresponding to each of the defect categories includes:
[0028] Normalizing the feature vector of the semiconductor wafer image;
[0029] The similarity between the normalized feature vector of the semiconductor wafer image and the normalized average feature vector corresponding to each of the defect categories is calculated.
[0030] Preferably, determining the first probability vector based on the training data set includes:
[0031] A first probability vector is determined based on the number of defect categories in the training data set, the number of wafer defect images in each defect category, and a first value of a preset parameter.
[0032] A second aspect of an embodiment of the present invention discloses a system for semiconductor wafer defect detection, the system comprising:
[0033] An acquisition unit, used for acquiring an image of a semiconductor wafer to be inspected;
[0034] A first processing unit is used to input the semiconductor wafer image into a defect detection model for feature extraction to obtain a feature vector of the semiconductor wafer image, wherein the defect detection model is obtained by training a deep learning model based on a training data stream determined by the probability of each defect category being sampled;
[0035] A second processing unit is used to determine the similarity between the feature vector of the semiconductor wafer image and the normalized average feature vector corresponding to each of the defect categories, where the normalized average feature vector corresponding to each of the defect categories is calculated during the training process of the defect detection model;
[0036] A determination unit is used to determine that the defect category corresponding to the highest similarity is the defect category detection result of the semiconductor wafer image.
[0037] A third aspect of an embodiment of the present invention discloses an electronic device, including a memory and a processor;
[0038] The memory is used to store computer programs;
[0039] The processor is used to execute the computer program, specifically to implement the method for semiconductor wafer defect detection disclosed in the first aspect of an embodiment of the present invention.
[0040] A fourth aspect of an embodiment of the present invention discloses a computer storage medium for storing a computer program. When the computer program is executed, it is specifically used to implement the method for semiconductor wafer defect detection disclosed in the first aspect of an embodiment of the present invention.
[0041] Based on the above-mentioned embodiment of the present invention, a method, system, electronic device and storage medium for semiconductor wafer defect detection are provided. The method is as follows: obtaining a semiconductor wafer image to be detected; inputting the semiconductor wafer image into a defect detection model for feature extraction to obtain a feature vector of the semiconductor wafer image; determining the similarity between the feature vector of the semiconductor wafer image and the normalized average feature vector corresponding to each defect category, and the normalized average feature vector corresponding to each defect category is calculated during the training process of the defect detection model; determining that the defect category corresponding to the highest similarity is the defect category detection result of the semiconductor wafer image. In this scheme, the semiconductor wafer image to be detected is input into the defect detection model to extract the feature vector of the semiconductor wafer image. Determine the similarity between the feature vector of the semiconductor wafer image and the normalized average feature vector corresponding to each defect category, determine the defect category with the highest similarity as the defect category detection result of the semiconductor wafer image, and use the feature nearest neighbor technology of deep learning to alleviate the problem of imbalanced training data, thereby improving the detection accuracy of the model for the defect category of semiconductor wafers. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.
[0043] Figure 1 An example diagram of data of wafer defect images of various defect categories in an actual production process provided by an embodiment of the present invention;
[0044] Figure 2 A flowchart of a method for detecting semiconductor wafer defects provided by an embodiment of the present invention;
[0045] Figure 3 A first flow chart of a defect detection model obtained by training provided by an embodiment of the present invention;
[0046] Figure 4A second flow chart of a defect detection model obtained by training provided in an embodiment of the present invention;
[0047] Figure 5 A third flow chart of a defect detection model obtained by training provided by an embodiment of the present invention;
[0048] Figure 6 A structural block diagram of a system for semiconductor wafer defect detection provided by an embodiment of the present invention;
[0049] Figure 7 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0050] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0051] In this application, the terms "comprises", "comprising" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element.
[0052] The production process of semiconductor wafers is very complicated, and the defect categories of the semiconductor wafers produced are also many (usually there are hundreds of defect categories), and the characteristics of different defect categories are not very clear; due to the subjectivity and fatigue of humans, quality inspectors are less effective when manually re-evaluating defect categories. Therefore, deep learning-based methods are usually used to automatically identify defect categories.
[0053] However, the inventors have found through research that the defects generated in the actual production process of semiconductor wafers are often concentrated in a limited number of defect categories, such as Figure 1 As shown in the data example of wafer defect images of various defect categories in the actual production process, Figure 1 The horizontal axis in is the index of the defect category. Figure 1 The vertical axis in is the number of wafer defect images in the defect category. Figure 1 There are 5000 wafer defect images in defect category 1, and Figure 1If there are 10 defect categories in the dataset, there are only 50 wafer defect images. If the deep learning model is trained according to the data generated in the actual production process of semiconductor wafers, the learning effects of the trained model for different defect categories will vary greatly. The larger the amount of data, the better the learning effect of the defect category, and vice versa. This will affect the model's detection results for the defect categories of semiconductor wafers.
[0054] To solve the above problems, this solution proposes a method, system, electronic device and storage medium for semiconductor wafer defect detection, and inputs the semiconductor wafer image to be detected into the defect detection model to extract the feature vector of the semiconductor wafer image. The similarity between the feature vector of the semiconductor wafer image and the normalized average feature vector corresponding to each defect category is determined, and the defect category with the highest similarity is determined as the defect category detection result of the semiconductor wafer image. The feature nearest neighbor technology of deep learning is used to alleviate the problem of imbalanced training data, thereby improving the detection accuracy of the model for the defect category of semiconductor wafers.
[0055] See also Figure 2 , shows a flow chart of a method for semiconductor wafer defect detection provided by an embodiment of the present invention, the method comprising:
[0056] Step S201: Acquire an image of a semiconductor wafer to be inspected.
[0057] In the specific implementation of step S201, an image of a semiconductor wafer whose defect category needs to be detected is obtained.
[0058] Step S202: inputting the semiconductor wafer image into the defect detection model for feature extraction to obtain a feature vector of the semiconductor wafer image.
[0059] It should be noted that the probability of each defect category being sampled is determined based on the training data set, and the training data set contains wafer defect images of multiple defect categories. The training data stream is sampled from the training data set by calculating the "probability of each defect category being sampled", and then the deep learning model is trained using the training data stream to obtain a defect detection model. That is, the defect detection model is obtained by training the deep learning model based on the "training data stream determined by the probability of each defect category being sampled".
[0060] Among them, this solution provides at least three ways to train a defect detection model. For details of these three ways to train a defect detection model, please refer to the subsequent embodiments of the present invention. Figures 3 to 5 Explanation of the.
[0061] In the specific implementation of step S202, the semiconductor wafer image to be detected is input into the defect detection model for feature extraction to obtain a feature vector of the semiconductor wafer image; that is, after the semiconductor wafer image is subjected to feature extraction by the defect detection model, the feature vector of the semiconductor wafer image is obtained.
[0062] Step S203: determining the similarity between the feature vector of the semiconductor wafer image and the normalized average feature vector corresponding to each defect category.
[0063] It should be noted that the normalized average feature vector corresponding to each defect category is calculated during the training process of the defect detection model, that is, the normalized average feature vector corresponding to each defect category is calculated in the process of training the defect detection model. The "each defect category" here specifically refers to the "each defect category" in the training data set.
[0064] In some embodiments, the specific implementation method of calculating the normalized average feature vector corresponding to each defect category is: in the process of training the defect detection model, dynamically calculating and updating the average feature vector corresponding to each defect category; after training the defect detection model, normalizing the average feature vector corresponding to each defect category to obtain the normalized average feature vector corresponding to each defect category.
[0065] Specifically, in the process of training the deep learning model to obtain the defect detection model, the feature vectors of the wafer defect images corresponding to each defect category after feature extraction by the deep learning model in the training data set are recorded in real time, and the average feature vector corresponding to each defect category is dynamically calculated and updated; among which, the average feature vector corresponding to defect category j is denoted as v j , j∈[1,2,3,…,C], C is the number of defect categories in the training data set (equivalent to the total number of categories).
[0066] After the training is completed, the average feature vector corresponding to each defect category is normalized (for example, L2 normalization) to obtain the normalized average feature vector corresponding to each defect category; the normalized average feature vector corresponding to defect category j is recorded as , j∈[1,2,3,…,C].
[0067] In the specific implementation process of step S203, for the semiconductor wafer image to be inspected, the feature vector of the semiconductor wafer image is normalized (for example, L2 normalization), and the normalized feature vector of the semiconductor wafer image is denoted as w; it should be noted that the method of normalizing the feature vector of the semiconductor wafer image is the same as the method of normalizing the average feature vector corresponding to each defect category.
[0068] Calculate the similarity between the normalized feature vector of the semiconductor wafer image and the normalized average feature vector corresponding to each defect category; that is, calculate w and the above respectively The similarity between the normalized feature vector w of the semiconductor wafer image and the normalized average feature vector corresponding to the defect category j is The similarity between them is denoted as s j , j∈[1,2,3,…,C] , the similarity calculation of this scheme can use Euclidean distance or cosine distance.
[0069] Step S204: Determine that the defect category corresponding to the highest similarity is the defect category detection result of the semiconductor wafer image.
[0070] In the specific implementation of step S204, w and the above The similarity between j Afterwards, the defect category corresponding to the highest similarity is determined as the defect category detection result of the semiconductor wafer image; that is, the maximum s j The defect category corresponding to the value is used as the defect category detection result of the semiconductor wafer image (or as the final judgment category of the semiconductor wafer image by the defect detection model).
[0071] In an embodiment of the present invention, a semiconductor wafer image to be detected is input into a defect detection model to extract a feature vector of the semiconductor wafer image. The similarity between the feature vector of the semiconductor wafer image and the normalized average feature vector corresponding to each defect category is determined, and the defect category with the highest similarity is determined as the defect category detection result of the semiconductor wafer image. The feature nearest neighbor technology of deep learning is used to alleviate the problem of imbalanced training data, thereby improving the detection accuracy of the model for the defect category of semiconductor wafers.
[0072] For the above-mentioned embodiment of the present invention Figure 2 The process of training the deep learning model to obtain the defect detection model mentioned in step S202 is described in detail in Figure 3 , shows a first flow chart of a defect detection model obtained by training provided by an embodiment of the present invention, Figure 3 The following steps are involved:
[0073] Step S301: Determine a first probability vector based on a training data set.
[0074] It should be noted that a training data set is collected in advance, and the training data set contains wafer defect images of multiple defect categories.
[0075] In the specific implementation of step S301 , a first probability vector is determined based on the training data set, and the first probability vector includes the probability of each defect category being sampled.
[0076] In some embodiments, a specific method for determining the first probability vector is: based on the number of defect categories in the training data set (equivalent to the total number of categories), the number of wafer defect images in each defect category, and a first value of a preset parameter (denoted as q), the first probability vector is determined.
[0077] The above-mentioned preset parameter (q) is a value between [0, 1], and the "first value" of the preset parameter is obtained by experiments based on the training data set.
[0078] It should be noted that when determining the probability of a defect category being sampled, the probability of a defect category being sampled can be determined specifically by formula (1).
[0079] (1);
[0080] In formula (1), p j is the probability of defect category j being sampled in the training data set (equivalent to the probability of sampling from defect category j), n j is the number of wafer defect images in defect category j, C is the number of defect categories in the training data set, q is the preset parameter mentioned above, and q is a value between [0,1].
[0081] In a specific implementation, formula (1) is used in combination with a first value of a preset parameter (q) (the first value is obtained through experiments based on a training data set) to determine the probability of each defect category being sampled in the first probability vector.
[0082] It is worth noting that j∈[1,2,3,…,C] in the above formula (1), so before sampling from the training data set, the first probability vector determined by formula (1) is a C-dimensional first probability vector; for example: the first probability vector is [0.01,0.3,0.23,…,0.04], and random sampling is performed from the training data set according to the size of the probability in the first probability vector to obtain the final sampled defect category.
[0083] Step S302: determining a first training data stream from the training data set according to the first probability vector.
[0084] In the process of specifically implementing step S302, a first probability vector is determined when the preset parameter is a first value, and a first training data stream is determined from the training data set, wherein the first training data stream includes a wafer defect image sampled from the training data set according to the first probability vector.
[0085] It should be noted that in the process of sampling from the training data set according to the first probability vector, at each sampling, the defect category of this sampling is first determined, and then a wafer defect image is randomly selected from the defect category of this sampling by balanced sampling (also called random sampling).
[0086] For example: in the process of sampling from the training data set according to the first probability vector, assuming that the defect category of this sampling is determined to be defect category 2, then a wafer defect image is randomly selected from defect category 2 in a balanced sampling manner; by performing multiple samplings in the above manner, the first training data stream can be obtained.
[0087] In some embodiments, after sampling to obtain the first training data stream, data enhancement processing is performed on the first training data stream; specifically, data enhancement processing is performed on the wafer defect images in the first training data stream.
[0088] For example, data enhancement processing such as rotation, scaling, translation, color conversion, Mixup or Cutmix is performed on the wafer defect image in the first training data stream.
[0089] It is worth noting that in the actual process of wafer defect detection, it is necessary to impose certain restrictions on the transformation amplitude of the wafer defect image transformation to keep the image label of the wafer defect image unchanged before and after the transformation; for example: if a customer site is sensitive to the size of the wafer defect (for the same defect, a large defect area is judged as a defect, and a small defect area is judged as no defect), then the amplitude of the wafer defect image scaling transformation must be controlled.
[0090] Step S303: training a deep learning model according to the first training data stream and a preset number of training rounds to obtain a defect detection model.
[0091] In the process of specifically implementing step S303, after sampling to obtain the first training data stream, the deep learning model is trained according to the first training data stream and the preset number of training rounds in combination with the preset model training method to obtain the defect detection model.
[0092] For example: the preset number of training rounds for training the deep learning model is set to 90 rounds; according to the first training data stream, the preset number of training rounds (90 rounds) and the preset model training method, the deep learning model is trained to obtain a defect detection model.
[0093] It should be noted that in the actual process of training the deep learning model, a fixed preset number of training rounds may not be set. The defect detection model can be obtained by training the deep learning model according to the first training data stream and the preset training method until the deep learning model converges.
[0094] The above embodiments of the present invention Figure 3 This is an explanation of the first training process of the defect detection model.
[0095] In the above Figure 2 and Figure 3 Based on this, the first application scenario of this solution will be:
[0096] A1. According to the customer's requirements for wafer defect determination, a large number of wafer defect images are collected; the collected wafer defect images are counted by defect category to obtain the number of wafer defect images of each defect category, thereby obtaining a training data set.
[0097] A2. Design a deep learning model according to actual needs, set initial parameters of the deep learning model, and determine the first values of the preset parameters.
[0098] A3. Determine a first probability vector according to formula (1) and a first value of a preset parameter; determine a first training data stream from a training data set according to the first probability vector, and train a deep learning model to obtain a defect detection model according to the first training data stream, a preset model training method, and a preset number of training rounds (90 rounds). At the same time, when training the deep learning model, record the feature vectors of each defect category after feature extraction by the deep learning model in real time, and dynamically calculate and update the average feature vector v corresponding to each defect category. j .
[0099] A4. After the deep learning model is trained in A3, the average feature vector v corresponding to each defect category is obtained. j At this time, continue to perform L2 normalization on the average feature vector corresponding to each defect category to obtain the normalized average feature vector corresponding to each defect category .
[0100] A5. When performing inference detection on a semiconductor wafer image that needs to be inspected, the semiconductor wafer image is input into a defect detection model for feature extraction to obtain a feature vector of the semiconductor wafer image, and the feature vector of the semiconductor wafer image is L2 normalized to obtain a normalized feature vector w of the semiconductor wafer image.
[0101] A6. Determine w and the above The similarity between j ; Take the maximum s j The defect category corresponding to the value is used as the defect category detection result of the semiconductor wafer image.
[0102] Through the above embodiments of the present invention Figure 2 and Figure 3It can be seen from the content that this solution introduces the feature nearest neighbor technology of deep learning shown in the above embodiments to effectively alleviate the data imbalance problem actually generated in the semiconductor wafer production process, improve the training accuracy of the deep learning model, and effectively improve the accuracy and speed of production line defect re-judgment when deploying and using the trained model; in addition, the feature nearest neighbor technology of deep learning shown in the above embodiments can be directly integrated into the model training code, and the iteration difficulty and maintenance difficulty of the model are relatively low.
[0103] For the above-mentioned embodiments of the present invention Figure 2 The process of training the deep learning model to obtain the defect detection model mentioned in step S202 is described in detail in Figure 4 , shows a second flow chart of a defect detection model obtained by training provided by an embodiment of the present invention, Figure 4 The following steps are involved:
[0104] Step S401: Determine a second probability vector and a third probability vector based on a training data set.
[0105] It should be noted that a training data set is collected in advance, and the training data set contains wafer defect images of multiple defect categories.
[0106] In the process of specifically implementing step S401, based on the training data set, a second probability vector and a third probability vector are determined, the second probability vector and the third probability vector contain the probabilities of each defect category being sampled, the probabilities of at least two defect categories being sampled in the second probability vector are different, and the probabilities of each defect category being sampled in the third probability vector are the same.
[0107] In some embodiments, the specific method for determining the second probability vector and the third probability vector is as follows: based on the number of defect categories in the training data set (equivalent to the total number of categories), the number of wafer defect images in each defect category, and the second value of a preset parameter (denoted as q), the second probability vector is determined; based on the number of defect categories in the training data set, the number of wafer defect images in each defect category, and the third value of the preset parameter, the third probability vector is determined.
[0108] Among them, the “second value” of the above preset parameter can be 1, and the “third value” of the preset parameter can be 0.
[0109] It should be noted that when determining the probability of a defect category being sampled, the probability of a defect category being sampled can be determined specifically by the above formula (1).
[0110] Therefore, the probability of each defect category being sampled in the second probability vector can be determined by using formula (1) in combination with the second value of the preset parameter. The probability of each defect category being sampled in the third probability vector can be determined by using formula (1) in combination with the third value of the preset parameter.
[0111] From formula (1), it can be seen that when q = 1 (the second value of the preset parameter), the probability of defect category j being sampled is , at this time, it can be characterized that the probability of a certain defect category being sampled is proportional to the number of wafer defect images in the defect category, that is, the probability of a certain defect category being sampled in the second probability vector is related to the number of wafer defect images in the defect category. The greater the number of wafer defect images in defect category j, the greater the probability of defect category j being sampled (conversely, the smaller the probability of being sampled); the probabilities of at least two defect categories being sampled in the second probability vector are different, and the probability of defect category j being sampled in the second probability vector is recorded as (Instance-balanced sampling).
[0112] From formula (1), it can be seen that when q = 0 (the third value of the preset parameter), the probability of defect category j being sampled is , it can be shown that the probability of different defect categories being sampled is independent of the number of wafer defect images in the defect category, that is, the probability of each defect category being sampled in the third probability vector is the same, and the probability of defect category j being sampled in the third probability vector is recorded as (Class-balanced sampling).
[0113] Step S402: During the t-th round of iterative training of the deep learning model, a fourth probability vector is determined based on the second probability vector, the third probability vector, the number of iterative training rounds t, and the preset number of training rounds.
[0114] It should be noted that the iterative training round number t is greater than 0, and the initial value of the iterative training round number t is 1; the fourth probability vector includes the probability of each defect category being sampled.
[0115] In the process of specifically implementing step S402, during the tth round of iterative training of the deep learning model, the fourth probability vector used in the tth round of iterative training is determined based on the second probability vector, the third probability vector, the number of iterative training rounds t, and the preset number of training rounds (denoted as T, the set value before training).
[0116] It should be noted that during the entire process of training the deep learning model, the deep learning model will undergo a total of T (preset number of training rounds) rounds of training, and the number of iterative training rounds t will increase as the iterative training progresses (the number of iterative training rounds t increases by 1 after each round of iterative training).
[0117] For example, when the deep learning model is subjected to the first round of iterative training, the number of iterative training rounds t is 1; after the first round of iterative training is completed, the deep learning model is subjected to the second round of iterative training, and the number of iterative training rounds t is 2; and so on, the number of iterative training rounds t increases as the iterative training progresses.
[0118] In some embodiments, the specific method for determining the fourth probability vector used in the t-th round of iterative training is: during the t-th round of iterative training of the deep learning model, using formula (2), determine the fourth probability vector used in the t-th round of iterative training.
[0119] (2);
[0120] In formula (2), T is the preset number of training rounds, is the probability of defect category j being sampled in the second probability vector, is the probability of defect category j being sampled in the third probability vector, is the probability of defect category j being sampled in the fourth probability vector determined during the t-th round of iterative training.
[0121] It is worth noting that j∈[1,2,3,…,C] in the above formula (2), so before sampling from the training data set, the determined fourth probability vector is a C-dimensional probability vector; for example: the determined fourth probability vector is [0.01,0.3,0.23,…,0.04], and random sampling is performed from the training data set according to the size of the probability in the fourth probability vector to obtain the final sampled defect category.
[0122] Step S403: Determine a second training data stream from the training data set according to the fourth probability vector.
[0123] In the process of implementing step S403, after calculating the fourth probability vector used in the t-th round of iterative training, data is sampled from the training data set according to the fourth probability vector to obtain the second training data stream used in the t-th round of iterative training. The second training data stream includes the wafer defect image sampled from the training data set according to the fourth probability vector.
[0124] It should be noted that in the process of sampling from the training data set according to the fourth probability vector, at each sampling, the defect category of this sampling is first determined, and then a wafer defect image is randomly selected from the defect category of this sampling by balanced sampling (also called random sampling).
[0125] For example: in the process of sampling from the training data set according to the fourth probability vector, assuming that the defect category of this sampling is determined to be defect category 2, then a wafer defect image is randomly selected from defect category 2 in a balanced sampling manner; by performing multiple samplings in the aforementioned manner, the second training data stream used in the tth round of iterative training can be obtained.
[0126] In some embodiments, after determining the second training data stream used in the t-th round of iterative training, data enhancement processing is performed on the second training data stream; specifically, data enhancement processing is performed on the wafer defect images in the second training data stream.
[0127] For example, data enhancement processing such as rotation, scaling, translation, color conversion, Mixup or Cutmix is performed on the wafer defect image in the second training data stream.
[0128] It is worth noting that in the actual process of wafer defect detection, it is necessary to impose certain restrictions on the transformation amplitude of the wafer defect image transformation to keep the image label of the wafer defect image unchanged before and after the transformation; for example: if a customer site is sensitive to the size of the wafer defect (for the same defect, a large defect area is judged as a defect, and a small defect area is judged as no defect), then the amplitude of the wafer defect image scaling transformation must be controlled.
[0129] Step S404: iteratively train the deep learning model according to the second training data stream until the number of iterative training rounds t is greater than the preset number of training rounds to obtain a defect detection model.
[0130] In the process of specifically implementing step S404, after determining the second training data stream used in the t-th round of iterative training, the deep learning model is subjected to the t-th round of iterative training according to the second training data stream; after completing the t-th round of iterative training, t=t+1 (i.e., the number of iterative training rounds t is increased by 1, which is equivalent to updating the number of iterative training rounds t).
[0131] Determine whether the updated iterative training round number t is greater than the preset training round number; if the updated iterative training round number t is not greater than the preset training round number, return to execute step S402; if the updated iterative training round number t is greater than the preset training round number, determine that the deep learning model has completed training and obtain a defect detection model.
[0132] In a specific implementation, after completing the tth round of iterative training and updating the number of iterative training rounds t, it is determined whether the updated number of iterative training rounds t is greater than the preset number of training rounds; if the updated number of iterative training rounds t is not greater than the preset number of training rounds (t≦T), it indicates that the deep learning model has not completed all rounds of training. At this time, the process returns to step S402 to continue determining a new fourth probability vector and continue the next round of training for the deep learning model.
[0133] If the updated iterative training round number t is greater than the preset training round number (t>T), it indicates that the deep learning model has completed all rounds of training. At this time, it is determined that the deep learning model has completed training and a defect detection model is obtained (that is, the deep learning model after completing the last round of training).
[0134] That is, if the updated iterative training round number t is not greater than the preset training round number (t≦T), the process returns to step S402 until the iterative training round number t is greater than the preset training round number to obtain a defect detection model.
[0135] For example: Assume that the preset number of training rounds T is 90; after the 89th round (t=89 at this time) of iterative training of the deep learning model, t=t+1=89+1, and the updated t(90) is not greater than T. At this time, return to step S402 to continue to determine the fourth probability vector used in the 90th round of iterative training, and then determine the second training data stream used in the 90th round of iterative training from the training data set based on the fourth probability vector, and use the second training data stream to train the deep learning model for the 90th round. After the 90th round (t=90 at this time) of iterative training of the deep learning model, t=t+1=90+1; at this time, the updated t(91) is greater than T(90), and it is determined that the deep learning model that has completed the 90th round of iterative training is a defect detection model.
[0136] It can be seen from the above formula (2) and the contents of steps S402 to S404 that when the deep learning model just starts iterative training, since the value of the iterative training round number t is small, the “ "This item accounts for a small proportion. The probability of each defect category being sampled in the fourth probability vector is mainly determined by " " is determined by this item; as the training of the deep learning model continues, the number of iterative training rounds t will gradually increase, and the probability of each defect category being sampled in the fourth probability vector will slowly approach " "This item is tilted (i.e. " "The proportion of this item will gradually increase); the above method can better alleviate the training problem of data imbalance in the model training process.
[0137] The above embodiments of the present invention Figure 4This is an explanation of the second training process of the defect detection model.
[0138] In the above Figure 4 On this basis, the second application scenario of this solution is:
[0139] B1. According to the customer's requirements for wafer defect determination, a large number of wafer defect images are collected; the collected wafer defect images are counted by defect category to obtain the number of wafer defect images of each defect category, thereby obtaining a training data set.
[0140] B2. Design a deep learning model according to actual needs and set the initial parameters of the deep learning model.
[0141] B3. According to the above formula (1), determine the probability of each defect category being sampled when q=1 , thus obtaining the second probability vector.
[0142] B4. According to the above formula (1), determine the probability of each defect category being sampled when q=0 , thus obtaining the third probability vector.
[0143] B5. Based on and According to the above formula (2), the probability of each defect category being sampled in the tth round of iterative training is determined , thus obtaining the fourth probability vector used in the t-th round of iterative training.
[0144] B6. According to the fourth probability vector calculated in B5, determine the second training data stream used in the tth round of iterative training from the training data set, and perform data enhancement processing on the second training data stream; the initial value of the iterative training round number t is 1.
[0145] B7. Perform the t-th round of iterative training on the deep learning model using the second training data stream used in the t-th round of iterative training; update the number of iterative training rounds t (t=t+1), and repeat B5 and B6 until the deep learning model completes T rounds of iterative training to obtain a defect detection model.
[0146] Through the above-mentioned embodiments of the present invention Figure 4It can be seen from the content that this solution introduces the progressive balanced sampling technology shown in the above embodiments to effectively alleviate the data imbalance problem actually generated in the semiconductor wafer production process, improve the training accuracy of the deep learning model, and effectively improve the accuracy and speed of production line defect re-judgment when deploying and using the trained model; in addition, the progressive balanced sampling technology shown in the above embodiments can be directly integrated into the model training code, and the iteration difficulty and maintenance difficulty of the model are relatively low.
[0147] For the above-mentioned embodiment of the present invention Figure 2 The process of training the deep learning model to obtain the defect detection model mentioned in step S202 is described in detail in Figure 5 , shows a third flow chart of training a defect detection model provided by an embodiment of the present invention, Figure 5 The following steps are involved:
[0148] Step S501: Determine a fifth probability vector based on a training data set and a root mean square calculation method.
[0149] It should be noted that a training data set is collected in advance, and the training data set contains wafer defect images of multiple defect categories.
[0150] In the specific implementation of step S501, a fifth probability vector is determined based on the training data set and the root mean square calculation method. The fifth probability vector includes the probability of each defect category being sampled.
[0151] In some embodiments, the specific method of determining the fifth probability vector is: using formula (3), determine the probability of each defect category being sampled in the training data set to obtain the fifth probability vector, that is, the fifth probability vector includes the "probability of each defect category being sampled" determined by formula (3).
[0152] (3);
[0153] In formula (3), C is the number of defect categories in the training data set, p j is the probability of the jth defect category (equivalent to defect category j) being sampled in the training data set, n j is the number of wafer defect images in the jth defect category, n i is the number of wafer defect images in the i-th defect category.
[0154] It should be noted that j∈[1,2,3,…,C] in the above formula (3), so before sampling from the training data set, the fifth probability vector determined is a C-dimensional probability vector; for example: the calculated fifth probability vector is [0.01,0.3,0.23,…,0.04], and random sampling is performed from the training data set according to the size of the probability in the fifth probability vector to obtain the final sampled defect category.
[0155] Step S502: Determine a third training data stream from the training data set according to the fifth probability vector.
[0156] In the specific implementation of step S502, a third training data stream is determined from the training data set according to the fifth probability vector, and the third training data stream includes a wafer defect image sampled from the training data set according to the fifth probability vector.
[0157] It should be noted that, in the process of sampling from the training data set according to the fifth probability vector, at each sampling, the defect category of this sampling is first determined, and then a wafer defect image is randomly selected from the defect category of this sampling.
[0158] For example: in the process of sampling from the training data set according to the fifth probability vector, assuming that the defect category of this sampling is determined to be defect category j, then a wafer defect image is randomly selected from defect category j; by performing multiple samplings in the above manner, the third training data stream can be obtained.
[0159] In some embodiments, after the third training data stream is determined, data enhancement processing is performed on the third training data stream; specifically, data enhancement processing is performed on the wafer defect images in the third training data stream.
[0160] For example, data enhancement processing such as rotation, scaling, translation, color conversion, Mixup or Cutmix is performed on the wafer defect image in the third training data stream.
[0161] It is worth noting that in the actual process of wafer defect detection, it is necessary to impose certain restrictions on the transformation amplitude of the wafer defect image transformation to keep the image label of the wafer defect image unchanged before and after the transformation; for example: if a customer site is sensitive to the size of the wafer defect (for the same defect, a large defect area is judged as a defect, and a small defect area is judged as no defect), then the amplitude of the wafer defect image scaling transformation must be controlled.
[0162] Step S503: Train the deep learning model using the third training data stream and a preset number of training rounds to obtain a defect detection model.
[0163] In the process of implementing step S503, the deep learning model is trained by the third training data stream obtained by the above steps S501 and S502 until the deep learning model converges (determining whether the model converges by specifying indicators), thereby obtaining a defect detection model.
[0164] The above embodiments of the present invention Figure 5 This is an explanation of the third training process for the defect detection model.
[0165] In the above Figure 5 On this basis, the third application scenario of this solution is:
[0166] C1. According to the customer's requirements for wafer defect determination, a large number of wafer defect images are collected; the collected wafer defect images are counted by defect category to obtain the number of wafer defect images of each defect category, thereby obtaining a training data set.
[0167] C2. Design a deep learning model according to actual needs and set the initial parameters of the deep learning model.
[0168] C3. Determine the probability of each defect category being sampled according to the above formula (3) to obtain a fifth probability vector; perform data sampling once according to the fifth probability vector; and then perform data enhancement processing on the sampled wafer defect image.
[0169] C4. Repeat the data sampling and other processing in C3 to obtain a third training data stream; use the third training data stream to train the deep learning model until the deep learning model converges to obtain a defect detection model.
[0170] Through the above-mentioned embodiments of the present invention Figure 5 It can be seen from the content that this solution introduces the root mean square sampling technology shown in the above embodiments to effectively alleviate the data imbalance problem actually generated in the semiconductor wafer production process, improve the training accuracy of the deep learning model, and effectively improve the accuracy and speed of production line defect re-judgment when deploying and using the trained model; in addition, the root mean square sampling technology shown in the above embodiments can be directly integrated into the model training code, and the iteration difficulty and maintenance difficulty of the model are relatively low.
[0171] Corresponding to the method for detecting semiconductor wafer defects provided by the above embodiment of the present invention, see Figure 6 , an embodiment of the present invention also provides a structural block diagram of a system for semiconductor wafer defect detection, the system comprising: an acquisition unit 601, a first processing unit 602, a second processing unit 603, and a determination unit 604;
[0172] The acquisition unit 601 is used to acquire the image of the semiconductor wafer to be inspected.
[0173] The first processing unit 602 is used to input the semiconductor wafer image into the defect detection model for feature extraction to obtain a feature vector of the semiconductor wafer image. The defect detection model is obtained by training a deep learning model based on a training data stream determined by the probability of each defect category being sampled.
[0174] The second processing unit 603 is used to determine the similarity between the feature vector of the semiconductor wafer image and the normalized average feature vector corresponding to each defect category, and the normalized average feature vector corresponding to each defect category is calculated during the training process of the defect detection model.
[0175] In a specific implementation, the second processing unit 603 is specifically used to normalize the feature vector of the semiconductor wafer image; and calculate the similarity between the normalized feature vector of the semiconductor wafer image and the normalized average feature vector corresponding to each defect category.
[0176] The determination unit 604 is used to determine that the defect category corresponding to the highest similarity is the defect category detection result of the semiconductor wafer image.
[0177] In an embodiment of the present invention, a semiconductor wafer image to be detected is input into a defect detection model to extract a feature vector of the semiconductor wafer image. The similarity between the feature vector of the semiconductor wafer image and the normalized average feature vector corresponding to each defect category is determined, and the defect category with the highest similarity is determined as the defect category detection result of the semiconductor wafer image. The feature nearest neighbor technology of deep learning is used to alleviate the problem of imbalanced training data, thereby improving the detection accuracy of the model for the defect category of semiconductor wafers.
[0178] Preferably, combined Figure 4 As shown in the figure, the first processing unit 602 includes a first calculation module, a first sampling module and a first training module, and the execution principle of each module is as follows:
[0179] The first calculation module is used to determine a first probability vector based on a training data set, the training data set includes wafer defect images of multiple defect categories, and the first probability vector includes the probability of each defect category being sampled.
[0180] In a specific implementation, the first calculation module is specifically used to determine a first probability vector based on the number of defect categories in the training data set, the number of wafer defect images in each defect category, and a first value of a preset parameter.
[0181] The first sampling module is used to determine a first training data stream from the training data set according to the first probability vector.
[0182] The first training module is used to train the deep learning model according to the first training data stream and a preset number of training rounds to obtain a defect detection model.
[0183] Preferably, combined Figure 4 As shown in the figure, the first processing unit 602 includes a second calculation module, a third calculation module, a second sampling module, and a second training module; the execution principle of each module is as follows:
[0184] The second calculation module is used to determine the second probability vector and the third probability vector based on the training data set, the training data set contains wafer defect images of multiple defect categories, the second probability vector and the third probability vector contain the probability of each defect category being sampled, the probabilities of at least two defect categories being sampled in the second probability vector are different, and the probabilities of each defect category being sampled in the third probability vector are the same.
[0185] In a specific implementation, the second calculation module is specifically used to: determine a second probability vector based on the number of defect categories in the training data set, the number of wafer defect images in each defect category, and the second value of the preset parameter;
[0186] A third probability vector is determined based on the number of defect categories in the training data set, the number of wafer defect images in each defect category, and a third value of the preset parameter.
[0187] The third calculation module is used to determine a fourth probability vector based on the second probability vector, the third probability vector, the number of iterative training rounds t, and the preset number of training rounds during the tth round of iterative training of the deep learning model. The fourth probability vector includes the probability of each defect category being sampled.
[0188] The second sampling module is used to determine a second training data stream from the training data set according to the fourth probability vector.
[0189] The second training module is used to iteratively train the deep learning model according to the second training data stream until the number of iterative training rounds t is greater than the preset number of training rounds to obtain a defect detection model.
[0190] Preferably, combined Figure 4 As shown in the figure, the first processing unit 602 includes a fourth calculation module, a third sampling module and a third training module; the execution principle of each module is as follows:
[0191] The fourth calculation module is used to determine a fifth probability vector based on a training data set and a root mean square calculation method. The training data set includes wafer defect images of multiple defect categories, and the fifth probability vector includes the probability of each defect category being sampled.
[0192] The third sampling module is used to determine a third training data stream from the training data set according to the fifth probability vector.
[0193] The third training module is used to train the deep learning model using the third training data stream and a preset number of training rounds to obtain a defect detection model.
[0194] Preferably, combined Figure 4 As shown in the figure, the second processing unit 603 includes a calculation module and a normalization module; the execution principle of each module is as follows:
[0195] The calculation module is used to dynamically calculate and update the average feature vector corresponding to each defect category in the process of training the defect detection model.
[0196] The normalization module is used to normalize the average feature vectors corresponding to each defect category after the defect detection model is trained to obtain the normalized average feature vectors corresponding to each defect category.
[0197] An embodiment of the present invention further provides a computer storage medium for storing a computer program. When the computer program is executed, it is specifically used to implement the method for semiconductor wafer defect detection provided by the above method embodiment.
[0198] The present invention also provides an electronic device, see Figure 7 , Figure 7 is a schematic diagram of the structure of the electronic device, which may include a memory 701 and a processor 702;
[0199] The memory 701 is used to store computer programs;
[0200] The processor 702 is used to execute a computer program, specifically to implement the method for semiconductor wafer defect detection provided by the above method embodiment.
[0201] In summary, the embodiments of the present invention provide a method, system, electronic device and storage medium for semiconductor wafer defect detection, wherein the semiconductor wafer image to be detected is input into a defect detection model to extract the feature vector of the semiconductor wafer image. The similarity between the feature vector of the semiconductor wafer image and the normalized average feature vector corresponding to each defect category is calculated, and the defect category with the highest similarity is determined as the defect category detection result of the semiconductor wafer image. The feature nearest neighbor technology of deep learning is used to alleviate the problem of imbalanced training data, thereby improving the detection accuracy of the model for the defect category of semiconductor wafers.
[0202] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can refer to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiment. The system and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without creative work.
[0203] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.
[0204] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A method for detecting semiconductor wafer defects, characterized in that: The method comprises: Acquire an image of a semiconductor wafer to be inspected; Inputting the semiconductor wafer image into a defect detection model for feature extraction to obtain a feature vector of the semiconductor wafer image, wherein the defect detection model is obtained by training a deep learning model based on a training data stream determined by the probability of each defect category being sampled; Determining the similarity between the feature vector of the semiconductor wafer image and the normalized average feature vector corresponding to each of the defect categories, the normalized average feature vector corresponding to each of the defect categories being calculated during the training process of the defect detection model; Determine that the defect category corresponding to the highest similarity is the defect category detection result of the semiconductor wafer image.
2. The method according to claim 1, characterized in that The process of training a defect detection model includes: Determine a first probability vector based on a training data set, wherein the training data set includes wafer defect images of multiple defect categories, and the first probability vector includes the probability of each defect category being sampled; Determining a first training data stream from the training data set according to the first probability vector; The deep learning model is trained according to the first training data stream and a preset number of training rounds to obtain a defect detection model.
3. The method according to claim 1, characterized in that The process of training a defect detection model includes: Determine a second probability vector and a third probability vector based on a training data set, wherein the training data set includes wafer defect images of multiple defect categories, the second probability vector and the third probability vector include probabilities of each defect category being sampled, the probabilities of at least two defect categories being sampled in the second probability vector are different, and the probabilities of each defect category being sampled in the third probability vector are the same; In the process of performing the tth round of iterative training on the deep learning model, based on the second probability vector, the third probability vector, the number of iterative training rounds t, and the preset number of training rounds, a fourth probability vector is determined, wherein the fourth probability vector includes the probability of each of the defect categories being sampled; determining a second training data stream from the training data set according to the fourth probability vector; The deep learning model is iteratively trained according to the second training data stream until the number of iterative training rounds t is greater than the preset number of training rounds to obtain a defect detection model.
4. The method according to claim 1, characterized in that: The process of training a defect detection model includes: Determine a fifth probability vector based on a training data set and a root mean square calculation method, wherein the training data set includes wafer defect images of multiple defect categories, and the fifth probability vector includes the probability of each defect category being sampled; Determining a third training data stream from the training data set according to the fifth probability vector; The deep learning model is trained using the third training data stream and a preset number of training rounds to obtain a defect detection model.
5. The method according to any one of claims 1 to 4, characterized in that: The process of calculating and obtaining the normalized average feature vector corresponding to each defect category includes: In the process of training the defect detection model, dynamically calculating and updating the average feature vector corresponding to each defect category; After the defect detection model is trained, the average feature vectors corresponding to the defect categories are normalized to obtain the normalized average feature vectors corresponding to the defect categories.
6. The method according to any one of claims 1 to 4, characterized in that: Determining the similarity between the feature vector of the semiconductor wafer image and the normalized average feature vector corresponding to each of the defect categories includes: Normalizing the feature vector of the semiconductor wafer image; The similarity between the normalized feature vector of the semiconductor wafer image and the normalized average feature vector corresponding to each of the defect categories is calculated.
7. The method according to claim 2, characterized in that: Determining a first probability vector based on a training data set includes: A first probability vector is determined based on the number of defect categories in the training data set, the number of wafer defect images in each defect category, and a first value of a preset parameter.
8. A system for detecting semiconductor wafer defects, characterized in that: The system comprises: An acquisition unit, used for acquiring an image of a semiconductor wafer to be inspected; A first processing unit is used to input the semiconductor wafer image into a defect detection model for feature extraction to obtain a feature vector of the semiconductor wafer image, wherein the defect detection model is obtained by training a deep learning model based on a training data stream determined by the probability of each defect category being sampled; A second processing unit is used to determine the similarity between the feature vector of the semiconductor wafer image and the normalized average feature vector corresponding to each of the defect categories, where the normalized average feature vector corresponding to each of the defect categories is calculated during the training process of the defect detection model; A determination unit is used to determine that the defect category corresponding to the highest similarity is the defect category detection result of the semiconductor wafer image.
9. An electronic device, characterized in that: including memory and processor; The memory is used to store computer programs; The processor is used to execute the computer program, specifically to implement the method for semiconductor wafer defect detection as described in any one of claims 1 to 7.
10. A computer storage medium, characterized in that: Used to store a computer program, which, when executed, is specifically used to implement the method for semiconductor wafer defect detection as described in any one of claims 1 to 7.