Small sample expansion and identification method for wafer overlay analysis

Abstract

The invention provides a small sample expansion and identification method for wafer overlay analysis, which comprises the following steps of: 1, automatically classifying overlay samples to obtain different classification results; 2, performing corresponding defect simulation according to a classification result to obtain a corresponding defect result; 3, performing defect fusion on different types of defect results obtained in the step 2 to generate a defect picture; 4, training the recognition model by utilizing the defect pictures obtained in the step 3; 5, after training of the identification model is completed, model verification is carried out; and step 6, carrying out automatic judgment on stacked graph identification on the to-be-processed stacked graph sample by utilizing the identification model output in the step 5. According to the method, the problems of training and sample expansion and multiplication under the conditions of small samples and a small number of pictures are solved.

Description

Technical Field

[0001] The present invention relates to the technical field of industrial image detection, and particularly relates to a small sample expansion and recognition method for wafer overlay analysis. Background Art

[0002] In the field of industrial automation quality inspection based on image detection algorithms, especially in the semiconductor wafer related fields, the overall defect analysis method is often used. Specifically: on a virtual wafer image map, all defects are marked on the virtual map in the form of dot marks. Then, by observing the shape formed by the defect clusters in the virtual map, the personnel can generally judge the defect classification of the wafer. This method is not only limited to a single wafer, but also the defects of multiple wafers can be unified and superimposed on a single virtual map for unified discrimination. The superimposition range includes superimposing in different ways such as by lot (production batch number), by machine, by process, by type, by time, etc. This method of using overlay analysis and determination is widely used in the wafer semiconductor manufacturing industry, which can effectively locate the causes of defects, timely optimize process parameters, improve the yield, and assist users in optimizing the process in the production process.

[0003] The existing overlay analysis methods are mainly divided into three categories: The first is pure manual detection, which is identified by the human eye on the screen of the overlay software. For the specific classification of the current screenshot results, the staff needs to observe the current image while comparing with the defect template map to determine whether it is a defect. This method has low efficiency, strong subjectivity, and a high error rate, and is extremely prone to over-inspection or missed inspection, and its significance for improving the user's yield and efficiency is not obvious. The second method is defect recognition based on image algorithms or big data algorithm template matching. The core principle of this recognition is to use a small number of template maps as the matching basis and add some simple rotation, translation, or scaling functions, so that the algorithm has relative robustness. During the operation process, the actual image is template-matched with the template after certain changes, and the similarity is output. This method also has the problems of low flexibility, poor accuracy, high over-sand rate, and high missed inspection rate. The third method is the artificial intelligence model detection method based on deep learning. In this method, the results of different classifications are uniformly input into the classifier or detector model for training, so that the model has the ability to recognize its features. The main current drawback of this method is that the available data samples are few and cannot be used. It is difficult to improve the fast training accuracy and recognition rate.

[0004] The current deficiency is that: during the use of overlay analysis, users will provide a list of common defects based on experience, and the defects are illustrated by arranging them in different shapes in the virtual map. There is only one or a few pictures of each defect for people to refer to. Therefore, when users want to use algorithms based on deep learning or intelligent recognition to replace manual work, the one or a few pictures of defect examples cannot meet the requirements of model training. Summary of the Invention

[0005] In view of the above problems, the present invention provides a small-sample expansion and recognition method for wafer overlay analysis. Based on the technical route of using an artificial intelligence deep learning model for overlay defect recognition, it solves the problems of training, sample expansion, and doubling in the case of small samples and a small number of pictures, and organically integrates data doubling and defect recognition by combining recognition algorithms.

[0006] To achieve the above object, the present invention adopts the following technical solutions:

[0007] The present invention provides a small-sample expansion and recognition method for wafer overlay analysis, including the following steps:

[0008] Step 1: Automatically classify the overlay samples to obtain different classification results;

[0009] Step 2: Perform corresponding defect simulations according to the classification results to obtain corresponding defect results;

[0010] Step 3: Fuse the different types of defect results obtained in Step 2 to generate defect pictures;

[0011] Step 4: Use the defect pictures obtained in Step 3 to train the recognition model;

[0012] Step 5: After the recognition model is trained, perform model verification;

[0013] Step 6: Use the recognition model output in Step 5 to automatically determine the overlay recognition of the overlay samples to be processed.

[0014] Further, in the small-sample expansion and recognition method for wafer overlay analysis provided by the present invention: when the overlay samples enter the automatic classification process of Step 1, there are parameter setting operations, and the parameters involved include defect range, blur degree, sampling frequency, maximum natural estimation degree, self-test autocorrelation, and passing threshold.

[0015] Further, in the small-sample expansion and recognition method for wafer overlay analysis provided by the present invention: in Step 1, the automatic classification classifies the overlay samples into free defects and overall shape defects according to the size and connectivity of the defects. Free defects include scratches and fractures, and overall shape defects include horizontal lines, vertical lines, and concentric circles.

[0016] Furthermore, in the small sample expansion and identification method for wafer overlay analysis provided by the present invention: step 2 performs corresponding free defect simulation and overall shape defect simulation based on the classification results. The free defect simulation is to extract the defect features within the free defect identification range and multiply the data thereof. The overall shape defect simulation is to identify the shape contour, extract the main elements and main points that constitute the shape, and multiply the data based on the main points.

[0017] Furthermore, in the small sample expansion and identification method for wafer overlay analysis provided by the present invention: the free defect simulation specifically includes the following processes: 1) extracting defect feature information within the free defect mark identification range, including length, width, curvature, connectivity, area, shape, and density; 2) rotating, translating, scaling, discrete sampling, random sampling, and edge cropping the defects, and then pasting them on a blank background image.

[0018] Furthermore, in the small sample expansion and identification method for wafer overlay analysis provided by the present invention: the overall shape defect simulation specifically includes the following processes: 1) identifying the core area where the shape is located, eliminating the background interference area, extracting the main points and main elements that constitute the shape, and calculating the probability estimate of defects at each point in the image to form a probability estimation depth map; 2) multiplying the data by setting the sampling rate, density, degree, rotation, and translation, and then collage it on a blank background image.

[0019] Furthermore, in the small sample expansion and identification method for wafer overlay analysis provided by the present invention: the defect fusion in step 3 is to randomly fuse the free defects and the overall shape defects.

[0020] Furthermore, in the small sample expansion and identification method for wafer overlay analysis provided by the present invention: random fusion is random selection and splicing, and noise is added during splicing as a simulation of the real situation. At the same time, OK pictures are generated. OK pictures refer to pictures with no defects and only noise. After all defective pictures are generated, the defective pictures are labeled accordingly as model training files.

[0021] Furthermore, in the small sample expansion and recognition method for wafer overlay analysis provided by the present invention: in step 4, the training of the recognition model adopts a convolutional neural network to extract the features of all generated samples, and associate and compare them with the labeled features, and perform model verification after the training is completed; in step 5, the verification of the recognition model uses the original overlay sample and the defect image to verify the trained convolutional neural network, compares the autocorrelation function of each feature layer to measure the similarity between the two, sets a similarity threshold, and compares it with the similarity threshold. If it meets the similarity threshold, it is determined that the defect image simulation is successful, and if it does not meet the similarity threshold, it is determined that the defect image simulation fails.

[0022] Furthermore, in the small sample expansion and identification method for wafer overlay analysis provided by the present invention: when it is determined in step 5 that the defect image simulation fails, the following parameter optimization link is entered: an optimization model of input parameters, objective functions and similarities is constructed, and the input parameters are optimized using the Newton iteration method. After optimizing the input parameters, the simulation process is re-performed on all defect images, and the simulation results are re-input into the training network for iterative training until the defect image simulation is determined to be successful by meeting the similarity threshold, and then the recognition model is output.

[0023] Compared with the prior art, the present invention has the following advantages:

[0024] The present invention provides a small sample expansion and identification method for wafer overlay analysis. This method uses an artificial intelligence deep learning model. Based on the technical route of overlay defect identification, it solves the problems of training, sample expansion and multiplication in the case of small samples and a small number of images, and organically integrates data multiplication and defect identification in combination with the recognition algorithm.

[0025] In addition, since the samples used in the expansion of the present invention are real samples, the authenticity of their data features is guaranteed. On the premise of meeting the authenticity of the pictures, the overlay defects can be effectively simulated and expanded, allowing the deep learning model to identify the defect features faster and more accurately, thereby achieving a rapid improvement in model accuracy.

[0026] In addition, the present invention innovatively divides defects into free defects and overall shape defects, first uses an artificial intelligence algorithm to automatically distinguish between the two defects, and then sends sample images of the two defects to different simulation links for simulation. The simulated image retains the defect characteristics of the real image to a great extent, and performs effective data expansion. Finally, the defects are fused to form defect samples that can be used for training, and a training annotation file is automatically formed. While directly training the data obtained by data multiplication, the degree of aggregation of the feature distribution after training will be evaluated to assist in judging the generation effect. When the generation effect is not good, the parameters can be adjusted according to the threshold setting and the samples can be eliminated at the same time. After the overall algorithm process automatically enters the loop, multiple rounds of data training can be performed for iteration, and manual parameter adjustment is supported to finally output an effective recognition model. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 This is a flow chart of a small sample expansion and identification method for wafer overlay analysis according to an embodiment of the present invention;

[0028] Figure 2 It is a schematic diagram of various defects on the wafer;

[0029] Figure 3 Schematic diagram of automatic classification in an embodiment of the present invention, the left figure is the input image, and the right figure is the automatic classification result;

[0030] Figure 4 It is a schematic diagram of data multiplication for free defect simulation in an embodiment of the present invention;

[0031] Figure 5 It is a schematic diagram of overall shape defect simulation in an embodiment of the present invention. The left figure is the input image, and the right figure is the probability estimation depth map;

[0032] Figure 6 It is a schematic diagram of data multiplication for overall shape defect simulation in an embodiment of the present invention. Specific implementation manners

[0033] In order to make the technical means, creative features, achieved purposes and effects realized by the present invention easy to understand, the following embodiments will specifically elaborate on the technical solutions of the present invention in conjunction with the accompanying drawings.

[0034] See Figure 1 , an embodiment of the present invention provides a small sample expansion and recognition method for wafer overlay analysis, including the following steps:

[0035] Step 1: Automatically classify the overlay samples to obtain different classification results.

[0036] There are various defects in wafers, Figure 2 Some common defect forms are illustrated in. The automatic classification in this step classifies the overlay samples into free defects and overall shape defects according to the size and connectivity of the defects through an automatic classification algorithm model. Free defects include scratches, fractures, etc. Overall shape defects include horizontal lines, vertical lines, concentric circles, etc.

[0037] The automatic classification algorithm model is an artificial intelligence classification algorithm model based on a convolutional neural network and is a model trained in advance based on annotations. See Figure 1 , after the overlay samples enter the process of the present invention, they first enter the automatic classification link. The automatic classification process has parameter setting operations. The parameters involved include but are not limited to defect range, fuzziness, sampling frequency, maximum natural estimation degree, self-test self-correlation, and passing threshold. The initial parameters are set manually, and the subsequent parameter settings are automatically adjusted during the training and verification process of the recognition model. In addition, during the training process, if the personnel monitor that the training gets stuck in the host or the training parameters are abnormal, etc., manual settings can be performed again.

[0038] After completing the automatic defect classification, free defects and overall shape defects are automatically divided in the image, and the results are as Figure 3 shown. Different types of defect areas are represented by frames of different colors.

[0039] Step 2: According to the classification results, perform free defect simulation and overall shape defect simulation respectively to obtain corresponding defect results.

[0040] The free defect simulation is to extract defect features within the recognition range of free defect labels and perform data multiplication on them.

[0041] The free defect simulation specifically includes the following processes:

[0042] 1) Defect feature extraction: Automatically extract defect feature information within the recognition range of free defect labels through a deep learning network model (i.e., an artificial intelligence classification algorithm model based on a convolutional neural network, with several multi-channel convolutional layers), including length, width, radian, connectivity, area, shape, and density. The deep learning network model used for defect feature extraction does not perform parameter iteration during the training process.

[0043] 2) Data multiplication: After extracting local features within the free defect area, according to the pre-input parameters, rotate, translate, scale, discretely sample, randomly sample, and edge clip the defects, and then tile them on a blank background image to achieve data multiplication. The result is shown in Figure 4 .. Different pre-input parameters result in different effects of random tiling and multiplication of the image. The pre-input parameters in this step are adjusted according to the verification results during the subsequent model training process.

[0044] The overall shape defect simulation is to identify the shape contour, extract the main elements and main points that make up the shape, and perform data multiplication based on the main points.

[0045] The overall shape defect simulation specifically includes the following processes:

[0046] 1) Extract the main points and main elements of the shape: Automatically identify the core area where the shape is located through a deep learning network model (i.e., an artificial intelligence classification algorithm model based on a convolutional neural network, with several multi-channel convolutional layers, which has the functions of area recognition and target classification), then remove the background interference area on this basis, extract the main points and main elements that make up the shape, and calculate the probability estimate of the occurrence of defects at each point in the image to form a probability estimate depth map, as shown in Figure 5 The deep learning network model used to extract the main points and main elements of the shape does not perform parameter iteration during the training process.

[0047] 2) Data multiplication: Multiply the data by setting parameters such as sampling rate, density, degree, rotation, and translation, and then tile it on a blank background image, as shown in Figure 6 The pre-input parameters in this step are adjusted according to the verification results during the subsequent model training process.

[0048] Step 3: Fuse the different types of defect results obtained in Step 2 to generate defect pictures.

[0049] Defect fusion is to randomly fuse free defects and overall shape defects. Random fusion means randomly selecting and piecing together. During the piecing together, noise is added as a simulation of the real situation. At the same time, according to the sample distribution setting in the parameter setting, OK pictures are generated. OK pictures refer to pictures without defects but only noise. After all defect pictures are generated, corresponding annotations are made on the defect pictures as model training files.

[0050] Step 4: Use the defect pictures obtained in Step 3 to train the recognition model.

[0051] Adopt a convolutional neural network to extract the features of all generated samples, and correlate and compare them with the annotated features. After training is completed, model verification is carried out.

[0052] Step 5: After the recognition model is trained, carry out model verification.

[0053] Use the original superimposed picture samples and defect pictures to verify the trained convolutional neural network. Compare the autocorrelation functions of each feature layer to measure the similarity between the two. Set a similarity threshold. By comparing with the similarity threshold, if it meets the similarity threshold, it is determined that the defect picture simulation is successful; if it does not meet the similarity threshold, it is determined that the defect picture simulation fails.

[0054] When it is determined that the defect picture simulation fails, the following parameter optimization process is entered: construct an optimization model of the input parameters, the objective function and the similarity, that is, use the Euclidean distance between the objective function and the parameters as the optimization objective, and use the Newton iteration method to optimize the input parameters. After optimizing the input parameters, re - simulate all defect pictures, and re - input the simulation results into the training network for iterative training. The training termination condition is: until it meets the similarity threshold and it is determined that the defect picture simulation is successful, which is regarded as the recognition model having excellent recognition ability. Then, output the recognition model.

[0055] Step 6: Deploy the recognition model obtained in Step 5 in the automatic determination process of superimposed picture recognition, and the superimposed picture samples to be processed can be automatically determined for superimposed picture recognition.

[0056] The above embodiments are only the preferred embodiments of the present invention and are not used to limit the protection scope of the present invention. For those skilled in the art, various changes and modifications can be made to the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A small-sample expansion and recognition method for wafer overlay analysis, characterized in that It includes the following steps: Step 1: Automatically classify the overlay samples to obtain different classification results; Step 2: Perform corresponding defect simulations according to the classification results to obtain corresponding defect results; Step 3: Fuse the different types of defect results obtained in Step 2 to generate defect pictures; Step 4: Use the defect pictures obtained in Step 3 to train the recognition model; Step 5: After the recognition model is trained, perform model verification; Step 6: Use the recognition model output in Step 5 to automatically determine the overlay recognition of the overlay samples to be processed.

2. The small sample expansion and recognition method for wafer overlay analysis according to claim 1, characterized in that: When the overlay sample enters the automatic classification process of Step 1, there is a parameter setting operation, and the parameters involved include defect range, blur degree, sampling frequency, maximum natural estimation degree, self-test autocorrelation, and passing threshold.

3. The small sample expansion and recognition method for wafer overlay analysis according to claim 1, characterized in that: In Step 1, the automatic classification classifies the overlay samples into free defects and overall shape defects according to the size and connectivity of the defects. The free defects include scratches and fractures, and the overall shape defects include horizontal lines, vertical lines, and concentric circles.

4. The small sample expansion and recognition method for wafer overlay analysis according to claim 3, characterized in that: Step 2 performs corresponding free defect simulation and overall shape defect simulation according to the classification results, The free defect simulation is to extract the defect features within the recognition range of the free defect label and multiply the data thereof, The overall shape defect simulation is to identify the shape contour, extract the main elements and main points constituting the shape, and multiply the data based on the main points.

5. The small sample expansion and recognition method for wafer overlay analysis according to claim 3, characterized in that: The free defect simulation specifically includes the following process: 1) Extract the defect feature information within the recognition range of the free defect label, including length, width, radian, connectivity, area, shape, and density; 2) Rotate, translate, scale, discretely sample, randomly sample, and edge cut the defects, and then paste them on a blank background image.

6. The small sample expansion and recognition method for wafer overlay analysis according to claim 3, characterized in that: The overall shape defect simulation specifically includes the following process: 1) Identify the core area where the shape is located, eliminate the background interference area, extract the main points and main elements constituting the shape, and calculate the probability estimate of each point in the image having a defect to form a probability estimate depth map; 2) Multiply the data by setting the sampling rate, density, degree, rotation, and translation, and then paste them on a blank background image.

7. The small sample expansion and recognition method for wafer overlay analysis according to claim 3, characterized in that: In Step 3, the defect fusion is to randomly fuse the free defects and the overall shape defects.

8. The small sample expansion and recognition method for wafer overlay analysis according to claim 7, characterized in that: The random fusion is random selection and splicing. During splicing, noise is added to simulate the real situation. At the same time, OK pictures are also generated. The OK pictures refer to pictures with only noise and no defects. After all defective pictures are generated, the defective pictures are correspondingly labeled as model training files.

9. The small sample expansion and recognition method for wafer overlay analysis according to claim 8, characterized in that: In step 4, for the training of the recognition model, a convolutional neural network is used to extract the features of all generated samples and correlate and compare them with the labeled features. After the training is completed, model verification is carried out. In step 5, for the verification of the recognition model, the original overlay samples and defective pictures are used to verify the trained convolutional neural network. The similarity between the two is measured by comparing the autocorrelation functions of each feature layer. A similarity threshold is set. By comparing with the similarity threshold, if it meets the similarity threshold, it is determined that the simulation of the defective picture is successful; if it does not meet the similarity threshold, it is determined that the simulation of the defective picture fails.

10. The small sample expansion and recognition method for wafer overlay analysis according to claim 9, characterized in that: In step 5, when it is determined that the simulation of the defective picture fails, the following parameter optimization link is entered: construct an optimization model of the input parameters, the objective function and the similarity, use the Newton iteration method to optimize the input parameters. After optimizing the input parameters, re-perform the simulation process on all defective pictures, and re-enter the simulation results into the training network for iterative training until it meets the similarity threshold and it is determined that the simulation of the defective picture is successful, and then output the recognition model.

Images