A hybrid type wafer map defect recognition method based on feature enhanced residual network

Abstract

The application provides a novel feature enhancement residual network for hybrid wafer map defect recognition and classification. The method comprises the following steps: obtaining a wafer map defect dataset, and setting the wafer map dataset to a uniform size; then filtering the wafer map dataset to remove noise defect points on the wafer map; dividing the wafer map dataset into a training set and a test set; adopting the proposed feature enhancement residual network as a backbone network to construct a recognition model, training the model by using the training set, and verifying the model performance by using the test set. Experimental results show that the method can effectively recognize wafer map defects, and improve the accuracy and robustness of hybrid wafer map defect recognition.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of machine vision and deep learning, and in particular to a hybrid wafer map defect identification method based on feature enhancement residual network. BACKGROUND

[0002] In the process of manufacturing semiconductor chips, wafer testing is a crucial link. By accurately identifying the type of wafer defects, the problem in the manufacturing process can be precisely located, thereby improving the yield of chip production. In recent years, many researchers have used deep learning methods to study wafer map defect identification, but most deep learning models used to classify wafer map defects are complex, and as the model deepens, it will cause gradient dispersion and overfitting problems, resulting in a decrease in accuracy. At the same time, the defect pattern size, distribution position, shape, etc. of complex wafer maps have great differences, which increases the difficulty of wafer map defect identification. How to extract more important wafer map defect features and reduce the parameter quantity and computational quantity of the model is the key to solving the above problems. SUMMARY

[0003] The present application proposes a hybrid wafer map defect identification method based on feature enhancement residual network, which can reduce the computational quantity and parameter quantity of the model and improve the accuracy of hybrid wafer map defect identification.

[0004] The present application is implemented in the following way: a hybrid wafer map defect identification method based on feature enhancement residual network, the method comprising the following steps:

[0005] S1: Obtain a wafer map defect dataset, and use a bilinear interpolation method to set the size of the wafer map defect dataset to a uniform size;

[0006] S2: Use a mean filter method to filter the wafer map defect dataset to remove noise defect points on the wafer map;

[0007] S3: Divide the wafer map defect dataset into a training set and a test set;

[0008] S4: Use a feature enhancement residual network as the network, and use a cross-entropy loss function as the loss function of the output layer to construct a deep learning network model for wafer map defect identification;

[0009] S5: Use the training set to train the model, and use the test set to evaluate the performance of the model.

[0010] Further, the step S4 is based on the Pytorch 1.7.1 framework and uses Python to design the feature enhancement residual network.

[0011] Further, the step S4 adopts a feature enhancement residual network as a backbone network, adopts a cross-entropy loss function as a loss function of an output layer to construct a deep learning network model for wafer map defect recognition, and the method comprises the following steps of:

[0012] S21: receiving an input wafer map and performing feature extraction using a conventional convolution, containing two convolution layers and a pooling layer, the convolution layer 1 adopts a 11*11 convolution kernel and a ReLU activation function to perform convolution and linear activation on the input training set image, and then outputs a 64*35*35 feature matrix X1 after pooling by the pooling layer, X1 is input to the convolution layer 2, the convolution layer 2 adopts a 3*3 convolution kernel and a ReLU activation function to perform convolution and linear activation on X1, and outputs a 128*35*35 feature matrix X2;

[0013] S22: inputting the feature matrix X2 to a feature enhancement residual module for feature extraction, and outputting a 128*35*35 feature matrix X3;

[0014] S23: receiving the input feature matrix X3 and performing feature extraction using a conventional convolution, containing two convolution layers and a pooling layer, the convolution layer 3 uses a 3*3 convolution kernel, a ReLU activation function and batch normalization processing to perform convolution, linear activation and normalization processing on the input feature matrix X3, and outputs a 512*18*18 feature matrix X4, X4 is input to the convolution layer 4, the convolution layer 4 adopts a 3*3 convolution kernel, a ReLU activation function and batch normalization processing to perform convolution, linear activation and normalization processing on X4, and then outputs a 1024*1*1 feature matrix X5 after pooling by the pooling layer;

[0015] S24: inputting the feature matrix X5 to a full connection layer, the number of neurons of the full connection layer is 1024, 512, 512 and 38 respectively, and then obtaining a prediction output through a Softmax function.

[0016] Further, the step S22 inputs the feature matrix X2 to the feature enhancement residual module for feature extraction, and the operation steps are as follows:

[0017] S31: The feature matrix X2 is subjected to feature extraction by two channel-enhanced Ghost convolution blocks and two deformable convolution residual blocks respectively, the structure of the channel-enhanced Ghost convolution block is Ghost convolution, ReLU activation function and batch normalization layer, the structure of the deformable convolution residual block is 1*1 regular convolution, 3*3 deformable convolution and 1*1 regular convolution, the feature matrix X2 is subjected to the first and second channel-enhanced Ghost convolution blocks to obtain output feature matrices U1 and U2 respectively, and the feature matrix X2 is subjected to the first and second deformable convolution residual blocks to obtain output feature matrices U3 and U4 respectively;

[0018] S32: The feature matrices U1 and U2 are subjected to element-wise addition to obtain a fused feature matrix U12;

[0019] S33: The feature matrices U3 and U4 are subjected to element-wise addition to obtain a fused feature matrix U34;

[0020] S34: The feature matrices U2 and U3 are subjected to element-wise addition to obtain a fused feature matrix U23;

[0021] S35: The feature matrix U23 is subjected to global average pooling, a full connection layer and a Sigmoid function to obtain a feature matrix U23';

[0022] S36: The feature matrix U12 is multiplied by the feature matrix U23' to obtain a feature-enhanced matrix U12';

[0023] S37: The feature matrix U34 is multiplied by the feature matrix U23' to obtain a feature-enhanced matrix U34';

[0024] S38: The feature matrix U12' is subjected to element-wise addition with the feature matrix U34', and is subjected to a convolution with a 1*1 convolution kernel to obtain the output X3 in step S22.

[0025] Further, in the step S31, the Ghost convolution in the channel-enhanced Ghost convolution block is structured by adding a maximum pooling layer, an average pooling layer, a regular convolution layer and a sigmoid function to the Identity branch of the Ghost convolution, for performing channel feature enhancement.

[0026] Further, in the step S31, the deformable convolution residual block uses a spatial feature enhancement module introduced by a jump connection before the first convolution layer and after the second convolution layer based on a standard residual block, and the second convolution layer is a deformable convolution layer.

[0027] Compared with the prior art, the present application has the beneficial effects that:

[0028] The application takes mixed type wafer map defect recognition as a classification problem, and constructs a deep learning network model based on the proposed feature enhancement residual network. The proposed feature enhancement residual network uses a channel enhancement Ghost convolution block and a deformable convolution residual block to respectively extract features of wafer map defects. Compared with the conventional convolution, the deformable convolution can adaptively extract wafer map fault defects, and can effectively solve the problem that the wafer map defect patterns, distribution positions and shapes have great differences. At the same time, the structure design using residual connection reduces the risk of model gradient dispersion and overfitting. Experiments show that the feature enhancement residual network proposed in the application can effectively reduce the parameter quantity and calculation quantity of the model, and improve the accuracy of mixed type wafer map defect recognition. BRIEF DESCRIPTION OF DRAWINGS

[0029] Figure 1 The network structure of the mixed type wafer map defect recognition method based on the feature enhancement residual network;

[0030] Figure 2 The channel enhancement Ghost convolution of the application;

[0031] Figure 3 The deformable convolution residual block of the application;

[0032] Figure 4 The loss function change curve graph in the network model training process of the application;

[0033] Figure 5 The accuracy curve graph of the network model of the application. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical scheme and advantages of the application clearer, the application will be further described in detail below combined with embodiments. It should be understood that the specific embodiments described herein are only used to explain the application, and are not used to limit the application.

[0035] Referring to Figure 1 The mixed type wafer map defect recognition method based on the feature enhancement residual network comprises the following steps:

[0036] S1: Obtain a wafer map defect dataset, and use a bilinear interpolation method to set the size of the wafer map defect dataset to a uniform size;

[0037] S2: Use a mean filter method to filter the wafer map defect dataset, and remove the noise defect points on the wafer map;

[0038] S3: Divide the wafer map defect dataset into a training set and a test set;

[0039] S4: A feature-enhanced residual network is used as the backbone network, and a cross-entropy loss function is used as the loss function of the output layer to construct a deep learning network model for wafer map defect recognition.

[0040] S5: The model is trained using the training set, and the performance of the model is evaluated using the test set.

[0041] In step S1, the wafer map defect data set size is set to 224x224.

[0042] In step S3, the wafer map defect data set is divided into a training set and a test set in a ratio of 8:2.

[0043] In step S4, a feature-enhanced residual network is designed using Python based on the Pytorch 1.7.1 framework.

[0044] Further, in step S4, a feature-enhanced residual network is used as the backbone network, and a cross-entropy loss function is used as the loss function of the output layer to construct a deep learning network model for wafer map defect recognition, and the model is trained and verified. The feature-enhanced residual network deep learning network model includes a regular convolution layer, a pooling layer, a feature-enhanced residual module, and a plurality of fully connected layers, and the operation steps are as follows:

[0045] S21: A regular convolution is used to receive the input wafer map and perform feature extraction, including two convolution layers and a pooling layer. Convolution layer 1 uses an 11x11 convolution kernel and a ReLU activation function to convolve and linearly activate the input training set image, and then outputs a 64x35x35 feature matrix X1 after pooling. X1 is input to convolution layer 2, which uses a 3x3 convolution kernel and a ReLU activation function to convolve and linearly activate X1, outputting a 128x35x35 feature matrix X2;

[0046] S22: The feature matrix X2 is input to the feature-enhanced residual module for feature extraction, outputting a 128x35x35 feature matrix X3;

[0047] S23: A regular convolution is used to receive the input feature matrix X3 and perform feature extraction, including two convolution layers and a pooling layer. Convolution layer 3 uses a 3x3 convolution kernel, a ReLU activation function, and batch normalization to convolve, linearly activate, and normalize the input feature matrix X3, outputting a 512x18x18 feature matrix X4. X4 is input to convolution layer 4, which uses a 3x3 convolution kernel, a ReLU activation function, and batch normalization to convolve, linearly activate, and normalize X4, and then outputs a 1024x1x1 feature matrix X5 after pooling.

[0048] S24: The feature matrix X5 is input to the fully connected layer. The number of neurons in the fully connected layer is 1024, 512, 512, and 38 respectively, and then the prediction output is obtained through the Softmax function.

[0049] Furthermore, in step S22, the feature matrix X2 is input to the feature enhancement residual module for feature extraction, and the operation steps are as follows:

[0050] S31: The feature matrix X2 is respectively subjected to two channel-enhanced Ghost convolution blocks and two deformable convolution residual blocks for feature extraction. The structure of the channel-enhanced Ghost convolution block is Ghost convolution, ReLU activation function and batch normalization layer. The structure of the deformable convolution residual block is 1×1 regular convolution, 3×3 deformable convolution, and 1×1 regular convolution. The feature matrix X2 is respectively subjected to the first and second channel-enhanced Ghost convolution blocks to obtain the output feature matrices U1 and U2. The feature matrix X2 is respectively subjected to the first and second deformable convolution residual blocks to obtain the output feature matrices U3 and U4.

[0051] S32: The feature matrices U1 and U2 are added element by element to obtain a fused feature matrix U12;

[0052] S33: Feature matrices U3 and U4 are added element by element to obtain a fused feature matrix U34;

[0053] S34: The feature matrices U2 and U3 are added element by element to obtain a fused feature matrix U23;

[0054] S35: The feature matrix U23 is obtained by global average pooling, full connection layer, and Sigmoid function to obtain the feature matrix U23`;

[0055] S36: Multiply the feature matrix U12 by the feature matrix U23' to obtain a feature-enhanced matrix U12';

[0056] S37: Multiply the feature matrix U34 by the feature matrix U23' to obtain a feature-enhanced matrix U34';

[0057] S38: The feature matrix U12' and the feature matrix U34' are added element by element, and a convolution with a convolution kernel size of 1×1 is performed to obtain the output X3 in step S22.

[0058] Furthermore, the Ghost convolution in the channel-enhanced Ghost convolution block in step S31 is performed, see Figure 2 As shown in the figure, the structure is to add a maximum pooling layer, an average pooling layer, a regular convolution layer and a sigmoid function to the Identity branch of the Ghost convolution for channel feature enhancement.

[0059] Further, the step S31 can deform the convolution residual block, see Figure 3 As shown in the figure, on the basis of the standard residual block, a spatial feature enhancement module is introduced using a jump connection before the first layer of convolution layers and after the second layer of convolution layers, and the second layer of convolution layers is a deformable convolution layer.

[0060] The wafer map defect recognition is taken as a classification problem, and a deep learning network model is constructed based on the proposed feature enhancement residual network. The proposed feature enhancement residual network uses a channel enhancement Ghost convolution block and a deformable convolution residual block to respectively extract features of wafer map defects. The deformable convolution can adaptively extract wafer map fault defects compared with the conventional convolution, and can effectively solve the problem that wafer map defect patterns, distribution positions, shapes and the like have great differences. At the same time, the structure design using the residual connection reduces the risk of model gradient dispersion and overfitting. Experiments show that the feature enhancement residual network proposed in the application can effectively reduce the parameter quantity and calculation quantity of the model, and improve the accuracy of mixed type wafer map defect recognition.

[0061] The application will be further described in detail below in combination with an actual mixed type wafer map defect data set Mixed38WM. The wafer map defect recognition method can be expressed as the following steps:

[0062] First step: obtain the wafer map defect data set Mixed38WM, and perform image preprocessing to construct a wafer map defect image data set. The preprocessing link performs the following three processing steps:

[0063] Obtain the wafer map defect data set Mixed38WM, and set each wafer map defect picture to a size of 224x224, a total of 38015 pictures;

[0064] Use the mean filtering algorithm to filter process the 38015 wafer map images;

[0065] According to a proportion of 8:2, randomly divide the 38015 wafer map defect data sets into a training set and a test set, wherein the training set has 30413 pictures, and the test set has 7602 pictures.

[0066] Second step: based on the Pytorch 1.7.1 framework, use Python to design a feature enhancement residual network, use the feature enhancement residual network as a backbone network, use a cross entropy loss function as a loss function of an output layer to construct a wafer map defect recognition deep learning network model, and perform training and verification.

[0067] The feature enhancement residual network deep learning network model comprises a regular convolution layer, a pooling layer, a feature enhancement residual module, and a plurality of fully connected layers, and the operation steps are as follows:

[0068] S21: The input wafer map is received and feature extraction is performed using a regular convolution, including two convolution layers and a pooling layer, convolution layer 1 adopts a 11*11 convolution kernel and a ReLU activation function to perform convolution and linear activation on the input training set image, and then outputs a 64*35*35 feature matrix X1 after pooling by the pooling layer, X1 is input to convolution layer 2, convolution layer 2 adopts a 3*3 convolution kernel and a ReLU activation function to perform convolution and linear activation on X1, and outputs a 128*35*35 feature matrix X2;

[0069] S22: The feature matrix X2 is input to the feature enhancement residual module for feature extraction, and a 128*35*35 feature matrix X3 is output;

[0070] S23: The input feature matrix X3 is received and feature extraction is performed using a regular convolution, including two convolution layers and a pooling layer, convolution layer 3 uses a 3*3 convolution kernel, a ReLU activation function and batch normalization processing to perform convolution, linear activation and normalization processing on the input feature matrix X3, and outputs a 512*18*18 feature matrix X4, X4 is input to convolution layer 4, convolution layer 4 adopts a 3*3 convolution kernel, a ReLU activation function and batch normalization processing to perform convolution, linear activation and normalization processing on X4, and then outputs a 1024*1*1 feature matrix X5 after pooling by the pooling layer;

[0071] S24: The feature matrix X5 is input to the fully connected layer, the number of neurons of the fully connected layer is 1024, 512, 512 and 38 respectively, and then the prediction output is obtained through the Softmax function.

[0072] Figure 4 The figure shows the change of the loss value of the deep learning network model proposed in the application, and it can be seen that the loss value of the model proposed in the application decreases stably. Figure 5 The figure shows the change of the accuracy of the deep learning network model proposed in the application, and the accuracy on the test set finally exceeds 98.8%, so the application of the wafer map defect recognition model proposed in the application fully demonstrates the effectiveness and superiority of the application in the field of mixed type wafer map defect recognition, and greatly improves the accuracy of mixed type wafer map defect recognition.

[0073] The above only describes the preferred embodiments of the application and is not intended to limit the application.

Claims

1. A wafer image defect recognition method based on feature-enhanced residual network, characterized in that: The method comprises the following steps: S1: Obtain a wafer image defect dataset and use a bilinear interpolation method to set the size of the wafer image defect dataset to a uniform size; S2: Use the mean filtering method to filter the wafer image defect dataset to remove noise defect points on the wafer image; S3: Divide the wafer image defect dataset into training and test sets; S4: A deep learning network model for wafer image defect recognition was constructed using a feature-enhanced residual network as the backbone network and a cross-entropy loss function as the loss function of the output layer. S5: Use the training set to train the model and use the test set to evaluate the performance of the model; The step S4 uses a feature-enhanced residual network as the backbone network and a cross-entropy loss function as the loss function of the output layer to construct a deep learning network model for wafer image defect recognition, including: using Python to design and build a feature-enhanced residual network deep learning network model, and performing training and testing. The feature-enhanced residual network deep learning network model includes a conventional convolutional layer, a pooling layer, a feature-enhanced residual module, and multiple fully connected layers, and its operation steps are as follows: S21: Use conventional convolution to receive the input wafer image and perform feature extraction. It includes two convolution layers and one pooling layer. Convolution layer 1 uses an 11×11 convolution kernel and ReLU activation function to perform convolution and linear activation on the input training set image. Then, after pooling by the pooling layer, it outputs a 64×35×35 feature matrix X1. X1 is input to convolution layer 2. Convolution layer 2 uses a 3×3 convolution kernel and ReLU activation function to perform convolution and linear activation on X1, and outputs a 128×35×35 feature matrix X2. S22: The feature matrix X2 is input to the feature enhancement residual module for feature extraction, and the output is a 128×35×35 feature matrix X3 S23: Use conventional convolution to receive the input feature matrix X3 and perform feature extraction. It includes two convolution layers and one pooling layer. Convolution layer 3 uses a 3×3 convolution kernel, ReLU activation function and batch normalization to perform convolution, linear activation and normalization on the input feature matrix X3, and outputs a 512×18×18 feature matrix X4. X4 is input to convolution layer 4. Convolution layer 4 uses a 3×3 convolution kernel, ReLU activation function and batch normalization to perform convolution, linear activation and normalization on X4. Then, after pooling by the pooling layer, it outputs a 1024×1×1 feature matrix X5. S24: The feature matrix X5 is input to the fully connected layer. The number of neurons in the fully connected layer is 1024, 512, 512, and 38 respectively. Then, the prediction output is obtained through the Softmax function. In step S22, the feature matrix X2 is input to the feature enhancement residual module for feature extraction, and the steps are as follows: S31: The feature matrix X2 is extracted by two channel-enhanced Ghost convolution blocks and two deformable convolution residual blocks. The structure of the channel-enhanced Ghost convolution block is Ghost convolution, ReLU activation function and batch normalization layer. The structure of the deformable convolution residual block is 1×1 regular convolution, 3×3 deformable convolution, and 1×1 regular convolution. The feature matrix X2 is passed through the first and second channel-enhanced Ghost convolution blocks to obtain the output feature matrices U1 and U2 respectively. The feature matrix X2 is passed through the first and second deformable convolution residual blocks to obtain the output feature matrices U3 and U4 respectively. S32: The feature matrices U1 and U2 are added element by element to obtain a fused feature matrix U12; S33: The feature matrices U3 and U4 are added element by element to obtain a fused feature matrix U34; S34: The feature matrices U2 and U3 are added element by element to obtain a fused feature matrix U23; S35: The feature matrix U23 is obtained by global average pooling, full connection layer, and Sigmoid function to obtain the feature matrix U23`; S36: Multiply the feature matrix U12 by the feature matrix U23' to obtain the feature enhanced matrix U12'; S37: Multiply the feature matrix U34 by the feature matrix U23' to obtain the feature enhanced matrix U34'; S38: The feature matrix U12' and the feature matrix U34' are added element by element, and a convolution with a convolution kernel size of 1×1 is performed to obtain the output X3 in step S22.

2. The method according to claim 1, characterized in that The step S4 is based on the Pytorch 1.7.1 framework and uses Python to design a feature-enhanced residual network.

3. The method according to claim 1, characterized in that The Ghost convolution in the channel enhancement Ghost convolution block in step S31 has a structure of adding a maximum pooling layer, an average pooling layer, a conventional convolution layer and a sigmoid function on the Identity branch of the Ghost convolution to perform channel feature enhancement.

4. The method according to claim 1, wherein The deformable convolution residual block in step S31 is based on the standard residual block. A spatial feature enhancement module is introduced using a jump connection before the first convolution layer and after the second convolution layer, and the second convolution layer is a deformable convolution layer.

Images