Wafer image fault pattern recognition method based on attention spatial pyramid pooling

Abstract

The present invention proposes a wafer image fault pattern recognition method based on attention space pyramid pooling, which combines deep convolutional neural network (DCNN) and attention space pyramid pooling (ESPP) to construct a multi-scale feature fusion deep learning model for wafer image fault pattern recognition. The model proposed in the present invention is implemented based on the Pytorch framework. By using DCNN, wafer image fault features can be better extracted. The introduction of ESPP can well solve the inter-class similarity and intra-class dissimilarity problems of wafer image fault patterns. The use of multi-scale feature fusion can fuse the shallow features of wafer image defects with the deep features, thereby more accurately identifying wafer image fault patterns. The present invention can improve the recognition accuracy of wafer image faults in the semiconductor manufacturing process, is conducive to improving the yield of wafer production, and has great practical significance for the development of national semiconductor manufacturing technology.

Description

Technical Field

[0001] The present invention belongs to the field of wafer images generated in actual semiconductor production, and in particular relates to wafer image fault pattern recognition. Background Art

[0002] Semiconductor chips are the foundation for the rapid development of artificial intelligence and a key technology driving scientific and technological progress. Despite the advanced manufacturing technology and specialized technicians involved in wafer fabrication, the increasing complexity of chip design and the potential for faulty die due to factors such as incorrect parameter settings during production remain difficult to avoid. On wafer maps, defective die that fail electrical testing tend to exhibit specific spatial patterns. Different spatial patterns indicate different causes of the defect. By analyzing the defect type, problems can be traced back to the production process.

[0003] Traditionally, wafer image defect recognition and classification is performed manually by experienced professionals. However, due to subjectivity and prior knowledge, classification results vary. In recent years, deep learning has rapidly developed due to its ability to automatically extract rich image features. Convolutional neural networks (CNNs) are a typical example of deep learning. CNNs can automatically extract important image features and perform pattern recognition and classification without requiring domain knowledge. In wafer image defect pattern recognition and classification, a wafer image dataset is used as input for the CNN, and the CNN output provides the identified and classified defect types. CNNs can automatically extract wafer defect pattern features and classify them, significantly improving upon traditional manual and machine learning methods. However, surface defects on wafer images vary in size and shape. CNNs use convolutional kernels and pooling filters of the same size to extract features, resulting in missing information on some wafer image features and reducing the accuracy of wafer image fault recognition. Summary of the Invention

[0004] The purpose of the present invention is to provide a wafer image fault pattern recognition method based on attention space pyramid pooling for wafer image fault pattern recognition, so as to improve the accuracy of wafer image fault pattern recognition.

[0005] This method is implemented as follows: a wafer image fault pattern recognition method based on attention space pyramid pooling, the method includes the following steps:

[0006] Step 1. Preprocess the wafer image dataset and divide it into training and test sets.

[0007] Step 2. Build a deep convolutional neural network (DCNN) based on convolutional layers, pooling layers, batch normalization layers, and SE Block as the backbone network for feature extraction.

[0008] Step 3. Design an attention spatial pyramid pooling module based on the spatial pyramid pooling module;

[0009] Step 4. Combine DCNN and the attention space pyramid pooling module to build a multi-scale feature fusion deep learning model;

[0010] Step 5. Use the training set to train the deep learning model, use the test set to test the deep learning model, and use accuracy, precision, recall, and F1-Score to evaluate the model performance.

[0011] Furthermore, in step 1, the pre-processing process of the wafer image includes the following steps:

[0012] The wafer image is filtered and denoised using the mean filtering method;

[0013] Convert the wafer image to an image with a size of 224×224 and 3 pixel channels;

[0014] The wafer image dataset is divided into training set and test set in a ratio of 8:2.

[0015] Furthermore, in step 2, the deep convolutional neural network DCNN constructed includes a conventional convolution block and three channel enhanced convolution blocks;

[0016] The conventional convolution block is used to receive the input of the wafer image dataset, which includes a convolution layer, a batch normalization layer, and a ReLU activation function layer. The wafer image input to the conventional convolution block has a dimension of 3×224×224, the convolution layer uses a convolution kernel size of 11×11, and the output feature vector dimension of the conventional convolution block is 64×35×35.

[0017] Furthermore, the three channel enhancement convolution blocks, the channel enhancement convolution block uses SE Block to enhance the channel attention of convolution, and its structure is convolution layer, batch normalization layer, ReLU layer, SE Block, convolution layer, batch normalization layer, ReLU layer;

[0018] The first convolution block receives an input feature vector of dimension 64×35×35 and outputs a feature vector of dimension 128×35×35;

[0019] The second convolution block receives an input feature vector of dimension 128×35×35 and outputs a feature vector of dimension 256×17×17;

[0020] The third convolutional block receives an input feature vector of dimension 256×17×17 and outputs a feature vector of dimension 512×17×17.

[0021] Furthermore, the outputs of the three convolutional blocks are input into the attention space pyramid pooling module through skip connections for multi-scale feature fusion; the attention space pyramid pooling receives the feature vectors output by the previous three convolutional blocks, and the feature vector dimensions are 128×35×35, 256×17×17, and 512×17×17, respectively.

[0022] Furthermore, in step 3, the designed attention space pyramid pooling receives a feature vector with an input dimension of C×H×W and performs a pooling operation on the input feature vector using n pooling filters. At the same time, the feature vector with an input dimension of C×H×W is dimensionally compressed to obtain a feature vector with a dimension of C×1×1, and the feature vector is multiplied by the channel dimension with the feature vector output by the n pooling filters.

[0023] Furthermore, the dimensionality compression of the feature vector with an input dimension of C×H×W includes using global maximum pooling to reduce the dimension of the input feature vector C×W×H to C×1×1, and passing through a fully connected layer, a batch normalization layer and a ReLU activation function layer to obtain a feature vector dimension of C×1×1.

[0024] Furthermore, the feature vector output by the attention space pyramid pooling module is stretched in one dimension to obtain a one-dimensional feature vector as the input of the fully connected layer. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 is a schematic diagram of wafer image failure mode detection according to the present invention;

[0026] Figure 2 It is the deep learning model proposed by the present invention;

[0027] Figure 3 Is it the attention space pyramid pooling module in the deep learning model proposed in this invention? DETAILED DESCRIPTION

[0028] The present invention will be further described below with reference to the accompanying drawings.

[0029] The present invention provides a wafer image fault pattern recognition method based on attention space pyramid pooling for wafer image fault pattern recognition. The specific process is shown in the attached Figure 1As shown, the wafer image is pre-processed for filtering, denoising, size processing and data set division to obtain a wafer image image with a dimension of 3×224×224, as shown in S101; the training set is used to train a deep learning model based on attention space pyramid pooling, as shown in S102; the deep learning model is trained on the training data set to obtain a model with the optimal effect, and the test data set is used for testing. The deep learning model gives the defect category of the test image, as shown in S103.

[0030] The deep learning model proposed in this invention is as follows Figure 2 As shown, it includes five parts, namely S201, S202, S203, S204, and S205.

[0031] S201 is a conventional convolutional block. Its input is a wafer image dataset, which undergoes preprocessing, including filtering, denoising, resizing, and dataset partitioning, resulting in a wafer image with a dimension of 3×224×224. The S201 structure consists of a convolutional layer, a batch normalization layer, and a ReLU activation layer. The convolutional layer uses an 11×11 kernel size. Using a large kernel expands the receptive field and reduces the dimensionality of the input image, thereby reducing the number of model parameters. The input feature vector of the conventional convolutional block is 3×224×224, and the output feature vector is 64×35×35. After processing by the conventional convolutional block, the wafer image data is fed into the subsequent channel-enhanced convolutional block for further feature extraction, as shown in S202.

[0032] S202 contains three channel enhancement convolution blocks. The channel enhancement convolution block uses SE Block to enhance the channel attention of convolution. The internal structure of the three enhanced convolution blocks is the same, consisting of two convolution layers, two batch normalization layers, two ReLU layers and SE-Block modules, as shown in S203.

[0033] The channel-enhanced convolution block Conv Block1 receives the feature vector output from S201, the feature vector size is 64×35×35, and after processing by the channel-enhanced convolution block Conv Block1, the output feature vector size is 128×35×35; the output of the channel-enhanced convolution block Conv Block1 is divided into two paths, path one outputs to the channel-enhanced convolution block ConvBlock2, and path two outputs to the subsequent attention space pyramid pooling module S204.

[0034] The channel-enhanced convolution block Conv Block2 receives the feature vector output from the channel-enhanced convolution block Conv Block1. The feature vector size is 128×35×35. After being processed by the channel-enhanced convolution block Conv Block2, the output feature vector size is 256×17×17. The output of the channel-enhanced convolution block Conv Block2 is divided into two paths. Path one outputs to the channel-enhanced convolution block Conv Block3, and path two outputs to the subsequent attention space pyramid pooling module S204.

[0035] The channel-enhanced convolution block Conv Block3 receives the feature vector output by the channel-enhanced convolution block Conv Block2. The feature vector size is 256×17×17. After being processed by the channel-enhanced convolution block Conv Block3, the output feature vector size is 512×17×17. The output of the channel-enhanced convolution block Conv Block3 is passed to the attention spatial pyramid pooling module as input.

[0036] The attention spatial pyramid pooling module proposed in the present invention, as shown in S204, has as input the output of the previous three channel-enhanced convolution blocks, and the feature vector sizes are: 128×35×35, 256×17×17, and 512×17×17, respectively.

[0037] Inside the attention space pyramid pooling module, each input feature vector is pooled using n pooling filters of different sizes. In this invention, n is set to 4, and the corresponding pooling filter sizes are 4×4, 3×3, 2×2, and 1×1, respectively.

[0038] The specific structure of the attention space pyramid pooling module is shown in the attached Figure 3 As shown;

[0039] S301 represents the feature vector of the input attention space pyramid pooling module, with a size of H×W×C;

[0040] S302 represents four pooling filters with sizes of 4×4, 3×3, 2×2, and 1×1 respectively. The pooling method used is the weighted average of average pooling and maximum pooling.

[0041] S303 represents a dimensionality reduction operation on the feature vector input to the attention space pyramid pooling module. First, the global maximum pooling is used to reduce the dimension of the feature vector of size H×W×C to 1×1×C. Then, a fully connected layer, a batch normalization layer, and a ReLU layer are used to increase the nonlinearity of the feature vector to obtain a feature vector of size 1×1×C.

[0042] Each feature vector after the pooling operation in S302 is nonlinearly processed using a 1×1 convolution layer, and the obtained feature vectors are multiplied at the channel level with the 1×1×C feature vectors obtained in S303.

[0043] The multiplied feature vectors are stretched and spliced ​​to obtain the one-dimensional feature vector shown in S305 as the output of the attention space pyramid pooling module, and the output is passed to the attached Figure 2 The fully connected layer shown in S205.

[0044] Attachment Figure 2 The fully connected layer shown in S205 receives the feature vector output from the attention space pyramid pooling module. Its structure consists of a fully connected layer with 26880 neurons and two fully connected layers with 4096 neurons. The output layer uses softmax to calculate the probability of the defect category of the wafer image.

[0045] The optimal deep learning model obtained after training on the training set is tested using the test set, and the model performance is evaluated using accuracy, precision, recall, and F1-Score.

[0046] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention.

Claims

1. A deep convolutional neural network wafer image fault pattern recognition method, characterized in that: A deep learning model was designed, which consists of the first convolutional layer, three convolutional blocks, an attention spatial pyramid pooling module (ESPP), and a fully connected layer. 1-1. The first convolutional layer receives the wafer image dataset as input and consists of one convolutional layer, one batch normalization layer, and one ReLU activation function layer. The wafer image input to the convolution block has a dimension of 3×224×224, a convolution kernel size of 11×11, and the output feature map of the convolution block has a dimension of 64×35×35. 1-2. The three convolution blocks have a structure of convolution layer, batch normalization layer, ReLU layer, SE Block, convolution layer, batch normalization layer, and ReLU layer; 1-3. The Attention Spatial Pyramid Pooling (ESPP) module is characterized by receiving a feature vector of input dimension C×H×W and performing a pooling operation on the input feature vector using n pooling filters of different sizes. Simultaneously, the feature vector of input dimension C×H×W is dimensionally compressed to obtain a feature vector of dimension C×1×1, and then channel-wise multiplied with the feature vector output by the n pooling filters. 1-4. The fully connected layer, whose input is the output of the attention space pyramid pooling module, and whose output is the number of categories of the wafer image dataset.

2. A deep convolutional neural network wafer image fault pattern recognition method as described in claim 1, characterized in that The three convolution blocks described, the first convolution block receives an input feature vector with a dimension of 64×35×35 and an output feature vector with a dimension of 128×35×35; the second convolution block receives an input feature vector with a dimension of 128×35×35 and an output feature vector with a dimension of 256×17×17; the third convolution block receives an input feature vector with a dimension of 256×17×17 and an output feature vector with a dimension of 512×17×17.

3. A deep convolutional neural network wafer image fault pattern recognition method as described in claim 1, characterized in that The attention space pyramid pooling has a pooling filter number n of 4, and the corresponding pooling filter sizes are 1, 2, 3, and 4 respectively; the dimensionality compression of the input feature vector includes reducing the dimension of the input feature vector C×W×H to C×1×1, and passing it through a fully connected layer, a batch normalization layer, and a ReLU layer to obtain a feature vector dimension of C×1×1.

4. A deep convolutional neural network wafer image fault pattern recognition method as described in claim 1, characterized in that The feature vectors output by the three convolution blocks are input into the attention space pyramid pooling module after feature fusion.

5. A deep convolutional neural network wafer image fault pattern recognition method as described in claim 1, characterized in that: The attention space pyramid pooling module stretches its output to obtain a one-dimensional feature vector as the input of the fully connected layer.

Images