Wafer surface defect classification method based on deep learning network

Abstract

This invention proposes a wafer surface defect classification method based on deep learning networks. This method can quickly and accurately identify wafer surface defect patterns and pinpoint their causes. The method aims to address the problems of high manual labor intensity and low detection efficiency in traditional wafer defect detection methods. The specific process includes: constructing and training a wafer defect detection and classification model based on the WM-811K dataset. The defect detection model is a binary classification model used to determine the presence of defects in the wafer image, while the classification model is a multi-class model used to determine the specific defect pattern category. The sample to be tested is input into the trained wafer defect detection and classification model to determine its defect pattern. Finally, by analyzing the causes of defect patterns in known samples, the causes of defect patterns in the sample to be tested are inferred, continuously optimizing the process flow and improving wafer product yield.

Description

Technical Field

[0001] This invention relates to the field of wafer surface defect detection and classification technology in industrial production processes, specifically to a wafer surface defect classification method based on deep learning networks. Background Technology

[0002] The semiconductor industry is a core industry of the information technology era and a crucial pillar supporting the current technological revolution and accelerating the transformation of related high-tech industries. It possesses fundamental and pioneering characteristics. For the manufacturing industry, the simultaneous development of output and quality is a major factor determining a manufacturing enterprise's survival and competitiveness in the market. Product yield significantly impacts manufacturing costs, profits, and competitiveness; higher yields result in lower unit production costs and stronger market competitiveness for the enterprise. High yield is one of the key goals that the semiconductor manufacturing industry consistently pursues in a fiercely competitive market environment.

[0003] Wafer manufacturing involves hundreds of steps and is a highly precise process. Problems in any step can result in wafer defects. Defective wafers are discarded and not allowed to proceed to the next stage of manufacturing to avoid increasing costs. Furthermore, wafer defects manifest as specific spatial patterns on the wafer map. These patterns contain information about anomalies occurring at the manufacturing level, such as thin film deposition issues, etching problems, uneven cleaning, uneven UV exposure, damage during wafer transport, or improper wafer handling. Traditional wafer inspection and classification rely on manual inspection, which is subjective, inefficient, and slow. Deep learning-based wafer defect classification is faster, more accurate, avoids misclassification, and saves significant labor. Summary of the Invention

[0004] This invention aims to propose a wafer surface defect classification method based on deep learning networks, which can be applied to the wafer manufacturing process. Traditional wafer inspection and classification relies on manual inspection, which is subjective, inefficient, and slow. The deep learning-based wafer defect classification method is faster and more accurate than manual classification, avoids misclassification, and can save a significant amount of labor.

[0005] A method for classifying wafer surface defects based on deep learning networks. The specific workflow of this invention is as follows:

[0006] Step 1: Based on the wafer dataset, convert the wafer feature matrix in the dataset into wafer images. Then, construct and train a wafer defect detection and classification model using the wafer dataset (e.g., the WM-811K dataset). The defect detection model is an improvement on the classic LeNet-5 network, with the loss function set to the cross-entropy function. Defect-free wafer images are marked as 0, and defective wafer images are marked as 1, determining whether defect patterns exist in the dataset samples. The defect classification model is designed based on residual blocks of the ResNet network to determine wafer defect patterns. Separable convolutions are introduced to reduce model parameters and accelerate network training; an attention mechanism is also introduced to improve model accuracy.

[0007] Step 2, Preprocessing and Data Augmentation Model: The dataset samples are augmented and standardized, and normalized. Based on the WM-811K dataset in Step 1, the dataset is standardized and normalized, and augmented to improve classification accuracy and solve the class imbalance problem.

[0008] Step 3: Construct and train a wafer defect detection model. This model is used to determine whether a wafer image contains defects; defect-free wafer images are marked as 0, and defective wafer images are marked as 1. The defect detection model is an improvement on the classic LeNet-5 network. The network structure includes an input layer, convolutional layers, pooling layers, fully connected layers, and a sigmoid classification output layer. The loss function is set to the cross-entropy function, and the activation function is the ReLU function.

[0009] Step 4: Construct and train a wafer defect classification model. Input all wafer images marked as 1 in Step 3 into the wafer defect classification model for training. Use this model to determine the wafer defect pattern category. During model construction, the N-triplet loss function is used instead of the cross-entropy loss function to address the limitations of the cross-entropy loss function. Simultaneously, the CBAM attention mechanism is introduced, allowing the network model to focus more on the region of interest and ignore useless information, thereby improving model accuracy.

[0010] The defect classification model is designed based on the residual block of the ResNet network. The network structure includes convolutional layers, convolutional blocks, residual blocks, global average pooling, and dense layers. The loss function is set to the N-triplet function, the CBAM attention mechanism is introduced, and the activation function is the ReLU function.

[0011] Step 5: Based on the network model trained in Step 4, input the sample to be tested into the network model, and determine the cause of the defect in the sample to be tested by analyzing the causes of defects in known defective samples.

[0012] Based on the wafer defect detection and classification model constructed in steps 3 and 4, the wafer image defect features of the output layer are extracted. The defect patterns of the sample to be tested are compared with the defect patterns of known samples. By analyzing the defect patterns of known samples, the cause of the defects is determined, thereby determining the cause of the defects in the sample to be tested. The process flow is continuously optimized to improve the yield of the next batch of wafer products.

[0013] In step 2, the dataset needs to be preprocessed to transform the wafer feature matrix in the dataset into a wafer defect pattern map. The dataset is then augmented, standardized, and normalized. This invention uses Convolutional Autoencoder (CAE) technology to augment the image according to formula (1) to solve the problem of class imbalance in the dataset. Finally, the image is standardized and normalized to uniformly scale the selected wafer image size to 256×256, and the image pixels are limited to the range (0, 1).

[0014] (1)

[0015] In formula (1), m is the number of expanded dataset samples, M is the total number of original wafer dataset samples, N is the number of dataset sample categories, and p is the proportion of each category in the original dataset.

[0016] A convolutional autoencoder consists of an encoder and a decoder. The encoder comprises a convolutional layer and a pooling layer, while the decoder includes only a deconvolutional layer. The preprocessing and data augmentation model is built on the PyTorch framework, and its specific structure is shown in the table below:

[0017]

[0018] In step 3, the wafer defect detection model consists of three convolutional layers and two fully connected layers. The convolutional kernels are 3×3 with a stride of 1×1, and the activation function is ReLU. Each convolutional layer is followed by a max-pooling layer with a size of 3×3 and a stride of 1×1 for downsampling. The second fully connected layer uses the Sigmoid activation function to activate the output. To prevent overfitting, a Dropout operation is performed after the fully connected layer with a probability of 0.5. The defect detection model is built on the PyTorch framework, and the specific model structure is shown in the table below.

[0019]

[0020] In step 4, the wafer defect classification model includes two convolutional layers, two convolutional block layers, two residual block layers, and two fully connected layers. The convolutional block layers consist of a convolutional layer, a max pooling layer, a batch normalization (BN) layer, a ReLU activation function, and a dropout layer. The residual block layers consist of a separable convolutional layer, a max pooling layer, and a BN layer. A global average pooling layer is used between the residual block layers and the fully connected layers. Since it has no learnable parameters, overfitting can be avoided. Furthermore, adding a global average pooling layer increases spatial information, making the network more robust to spatial translations. The Dropout probability in the convolutional block layer and the first fully connected layer is set to 0.5. The second fully connected layer uses the Softmax activation function to activate the output. The convolutional kernel size is 3×3 with a stride of 1×1. The max pooling downsampling layer size is 3×3 with a stride of 1×1. An attention mechanism is introduced in the convolutional block layer and the residual block layer to obtain more detailed information related to the target and ignore other irrelevant information, thereby improving the model accuracy. The defect classification model is built on the PyTorch framework, and the specific model structure is shown in the table below:

[0021]

[0022] Step 4 uses the N-triplet loss function. While cross-entropy loss is commonly used in supervised learning, it has limitations, such as the lack of robustness due to noisy labels. This invention replaces the cross-entropy loss function with the N-triplet function, which trains the vector embeddings of the input images so that image representations within the same class are more similar than those across different classes. During model training, the N-triplet function outperforms the cross-entropy function in supervised training. Using N-triplet loss helps the sample images embed representations better across dimensions, improving the overall accuracy of the network. Training the model with N-triplet loss reduces the distance between similar embeddings, achieving better feature learning and aiding in the network model's classification. The formula is as follows:

[0023] (2)

[0024] In formula (2), {( ),…,(( )} represents N pairs of samples from N different categories, where m is a set threshold. Represents positive samples. j This represents a negative sample. Representing the corresponding samples Vector embedding, Representing vector embedding The transpose of .

[0025] Step 4 introduces the plug-and-play CBAM attention mechanism, a lightweight attention mechanism that can perform attention operations in both spatial and channel dimensions.

[0026] At each channel, the input feature map is passed through two parallel MaxPool and AvgPool layers, reducing the feature map from C×H×W to C×1×1. Then, it passes through a Share MLP module, where the number of channels is first compressed to 1 / r of the original number (reduction rate), then expanded to the original number of channels. After passing through a ReLU activation function, two activated results are obtained. These two outputs are added element-wise, then passed through a sigmoid activation function to obtain the ChannelAttention output. This output is then multiplied by the original image to restore the size to C×H×W. The formula is expressed as follows:

[0027]

[0028] (3)

[0029] Spatially, the output of Channel Attention is processed by max pooling and average pooling to obtain two 1×H×W feature maps. These two feature maps are then concatenated using a Concat operation, transformed into a 1-channel feature map by a 7*7 convolution, and then passed through a sigmoid function to obtain the Spatial Attention feature map. Finally, the output is multiplied by the original image to revert to C×H×W size. The formula is expressed as follows:

[0030] )

[0031] = (4)

[0032] During model training, 80% of the sample images were selected as the training set, 10% as the validation set, and 10% as the test set. The learning rate was 0.0001, the batch size was 128, the number of epochs was 150, and the loss function was optimized using the Adam optimizer.

[0033] The present invention has the following advantages:

[0034] This method employs deep learning technology to construct a wafer defect detection and classification model. It adopts the concept of residual blocks and introduces separable convolution and CBAM attention mechanism modules, which have the characteristics of fast classification speed and high efficiency. It can be applied to the field of defect detection and classification in wafer manufacturing process, replacing manual operation and saving labor. Attached Figure Description

[0035] Figure 1 Example diagram of wafer defect modes;

[0036] Figure 2 This is a flowchart of the convolutional autoencoder in this invention;

[0037] Figure 3 This is a flowchart illustrating the overall analytical process for wafer defect detection and classification in this invention.

[0038] Figure 4 This is a diagram of the defect detection network model of the present invention;

[0039] Figure 5 This is a diagram of the defect classification network model of the present invention; Detailed Implementation

[0040] The technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings and examples.

[0041] Figure 1 This image shows an example of wafer defect patterns from the WM-811K wafer dataset. The dataset contains eight classic defect pattern types and one patternless type (Nonpattern, labeled "None"). The eight classic defect pattern types are: Center, Donut, Loc, Edge-loc, Edge-Ring, Near-full, Random, and Scratch. The dataset consists of 811,457 wafer images, of which 172,950 are labeled. Only 3.1% of the labeled images exhibit defect patterns; the rest are patternless. In the wafer images with defect patterns, there are only 149 images of almost all defect types (labeled "Near-full"), and 9680 images of edge-ring types (labeled "Edge-Ring"). The disparity in the number of various defect pattern types in the dataset is too large, and the image sizes are inconsistent, which is not conducive to the training of deep convolutional neural networks and affects the performance of the constructed model. Therefore, the original dataset must first be augmented, standardized, and normalized before network training. The specific steps are as follows:

[0042] 1. Dataset preprocessing

[0043] There are many methods for dataset augmentation, including traditional methods such as rotation, flipping, scaling, and cropping, as well as image generation methods utilizing autoencoders and GANs (Generative Adversarial Networks). Convolutional Autoencoders (CAEs) are used for image augmentation. A CAE consists of two parts: an encoder and a decoder. The encoder can be a neural network composed of convolutional layers, pooling layers, and fully connected layers (to reduce dimensionality, convolutions use downsampling, or the matrices are converted into one-dimensional tensors before full connections, with the number of neurons in the fully connected layers decreasing layer by layer), and the output dimension is much smaller than the input. The decoder can be a combination of deconvolutional layers and fully connected layers (to increase dimensionality, convolutions usually use upsampling, or the number of neurons in the fully connected layers is gradually increased). The decoder takes the output of the encoder as input, and the final output has the same dimensionality as the input. A diagram of a convolutional autoencoder is shown below. Figure 2 As shown, image augmentation is performed based on convolutional autoencoder technology according to formula (1).

[0044] (1)

[0045] In formula (1), m is the number of expanded dataset samples, M is the total number of original wafer dataset samples, N is the number of dataset sample categories, and p is the proportion of each category in the original dataset.

[0046] The size of a wafer image is determined by the wafer size and the number of chips on the wafer. In actual production, due to differences in process technology and the different chip products manufactured on the wafer, there are wafers of various sizes and corresponding wafer images. When training models and batch processing wafer images, it is necessary to uniformly input the size of the wafer images and scale wafer images of different sizes to a uniform and appropriate size. In this invention, a bilinear interpolation algorithm is used to uniformly scale the amplified image samples to a size of 256×256, and finally, formula (5) is used to limit the image pixels to the range (0, 1).

[0047] (5)

[0048] In formula (5), Represents the three-channel pixel values ​​of the input image.

[0049] 2. Network Model Construction and Training

[0050] (1) Wafer defect detection model

[0051] The model is built on the PyTorch framework and contains three convolutional layers and two fully connected layers. The convolutional kernel size of the convolutional layer is 3×3, the stride is 1×1, and the activation function is ReLU, as shown in formula (6). After each convolutional layer, a max pooling layer is used for downsampling, with a size of 3×3 and a stride of 1×1. The second fully connected layer uses the Sigmoid activation function to activate the output, as shown in formula (7). The loss function is the cross-entropy loss function. To prevent overfitting, a Dropout operation is performed after the fully connected layer, and the Dropout probability is set to 0.5.

[0052] (6)

[0053] In formula (6), x is the value of the neuron node in the convolutional layer of the convolutional network.

[0054] (7)

[0055] In formula (7), z is the value of the neuron node in the convolutional layer of the convolutional network.

[0056] (2) Wafer defect classification model

[0057] The model is built on the PyTorch framework and includes two convolutional layers, two convolutional block layers, two residual block layers, and two fully connected layers. The convolutional block layers consist of a convolutional layer, a max pooling layer, a batch normalization (BN) layer, a ReLU activation function, and a dropout layer. The residual block layers consist of a separable convolutional layer, a max pooling layer, and a BN layer. A global average pooling layer (Equation (8)) is used between the residual block layers and the fully connected layers, significantly reducing the number of computational parameters. The Dropout probability in the convolutional block layer and the first fully connected layer is set to 0.5. The output of the second fully connected layer is activated by the Softmax activation function (Equation (9)). The kernel size of the convolutional layer is 3×3 and the stride is 1×1. The size of the max pooling downsampling layer is 3×3 and the stride is 1×1. The CBAM attention mechanism is introduced in the convolutional block layer and the residual block layer to obtain more detailed information related to the target, ignore other irrelevant information, and improve the accuracy of the model. The loss function is the N-triplet function.

[0058] (8)

[0059] There are no parameters to be learned in formula (8), which can effectively avoid overfitting. At the same time, adding a global average pooling layer can increase spatial information, making the network more robust to spatial translation.

[0060] (9)

[0061] During model training, 80% of the sample images are selected as the training set, 10% as the validation set, and 10% as the test set. The learning rate is set to... The batch size was set to 128, the number of iterations was 150, and the loss function was optimized using the Adam optimizer. First, the sample images were processed by a wafer defect detection model to identify defective wafer images marked as 1. These defective wafer images were then input into a wafer defect classification model to train the wafer defect detection and classification model. Finally, the samples to be tested were input into the trained network model for defect classification. By analyzing the causes of defects in known defective samples, the cause of defects in the samples to be tested was determined. The trained defect detection model achieved a 97% detection rate in defect sample detection. The defect sample classification model can quickly and accurately identify nine defect patterns: None, Center, Donut, Loc, Edge-loc, Edge-Ring, Near-full, Random, and Scratch, with test accuracies of 93.4%, 99%, 90%, 84.9%, 91.7%, 98.1%, 99.9%, 96%, and 98.7%, respectively. This invention features fast classification speed and high efficiency, and can be applied to the field of defect detection and classification in wafer manufacturing processes, replacing manual operations and saving a significant amount of labor.

Claims

1. A wafer surface defect classification method based on deep learning networks, characterized in that, The method includes the following steps: Step 1: Based on the wafer dataset, convert the wafer feature matrix in the dataset into a wafer map; Step 2: Construct a preprocessing and data augmentation model to augment and standardize the dataset samples; Step 3: Construct and train a wafer defect detection model. Use this model to determine whether there are defects in the wafer image. A defect-free wafer image is marked as 0, and a defective wafer image is marked as 1. Step four: Construct and train a wafer defect classification model. Input all wafer images marked as 1 in step three into the wafer defect classification model for training. Use this model to determine the wafer defect pattern category. During model construction, use the N-triplet loss function instead of the cross-entropy loss function to address certain limitations of the cross-entropy loss function. The specific calculation formula for the N-triplet loss function is as follows: (1) In formula 1, {( ),…,(( )}represent Different categories For the sample, To set a threshold, Representing the One input sample, Representative and Positive samples of the same category Representative and Negative samples of different categories, among which At the same time, the CBAM attention mechanism is introduced, which makes the network model pay more attention to the region of interest and ignore useless information. Step 5: Based on the network model trained in Step 4, input the sample to be tested into the network model, and determine the cause of the defect in the sample to be tested by analyzing the causes of defects in known defective samples.

2. The wafer surface defect mode detection and analysis method as described in claim 1, characterized in that, Step two is as follows: Image augmentation is performed using convolutional autoencoder (CAE) technology according to Equation 2. The CAE consists of an encoder and a decoder. The encoder consists of a convolutional layer and a pooling layer, and the decoder consists of only a deconvolutional layer. The model is built on the PyTorch framework. (2) In formula (2), m is the number of expanded dataset samples, M is the total number of original wafer dataset samples, N is the number of dataset sample categories, and p is the proportion of each category in the original dataset.

3. The wafer surface defect mode detection and analysis method as described in claim 1, characterized in that, Step three is as follows: The model is built on the PyTorch framework. The wafer defect detection model contains three convolutional layers and two fully connected layers. The convolutional kernels of the convolutional layers are 3×3 with a stride of 1×1, the activation function is ReLU, and the loss function is cross-entropy. Each convolutional layer is followed by a max pooling layer for downsampling, with a kernel size of 3×3 and a stride of 1×1. The second fully connected layer uses the Sigmoid activation function to activate the output. To prevent overfitting, a Dropout operation is performed after the fully connected layer with a probability of 0.

5. During training, the loss function is optimized using the Adam algorithm, and the learning rate is set to 0.0001.

4. The wafer surface defect mode detection and analysis method as described in claim 1, characterized in that, Step four is as follows: The model is built on the PyTorch framework. The wafer defect classification model includes two convolutional layers, two convolutional block layers, two residual block layers, and two fully connected layers. The convolutional block layers consist of convolutional layers, max pooling layers, batch normalization (BN) layers, ReLU activation functions, and dropout layers. The residual block layers consist of separable convolutional layers, max pooling layers, and BN layers. A global average pooling layer is used between the residual block layers and the fully connected layers. Since it has no learnable parameters, it avoids overfitting. Adding a global average pooling layer also increases spatial information, making the network more robust to spatial translations. The Dropout probability in the block layer and the first fully connected layer is set to 0.

5. The output of the second fully connected layer is activated by the Softmax activation function. The kernel size of the convolutional layer is 3×3 and the stride is 1×1. The size of the max pooling downsampling layer is 3×3 and the stride is 1×1. The loss function is the N-triplet function, as shown in formula (1). The CBAM attention mechanism is introduced in the convolutional block layer and the residual block layer to obtain more detailed information related to the target and ignore other irrelevant information. The loss function is optimized by the Adam algorithm during training, and the learning rate is set to 0.0001.

Images