Hybrid wafer graph defect detection method based on double-branch semantic segmentation

Abstract

The invention provides a deep learning model which combines a semantic segmentation network and defect number identification and is used for mixed type wafer graph defect detection. The invention provides a novel double-branch semantic segmentation network for hybrid wafer graph defect detection based on a Pytorch framework for realizing a detection algorithm for a hybrid wafer graph defect mode. The model is composed of an encoder, a decoder and an auxiliary branch, the encoder extracts key features of a wafer graph, the decoder generates pixel-level output through semantic segmentation up-sampling operation so as to deduce defect types, the auxiliary branch recognizes the number of defects in the wafer graph, and the defect type of the wafer graph is determined by restraining the number of the defects and combining the output of the decoder. And the defect type identification precision is improved. According to the method, segmentation and identification of the existing wafer graph can be realized, the method has the capability of processing unknown mixed defect types, the identification accuracy of the mixed wafer graph defect mode is improved, and the development of the semiconductor industry is facilitated.

Description

Technical Field

[0001] This invention belongs to the field of wafer patterns generated during semiconductor manufacturing, and specifically relates to the identification of defect patterns in hybrid wafer patterns. Background Technology

[0002] In the field of microelectronics manufacturing, semiconductor chips are of paramount importance. The manufacturing process of their core carrier, the wafer, is extremely complex, including silicon wafer manufacturing, front-end processing, and back-end processing. Silicon wafer manufacturing involves reducing and purifying silicon dioxide into polycrystalline silicon, pulling it into single-crystal silicon ingots, and then cutting it into patternless silicon wafers. The front-end involves building logic circuits in a cleanroom. The back-end involves testing, sorting, cutting, and packaging the silicon wafers to produce chips.

[0003] However, as chip design complexity increases, the probability of defects in wafer manufacturing rises. After manufacturing, wafer electrical testing is required to determine chip functionality, and the resulting wafer map reflects defects. Faulty die areas caused by problems in different processes appear as specific patterns on the wafer map. Classifying and segmenting the wafer map based on the defect percentage, location, and coverage helps engineers analyze and locate production line problems, repair faults, and improve production yield.

[0004] In the modern semiconductor industry, single-defect identification is insufficient to meet the demands, as mixed defects are random and superimposed, making it difficult for traditional methods to handle unknown mixed defects. Semantic segmentation technology, however, can segment images into semantically meaningful regions, address occlusion issues, and provide refined semantic information. A semantic segmentation scheme based on eight basic single-defect types can accurately identify and segment existing data while also handling unknown mixed defects, offering a new approach to solving wafer image defect analysis problems. Summary of the Invention

[0005] The purpose of this invention is to provide a deep learning model that combines semantic segmentation networks and defect count recognition for hybrid wafer image defect detection, thereby improving the accuracy of defect pattern recognition in hybrid wafer images.

[0006] To achieve the above objectives, the present invention provides a wafer pattern defect detection method, comprising:

[0007] Step 1. Preprocess the images in the wafer image semantic segmentation dataset and divide them into training and test sets;

[0008] Step 2. Input the training set data into the designed deep learning model and train the model using the training set;

[0009] Step 3. Test the trained deep learning model using the test set. Use semantic segmentation evaluation metrics, namely pixel accuracy and intersection over union, to evaluate the model's segmentation performance. At the same time, use classification evaluation metrics, namely accuracy, precision, and recall, to evaluate the model's classification performance.

[0010] In step 1, the preprocessing of the wafer image involves converting the wafer image data into an image of size 128×128 with 3 pixel channels; the dataset is divided into training and test sets in a ratio of 9:1.

[0011] In step 2, the designed deep learning model is a two-branch semantic segmentation model, which includes three parts: encoder, decoder and auxiliary branch.

[0012] The encoder is used to extract key features from the wafer map, laying the foundation for the subsequent operation of the decoder and auxiliary branches. It uses MobileNetV2 as the base network and contains one convolutional layer and nine Bottleneck blocks. The Bottleneck block structure is a convolutional layer, a ReLU 6 layer, a depthwise separable convolutional layer, a ReLU 6 layer, a convolutional layer, and a Linear layer. The encoder performs downsampling at the convolutional layer, the second Bottleneck block, and the third Bottleneck block. The wafer map dimension input to the encoder is [3, 128, 128], and the encoder output feature map dimension is [320, 16, 16].

[0013] The first downsampling input feature vector has dimensions [3, 128, 128], and the output feature vector has dimensions [32, 64, 64].

[0014] The second downsampling input feature vector has dimensions [16, 64, 64], and the output feature vector has dimensions [24, 32, 32].

[0015] The third downsampling input feature vector has dimensions [24, 32, 32], and the output feature vector has dimensions [32, 16, 16].

[0016] The decoder includes a pyramid pooling module, three upsampling operations, and one convolutional layer; the wafer map dimension of the input decoder is [320, 16, 16], and the dimension of the decoder output feature map is [3, 128, 128].

[0017] The pyramid pooling module uses four pooling filters to perform pooling operations on the input feature vector, with pooling filter sizes of 1×1, 2×2, 4×4, and 8×8, respectively.

[0018] The first upsampling input feature vector has dimensions [160, 16, 16], the skip connection feature vector has dimensions [96, 16, 16], and the output feature vector has dimensions [64, 32, 32].

[0019] The second upsampling input feature vector has dimensions [64, 32, 32], the skip connection feature vector has dimensions [24, 32, 32], and the output feature vector has dimensions [32, 64, 64].

[0020] The third upsampling input feature vector has dimensions of [32, 64, 64], the skip connection feature vector has dimensions of [16, 64, 64], and the output feature vector has dimensions of [32, 128, 128].

[0021] The auxiliary branch consists of a convolutional layer, an average pooling layer, a fully connected layer, and a classifier. The wafer image dimension input to the auxiliary branch is [320, 16, 16]. The output of the fully connected layer is the number of defects in the wafer image. The number of defects in the wafer image and the decoder output features are input into the classifier. By analyzing the decoder output data, the number of defects and the defect type can be obtained. By constraining the number of defects and combining the decoder analysis results, the possibility of misjudgment or omission of defect types can be reduced. The final output of the classifier is the defect type of the wafer image. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of the wafer pattern defect mode detection method according to the present invention;

[0023] Figure 2 This is the dual-branch semantic segmentation model proposed in this invention;

[0024] Figure 3 It is the pyramid pooling module in the semantic segmentation model proposed in this invention; Detailed Implementation

[0025] The present invention will be further described below with reference to the accompanying drawings.

[0026] This invention employs a deep learning model combining semantic segmentation networks and defect count recognition for hybrid wafer image defect detection. The specific process is attached. Figure 1As shown in Figure S101, the semantic segmentation wafer image data is preprocessed into an image of size 128×128 with 3 channels, and then input into the semantic segmentation model, as shown in Figure S102. The semantic segmentation model is trained on the training dataset to obtain the optimal model, and then tested using the test dataset. The semantic segmentation model provides the segmentation results for the test image, as shown in Figure S103. Based on these segmentation results, the defect category of the test image can be determined, as shown in Figure S104.

[0027] The deep learning model proposed in this invention is as follows: Figure 2 As shown, it consists of three parts: S201, S202, and S203.

[0028] S201 is the encoder. The input to the encoder is a semantic segmentation wafer image dataset, which needs to be preprocessed to a size of 128×128 with 3 channels. The main function of the encoder is to extract key features from the wafer image, laying the foundation for the subsequent operation of the decoder and auxiliary branches. The encoder uses MobileNetV2 as the base network, which contains one convolutional layer and nine Bottleneck blocks. The Bottleneck block structure consists of a convolutional layer, a ReLU6 layer, a depthwise separable convolutional layer, a ReLU6 layer, another convolutional layer, and a linear layer. The encoder performs downsampling operations at the convolutional layer, the second Bottleneck block, and the third Bottleneck block. The first downsampling operation results in an input feature vector dimension of [3, 128, 128] and an output feature vector dimension of [32, 64, 64]. The second downsampling operation results in an input feature vector dimension of [16, 64, 64] and an output feature vector dimension of [24, 32, 32]. The third downsampling operation results in an input feature vector dimension of [24, 32, 32] and an output feature vector dimension of [32, 16, 16].

[0029] S202 is the decoder, which includes a pyramid pooling module, three upsampling operations, and one convolutional layer. The input wafer map dimension of the decoder is [320, 16, 16], and the decoder output feature map dimension is [3, 128, 128]. The pyramid pooling module is shown in the attached diagram. Figure 3As shown, S301 represents the feature vector input to the pyramid pooling module, with a size of 320×16×16. S302 represents four pooling filters with sizes of 1×1, 2×2, 4×4, and 8×8, respectively. Average pooling is used. After upsampling to unify the feature size, the outputs of the four pooling filters are superimposed as the output of the pyramid pooling module, which is S303. Then, the decoder performs three upsampling operations. The first upsampling operation gives the input feature vector dimension [160, 16, 16], the skip connection feature vector dimension [96, 16, 16], and the output feature vector dimension [64, 32, 32]. The second upsampling operation gives the input feature vector dimension [64, 32, 32], the skip connection feature vector dimension [24, 32, 32], and the output feature vector dimension [32, 64, 64]. The third upsampling input feature vector has dimensions of [32, 64, 64], the skip connection feature vector has dimensions of [16, 64, 64], and the output feature vector has dimensions of [32, 128, 128].

[0030] S203 is the auxiliary branch, which contains a convolutional layer, an average pooling layer, a fully connected layer, and a classifier. The wafer image dimension input to the auxiliary branch is [320, 16, 16]. The output of the fully connected layer is the number of defects in the wafer image. The number of defects in the wafer image and the decoder output features are input into the classifier. By analyzing the decoder output data, the number of defects and defect types can be obtained. By constraining the number of defects and combining it with the decoder analysis results, the possibility of misclassification or omission of defect types can be reduced. Finally, the classifier output is the defect type of the wafer image.

Claims

1. A method for wafer image defect pattern recognition using a dual-branch semantic segmentation network, characterized in that, Includes the following steps: Step 1: Preprocess the wafer image semantic segmentation dataset and divide it into training and test sets; Step 2: Input the training set data into the designed deep learning model and train the model using the training set; Step 3: Test the trained deep learning model using the test set. Use semantic segmentation evaluation metrics, namely pixel accuracy and intersection over union, to evaluate the model's segmentation performance. At the same time, use classification evaluation metrics, namely accuracy, precision, and recall, to evaluate the model's classification performance.

2. The method for wafer image defect pattern recognition using a dual-branch semantic segmentation network as described in claim 1, characterized in that, The preprocessing of the wafer image involves converting the wafer image data into an image of 128×128 pixels with 3 channels. The dataset is divided into a training set and a test set in a ratio of 9:

1.

3. The method for wafer image defect pattern recognition using a dual-branch semantic segmentation network as described in claim 1, characterized in that, The designed deep learning model is a two-branch semantic segmentation model, which consists of an encoder, a decoder, and an auxiliary branch: 3-1. The encoder is used to extract key features of the wafer map. It uses MobileNetV2 as the base network and contains one convolutional layer and nine Bottleneck blocks. Downsampling is performed at the convolutional layer, the second Bottleneck block, and the third Bottleneck block, respectively. The wafer map dimension input to the encoder is [3, 128, 128], and the encoder output feature map dimension is [320, 16, 16]. 3-2. The decoder includes a pyramid pooling module, three upsampling operations, and one convolutional layer; the pyramid pooling module uses four pooling filters to perform pooling operations on the input feature vector, with pooling filter sizes of 1, 2, 4, and 8 respectively; the wafer map dimension of the input decoder is [320, 16, 16], and the dimension of the decoder output feature map is [3, 128, 128]. 3-3. The auxiliary branch includes a convolutional layer, an average pooling layer, a fully connected layer, and a classifier. The wafer image dimension input to the auxiliary branch is [320, 16, 16]. The output of the fully connected layer is the number of defects in the wafer image. The number of defects in the wafer image and the output features of the decoder are input into the classifier. The output of the classifier is the wafer image defect type.

4. The wafer pattern defect identification method as described in claim 3, characterized in that, The Bottleneck block has a structure consisting of a convolutional layer, a ReLU6 layer, a depth-separable convolutional layer, a ReLU6 layer, a convolutional layer, and a Linear layer.

5. The wafer pattern defect identification method as described in claim 3, characterized in that, The dimensions of the wafer image are [3, 128, 128], where 3 represents the number of channels in the feature vector, 128 represents the height of the feature vector, and 128 represents the width of the feature vector.

6. The wafer pattern defect identification method as described in claim 3, characterized in that, The encoder performs three downsampling operations. The first downsampling operation results in an input feature vector dimension of [3, 128, 128] and an output feature vector dimension of [32, 64, 64]. The second downsampling operation results in an input feature vector dimension of [16, 64, 64] and an output feature vector dimension of [24, 32, 32]. The third downsampling operation results in an input feature vector dimension of [24, 32, 32] and an output feature vector dimension of [32, 16, 16].

7. The wafer pattern defect identification method as described in claim 3, characterized in that, The decoder performs three upsampling operations. The first upsampling operation results in an input feature vector dimension of [160, 16, 16], a skip connection feature vector dimension of [96, 16, 16], and an output feature vector dimension of [64, 32, 32]. The second upsampling operation results in an input feature vector dimension of [64, 32, 32], a skip connection feature vector dimension of [24, 32, 32], and an output feature vector dimension of [32, 64, 64]. The third upsampling operation results in an input feature vector dimension of [32, 64, 64], a skip connection feature vector dimension of [16, 64, 64], and an output feature vector dimension of [32, 128, 128].

8. The wafer pattern defect identification method as described in claim 3, characterized in that, The auxiliary branch classifier inputs the number of defects in the wafer image and the output features of the decoder into the classifier. By analyzing the output data of the decoder, the number of defects and the defect type can be obtained. By constraining the number of defects and combining the analysis results of the decoder, the possibility of misjudgment or omission of defect types can be reduced. The classifier finally outputs the defect type of the wafer image.

Images