Wafer Defect Detection Method Based on Multi-Core Attention Fusion and Dynamic Large-Kernel Jumping

Abstract

The present invention discloses a wafer defect detection method based on multi-core attention fusion and dynamic large-kernel jumping, belonging to the field of semiconductor manufacturing technology. This method mainly includes image preprocessing, feature extraction, defect segmentation, and defect classification. It integrates the advantages of dynamic attention large-kernel jumping connection convolution and multi-core channel segmentation attention-aware fusion, aiming to achieve efficient wafer defect detection with fewer computing resources; through multi-core channel segmentation attention-aware fusion, deeply mines and fuses features of different scales and positions, enhancing the model's perception ability of complex defect patterns; while ensuring the detection accuracy, it significantly reduces the computational cost, thereby improving the efficiency and stability of wafer defect detection.

Description

Technical Field

[0001] The present invention relates to a method for detecting wafer defects, specifically a method for detecting wafer defects based on multi-core attention fusion and dynamic large-core jumping, belonging to the technical field of semiconductor manufacturing. Background Art

[0002] In recent years, with the continuous progress of semiconductor manufacturing technology, the demand for wafer defect detection has become increasingly urgent. Defects on the wafer surface not only seriously affect the yield and performance of chips, but also may lead to quality fluctuations in the large-scale production process. Therefore, accurately and efficiently identifying these defects is crucial for ensuring production quality. Common defect types include edge defects, center defects, and randomly distributed defects, and their diversity and complexity pose great challenges to the detection work. At the same time, with the continuous refinement of manufacturing processes, the amount of wafer data is growing rapidly, which further increases the difficulty of defect detection.

[0003] Traditionally, wafer defect detection relied on engineers to analyze defects in images through visual inspection. This method is not only inefficient, easily interfered by subjective factors, resulting in missed detections or misjudgments, but also time-consuming, laborious, and costly when faced with a large amount of data, and can no longer meet the needs of high-efficiency production. Therefore, the industry has gradually begun to explore more efficient and automated detection methods to improve the accuracy of defect identification and reduce operating costs.

[0004] In recent years, the rapid development of deep learning technology has brought new solutions to wafer defect detection. Systems based on convolutional neural networks can automatically learn complex features in images, achieve efficient classification of various defect types, and significantly improve the detection efficiency and accuracy. For example, on the one hand, the Residual Network (ResNet) uses residual blocks to deepen the network depth, effectively alleviating the problem of gradient disappearance, and thus improving the ability to extract deep features; on the other hand, the Dense Connectivity Network (DenseNet) achieves the maximum reuse of features through a dense connection structure, reducing redundant parameters. These methods have made important breakthroughs in improving detection performance, but at the same time, they also bring higher computational resource consumption and longer training time.

[0005] In addition, multi-scale feature extraction and attention mechanisms have also been introduced to enhance the model's capture of defect details. The multi-scale method helps to comprehensively capture the global and local features of defects by fusing feature information at different scales; while the attention mechanism improves the model's sensitivity to important defect regions by dynamically weighting key features. However, at present, these improved methods are often optimized independently and do not fully utilize lightweight network structures to reduce the amount of computation, and at the same time, they do not comprehensively use different scale information for detection. Summary of the Invention

[0006] Objective of the Invention: Aiming at the above problems, the objective of the present invention is to provide a wafer defect detection method based on multi-core attention fusion and dynamic large kernel jumping.

[0007] Technical Solution: The wafer defect detection method based on multi-core attention fusion and dynamic large kernel jumping of the present invention includes the following steps:

[0008] Obtain the wafer image to be detected and perform image preprocessing;

[0009] Input the preprocessed image into three feature extraction layers with the same structure in sequence for feature extraction, and obtain the first feature map, the second feature map, and the third feature map respectively. Then input the extracted third feature map into the fourth feature extraction layer and the fifth feature extraction layer in sequence for feature extraction, and obtain the fourth feature map and the fifth feature map;

[0010] Input the first feature map to the fourth feature map into the DALKConv-SC module respectively, and obtain four feature maps with different sizes, which are respectively denoted as the sixth feature map, the seventh feature map, the eighth feature map, and the ninth feature map;

[0011] Use the fifth feature map to the ninth feature map for fusion decoding to obtain the defect segmentation result;

[0012] Input the fifth feature map into the global average pooling layer and the activation function layer in sequence to obtain the defect classification result.

[0013] Further, the step of inputting the preprocessed image into three feature extraction layers with the same structure in sequence for feature extraction, and obtaining the first feature map, the second feature map, and the third feature map respectively. Then input the extracted third feature map into the fourth feature extraction layer and the fifth feature extraction layer in sequence for feature extraction, and obtain the fourth feature map and the fifth feature map includes:

[0014] Input the preprocessed image into the first MKCS-SE module for feature extraction, and then perform max-pooling processing on the extracted feature map to obtain the first feature map;

[0015] Input the first feature map into the second MKCS-SE module for feature extraction, and then perform max-pooling processing on the extracted feature map to obtain the second feature map;

[0016] Input the second feature map into the third MKCS-SE module for feature extraction, and then perform max-pooling processing on the extracted feature map to obtain the third feature map;

[0017] Input the third feature map into the fourth MKCS-SE module for feature extraction, and then perform average-pooling processing on the extracted feature map to obtain the fourth feature map;

[0018] Input the fourth feature map into the fifth MKCS-SE module for feature extraction, and then perform average pooling on the extracted feature map to obtain the fifth feature map.

[0019] Further, the steps of inputting the first to fourth feature maps into the DALKConv-SC module respectively include:

[0020] Take one of the first, second, third, or fourth feature maps as the input feature map, input the input feature map into the first depthwise separable convolution and the second depthwise separable convolution respectively for large kernel feature extraction, and obtain the first large kernel feature map and the second large kernel feature map respectively. After fusing the first large kernel feature map and the second large kernel feature map, perform average pooling and max pooling respectively to obtain the third large kernel feature map and the fourth large kernel feature map;

[0021] Input the third large kernel feature map and the fourth large kernel feature into the third depthwise separable convolution for channel fusion, and after activating the result of channel fusion through the Sigmod function, obtain the fifth large kernel feature;

[0022] Perform a Scale operation on the fifth large kernel feature map and the first large kernel feature map to obtain the seventh large kernel feature map;

[0023] Perform a Scale operation on the fifth large kernel feature map and the second large kernel feature map to obtain the eighth large kernel feature map;

[0024] Perform a merging operation on the seventh large kernel feature map, the eighth large kernel feature map and the input feature map to obtain the feature map finally output by the DALKConv-SC module.

[0025] Further, the steps of using the fifth to ninth feature maps for fusion decoding to obtain the defect segmentation result include:

[0026] After fusing the ninth feature map and the fifth feature map, use the Transpose-Conv module for decoding to obtain the tenth feature map;

[0027] After fusing the eighth feature map and the tenth feature map, use the Transpose-Conv module for decoding to obtain the eleventh feature map;

[0028] After fusing the seventh feature map and the eleventh feature map, use the Transpose-Conv module for decoding to obtain the twelfth feature map;

[0029] Fuse the sixth feature map and the twelfth feature map, and use the Transpose-Conv module for decoding to obtain the thirteenth feature map;

[0030] Resize the thirteenth feature map through depthwise separable convolution and use the Sigmod function to obtain the output segmentation result.

[0031] Furthermore, the structures of the first to fifth MKCS-SE modules are the same. The process of feature extraction includes:

[0032] Perform preliminary feature extraction on the input feature map through depthwise separable convolution, and then adjust the channels of the obtained preliminary feature map through depthwise separable convolution to make the number of channels a multiple of four.

[0033] Then perform channel segmentation of the adjusted feature map into four equal parts.

[0034] Use depthwise separable convolutions with four different sizes to perform feature extraction at different scales on the feature maps obtained by channel segmentation into four equal parts.

[0035] Merge the feature maps obtained by feature extraction at four different scales, perform residual connection with the preliminary feature map, and input the feature map after residual connection into the SE attention module.

[0036] Input the feature map processed by the SE attention module into depthwise separable convolution to adjust the channels and output with the preset number of channels.

[0037] Furthermore, the steps of image preprocessing include:

[0038] Adjust the size of the defective image to a preset size, perform median filtering, and perform data augmentation for unbalanced categories.

[0039] Furthermore, data augmentation includes vertically or horizontally flipping the image, vertically or horizontally moving the image, and rotating or scaling the image.

[0040] Advantageous effects: Compared with the prior art, the significant advantages of the present invention are:

[0041] (1) The present invention adopts a U-Net structure, and through multi-core channel segmentation attention perception fusion, deeply mines and fuses features at different scales and positions, extracts multi-scale image features, enables the model to perform better when processing wafer defects of different scales and shapes, significantly improves the accuracy of defect detection, and introduces dynamic attention large kernel skip connection convolution to improve the accuracy of the model in the segmentation task and reduce the error rate of mask generation; the present invention has good generalization ability, can adapt to different types of wafer defect detection tasks, and has strong practical value;

[0042] (2) The present invention uses a lightweight architecture of the U-Net structure and a multi-core channel segmentation attention-aware fusion structure, which can not only efficiently extract and process rich features in the wafer image, ensure high-quality feature extraction while reducing the computational complexity, but also can simultaneously perform multi-task collaboration of defect segmentation and classification of the wafer map;

[0043] (3) By introducing the squeeze-and-excitation attention mechanism, the present invention can adaptively adjust the weights of each feature channel, making the model pay more attention to key defect information and suppressing irrelevant features, thereby enhancing the feature representation ability and detection performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 is the overall structure diagram of the MK-DANet network in the specific implementation of the present invention;

[0045] Figure 2 is the category picture of the wafer map defect dataset;

[0046] Figure 3 is the result after median filtering of the category picture of the wafer map defect dataset;

[0047] Figure 4 is the structural schematic diagram of the MKCS-SE feature extraction layer;

[0048] Figure 5 is the structural schematic diagram of the SE attention module;

[0049] Figure 6 is the specific structure diagram of the DALKConv-SC module. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0050] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments.

[0051] The wafer defect detection method based on multi-core attention fusion and dynamic large kernel skipping described in this embodiment has an overall block diagram as Figure 1 shown, and includes the following steps:

[0052] Step 1, obtain the wafer image to be detected and perform image preprocessing.

[0053] Further, the steps of performing image preprocessing include:

[0054] Adjust the size of the defect image to a preset size, such as it can be adjusted to , perform median filtering, and perform data augmentation for unbalanced categories.

[0055] Further, data augmentation includes vertically or horizontally flipping the image, vertically or horizontally shifting the image, and rotating or scaling the image.

[0056] In one example, a detected wafer image is obtained and its size is adjusted, such as being adjusted to a three-channel RGB image, Figure 2 is a category picture of the wafer map defect dataset, which contains eight different wafer map defects. In the figure, Center represents the center defect, denoted as C; Donut represents the annular defect, denoted as D; Edge-Loc represents the edge local defect, denoted as EL; Edge-Ring represents the edge annular defect, denoted as ER; Local represents the local defect, denoted as L; Random represents the random defect, denoted as R; Scratch represents the scratch defect, denoted as S; Near-full represents the near-full defect, denoted as NF. Then, median filtering is performed on the input defect image, Figure 3 is the result after median filtering of the category picture of the wafer defect dataset. After filtering, data augmentation for unbalanced classes is performed on the filtered image; data augmentation includes vertically or horizontally flipping the image, vertically or horizontally shifting the image, and slightly rotating or scaling the image. Through preprocessing, it helps to have a high tolerance for changes in the size, position, and orientation of defects in the picture.

[0057] Step 2: The preprocessed image is sequentially input into three feature extraction layers with the same structure for feature extraction, respectively obtaining the first feature map, the second feature map, and the third feature map. Then, the extracted third feature map is sequentially input into the fourth feature extraction layer and the fifth feature extraction layer for feature extraction, obtaining the fourth feature map and the fifth feature map.

[0058] Further, the step of sequentially inputting the preprocessed image into three feature extraction layers with the same structure for feature extraction, respectively obtaining the first feature map, the second feature map, and the third feature map, and then sequentially inputting the extracted third feature map into the fourth feature extraction layer and the fifth feature extraction layer for feature extraction, obtaining the fourth feature map and the fifth feature map includes:

[0059] The preprocessed image is input into the first MKCS-SE module for feature extraction. Then, after performing max pooling (MP) processing on the extracted feature map, the first feature map is obtained. The number of filters f in the first MKCS-SE module is 8;

[0060] The first feature map is input into the second MKCS-SE module for feature extraction. Then, after performing max pooling processing on the extracted feature map, the second feature map is obtained. The number of filters f in the second MKCS-SE module is 16;

[0061] The second feature map is input into the third MKCS-SE module for feature extraction. After the extracted feature map is subjected to max pooling, the third feature map is obtained. The number of filters f in the third MKCS-SE module is 16;

[0062] The third feature map is input into the fourth MKCS-SE module for feature extraction. After the extracted feature map is subjected to average pooling (AP), the fourth feature map is obtained. The number of filters f in the fourth MKCS-SE module is 32;

[0063] The fourth feature map is input into the fifth MKCS-SE module for feature extraction. After the extracted feature map is subjected to average pooling, the fifth feature map is obtained. The number of filters f in the fifth MKCS-SE module is 64.

[0064] Furthermore, the structures of the first to fifth MKCS-SE modules are the same. The process of feature extraction includes:

[0065] The input feature map is subjected to initial feature extraction through depthwise separable convolution, and then the obtained initial feature map is adjusted in channels through depthwise separable convolution to adjust the channels to a multiple of four;

[0066] Then, the adjusted feature map is subjected to channel splitting into four equal parts;

[0067] The feature maps obtained by channel splitting into four equal parts are respectively subjected to feature extraction at different scales using four depthwise separable convolutions with different sizes;

[0068] The four feature maps obtained by extraction at different scales are merged, and residual connection is performed with the initial feature map. The feature map after residual connection is input into the SE attention module;

[0069] The feature map obtained after being processed by the SE attention module is input into depthwise separable convolution for channel adjustment, and the channels are adjusted to a preset number of channels for output.

[0070] The MKCS-SE feature extraction layer is a multi-scale feature extraction module, as Figure 4 shown, including convolution kernels of different sizes and a channel splitting layer. In the example, the defective picture with a size of after preprocessing is input into the MKCS-SE feature extraction layer for feature extraction. The process is as follows: First, the defective picture is input into a depthwise separable convolution (DWConv) with a convolution kernel size of , an input stride of 1, and a padding method of same for initial extraction. Then, the obtained feature map is subjected to a convolution kernel size of , the depthwise separable convolution with an input stride of 1 and a padding method of same is used to adjust the channels to a multiple of 4; then the adjusted feature map is subjected to channel splitting (Split) into four equal parts; then the feature maps of the four equal-channel parts are respectively input into depthwise separable convolutions with a convolution kernel size of , , , , the depthwise separable convolution with an input stride of 1 and a padding method of same is used to extract features of different scales, and the four feature maps extracted at different scales are merged and subjected to residual connection with the feature map passing through a depthwise separable convolution with a convolution kernel size of , the depthwise separable convolution with an input stride of 1 and a padding method of same is used for residual connection with the feature map; finally, the result after residual connection is input into the SE attention module (SE-Attention), and the structure of the SE attention module is as shown in Figure 5 . The input is processed in two paths. The first path keeps the original structure unchanged; the second path first inputs a global average pooling layer (GAP) for pooling operation, then inputs the pooling result into a fully connected layer (FullyConnected) for full connection, then, after activation by the ReLU function, it is fully connected again through a fully connected layer, and then the fully connected result is input into the Sigmod function for operation to obtain a feature map of the same size as the original input, and its content is the specific weight for each channel. The obtained feature map is subjected to channel weighting (Scale) operation with the first path input, and finally the feature map processed by the SE attention layer is obtained. Finally, the feature map processed by the SE attention module is input into a depthwise separable convolution with a convolution kernel size of , the depthwise separable convolution with an input stride of 1 and a padding method of same is used to adjust the channels to a preset number of channels for output.

[0071] Among them, the attention mechanism layer SE in the second path mainly includes two operations: squeeze and excitation. The following are the detailed definitions and formula descriptions:

[0072] In the Squeeze process, global average pooling is performed on the input feature map with a shape of to obtain the global features of each channel. The specific formula for this process is:

[0073] ,

[0074] where, Z C represents the global feature of the Cth channel, and X C (i,j) represents the pixel value of the Cth channel at the position (i,j) in the input feature map.

[0075] In the Excitation operation, a series of non-linear transformations are performed on the channel descriptor z to generate the attention weights for each channel. The specific steps are as follows: First, enter the fully connected layer. The dimension of the channel descriptor is reduced to a fixed proportion of the original through a fully connected layer, usually , where r is a compression rate parameter, and the attention weight for each channel is obtained, expressed as:

[0076] ,

[0077] where, is the attention weight, W1 and b1 are the weights and biases of the fully connected layer, represents the LeakyReLU activation function, and the formula is:

[0078] ,

[0079] where X is the input data and leaky is a constant.

[0080] Then, enter another fully connected layer to restore the dimension to C, and the attention weight for each channel is obtained, expressed as:

[0081] ,

[0082] where W2 and b2 are the weights and biases of the fully connected layer.

[0083] The finally generated attention weight is used to weight each channel of the original feature map , that is, the final Scale operation, expressed as:

[0084] ,

[0085] where, is the output feature map after weighting.

[0086] Among them, the Sigmoid function is one of the commonly used activation functions in machine learning and deep learning, especially in binary classification problems. It maps the input to the interval (0, 1), so it is often used in the output layer to predict probability values and is defined as follows:

[0087] ,

[0088] where x is the input value, which can be any real number, and e is the base of the natural logarithm.

[0089] Step 3: Input the first to fourth feature maps into the DALKConv-SC module respectively to obtain four feature maps of different sizes, denoted as the sixth feature map, the seventh feature map, the eighth feature map, and the ninth feature map respectively.

[0090] Input the first feature map into the DALKConv-SC module to obtain the sixth feature map; input the second feature map into the DALKConv-SC module to obtain the seventh feature map; input the third feature map into the DALKConv-SC module to obtain the eighth feature map; input the fourth feature map into the DALKConv-SC module to obtain the ninth feature map.

[0091] Furthermore, the steps of inputting the first to fourth feature maps into the DALKConv-SC module respectively include:

[0092] Take one of the first, second, third, or fourth feature maps as the input feature map, input the input feature map into the first depthwise separable convolution and the second depthwise separable convolution respectively for large kernel feature extraction to obtain the first large kernel feature map and the second large kernel feature map respectively. After fusing the first large kernel feature map and the second large kernel feature map, perform average pooling and max pooling respectively to obtain the third large kernel feature map and the fourth large kernel feature map;

[0093] Input the third large kernel feature map and the fourth large kernel feature into the third depthwise separable convolution for channel fusion, and after activating the result of channel fusion through the Sigmod function, obtain the fifth large kernel feature;

[0094] Perform a Scale operation on the fifth large kernel feature map and the first large kernel feature map to obtain the seventh large kernel feature map;

[0095] Perform a Scale operation on the fifth large kernel feature map and the second large kernel feature map to obtain the eighth large kernel feature map;

[0096] Perform a merge operation on the seventh large kernel feature map, the eighth large kernel feature map, and the input feature map to obtain the feature map finally output by the DALKConv-SC module.

[0097] In the example, the structural schematic diagram of the DALKConv-SC module is as Figure 6 shown. First, input into the depthwise separable convolution with a convolution kernel size of , an input stride of 1, and a padding method of same, and a convolution kernel size of , the depthwise separable convolution with an input stride of 1 and a padding method of same is used for large kernel feature extraction; then the outputs of the two are fused and then average pooling (AVG) and max pooling (MAP) are performed respectively; finally, the two results obtained after average pooling and max pooling are input into a convolution kernel with a size of , the depthwise separable convolution with an input stride of 1 and a padding method of same is used for channel fusion, and the result of channel fusion is activated by the Sigmod function and then combined with the result after passing through a convolution kernel with a size of , the depthwise separable convolution with an input stride of 1 and a padding method of same and a convolution kernel with a size of , the depthwise separable convolution with an input stride of 1 and a padding method of same is subjected to a Scale operation on the result to obtain two feature maps; finally, the two feature maps obtained after the Scale operation are combined with the input to obtain the final feature map.

[0098] Step 4, use the fifth to ninth feature maps for fusion decoding to obtain the defect segmentation result;

[0099] Furthermore, the steps of using the fifth to ninth feature maps for fusion decoding to obtain the defect segmentation result include:

[0100] After fusing the ninth feature map and the fifth feature map, use the Transpose-Conv module for decoding to obtain the tenth feature map;

[0101] After fusing the eighth feature map and the tenth feature map, use the Transpose-Conv module for decoding to obtain the eleventh feature map;

[0102] After fusing the seventh feature map and the eleventh feature map, use the Transpose-Conv module for decoding to obtain the twelfth feature map;

[0103] Fuse the sixth feature map and the twelfth feature map, and use the Transpose-Conv module for decoding to obtain the thirteenth feature map;

[0104] Adjust the size of the thirteenth feature map through a depthwise separable convolution, use the Sigmod function, and obtain the output segmentation result. The convolution kernel size of the depthwise separable convolution is , the input stride is 1, the padding method is same, and the feature map is adjusted to a size of , the activation function selects the Sigmod function, and finally the output segmentation result is obtained.

[0105] In the above step 4, decoding is performed four times in sequence, and the number of channels for the four times are 32, 16, 16, and 8 respectively. Finally, the output is feature map.

[0106] The Transpose-Conv module is the transposed convolution (also known as deconvolution), which is the core operation module for upsampling feature maps in deep convolutional neural networks. This operation realizes the expansion of the spatial dimension in a learnable parametric way and shows significant advantages in tasks such as image segmentation, generative adversarial networks, and super-resolution reconstruction. The transposed convolution is not the inverse process of the traditional convolution operation, but the spatial transformation form of the gradient calculation in the backpropagation process of the standard convolution. Specifically, for the input feature map and the convolution kernel, the transposed convolution maps the low-dimensional input to the high-dimensional space through the transpose operation of the kernel matrix. Its output size follows a specific rule: when the stride is s, s - 1 zero values are inserted between input units to achieve implicit upsampling; the output boundary effect can be precisely controlled by adjusting the padding parameter. It should be noted that the introduction of the output padding parameter effectively solves the problem of output size ambiguity that may occur when the stride is greater than 1.

[0107] At the actual implementation level, a three-stage calculation process is adopted: first, the input features are expanded in spacing (zero value insertion), then boundary padding is applied, and finally, a conventional convolution operation is performed. This design not only maintains the computational efficiency but also seamlessly connects with modules such as batch normalization and activation functions in the network. Compared with the traditional interpolation upsampling method, the core advantage of the transposed convolution lies in the learnability of its parameters. The kernel weights are automatically optimized through gradient descent to adapt to the spatial feature distribution of specific tasks.

[0108] Step 5: Input the fifth feature map into the global average pooling layer and the activation function layer in sequence to obtain the defect classification result.

[0109] Input the feature map that has gone through the feature extraction part into the global average pooling layer for processing to obtain a feature map; then input the obtained feature map into a fully connected layer or a linear layer (Dense Layer) with the activation function ReLU to obtain a feature map; finally, input the obtained result into a Dense layer with the activation function SoftMax for classification output to obtain the final classification result.

[0110] Among them, the SoftMax layer is an activation function layer commonly used in classification problems. It converts the output of the fully connected layer into a probability distribution. Its output value range is between 0 and 1, and the sum of all outputs is 1, representing the probabilities of each class. Its formula can be expressed as:

[0111] ,

[0112] where, is the output probability of the j-th class, z j is the j-th output of the fully connected layer, m is the number of output classes, and the SoftMax function ensures that the output values form a probability distribution, enabling the interpretation of the model's output as the probability of each class, which is convenient for classification decisions.

[0113] Using the detection method described in the present invention to construct a detection model, before testing the detection effect of the wafer detection method described in the present invention, it is necessary to train this detection model using wafer detection defect images and their corresponding labels. To further illustrate the superiority of the method described in the present invention for wafer defect detection, by comparing with traditional models, the evaluation metrics include accuracy, precision, recall, F1-score, intersection over union (IOU), and Dice coefficient, and their expressions are respectively:

[0114] ,

[0115] ,

[0116] ,

[0117] ,

[0118] ,

[0119] ,

[0120] where TP is the number of positive samples predicted as positive samples; FP is the number of positive samples predicted as negative samples; TN is the number of negative samples predicted as negative samples; FN is the number of negative samples predicted as positive samples; |A| is the number of pixels in the predicted region; |B| is the number of pixels in the true label; |A∩B| is the number of intersection pixels between the prediction and the true label; |A∪B| is the number of union pixels between the prediction and the true label; the F1-score is a weighted average of accuracy and recall, and the larger its value, the better the model; IoU is an index used to measure the degree of overlap between two regions, widely used in tasks such as object detection and image segmentation. It evaluates the prediction accuracy of the model by calculating the ratio of the intersection to the union of the predicted region and the true label region; the Dice Coefficient is an index used to measure the similarity between two sets, widely used in tasks such as image segmentation, especially suitable for evaluating the degree of overlap between the segmentation result and the true label, and is one of the indexes for evaluating the prediction accuracy of the model.

[0121] The detection results are shown in Tables 1 to 6. The wafer defect detection model constructed by the method in the present invention is denoted as the MK-DANet model. Table 1 shows the precision, recall, and F1-score of the MK-DANet model on the WM-811K dataset. This table shows that for 9 types of wafer patterns, the model proposed in the present invention achieves very high precision, recall, and F1-score, with an average of 97% for each.

[0122] Table 2 shows the accuracy comparison of different models for each defect category in the WM-811K dataset. The comparison models used in this example are: Wafer Map Defect Pattern Recognition Model (WMDPI), Wafer Defect Detection Network Model Based on Separable Convolution and Attention Mechanism (WDD-SCA), Wafer Segmentation Classification Network Model (WSCN), and Wafer Map Classification Network Model Based on PeleeNet (WM-PeleeNet). It can be seen from Table 2 that the average classification accuracy of the model in the present invention on the WM-811K dataset is 97%, which is better than all other comparison models, indicating the advantage of the MK-DANet model in single-wafer defect classification.

[0123] Table 1

[0124]

[0125] Table 2

[0126]

[0127] Table 3

[0128]

[0129] Table 4

[0130]

[0131] Table 3 shows the precision, recall, and F1-score of the MK-DANet model on the MixedWM38 dataset. This table shows that for 38 types of wafer patterns, the model proposed in the present invention achieves very high precision, recall, and F1-score, with averages of 98%, 98%, and 97% respectively.

[0132] Table 5

[0133]

[0134] Table 6

[0135]

[0136] Table 4 shows the comparison of the model proposed in the present invention with the wafer defect pattern classification methods in recent years, including Residual Network (ResNet), Deep Cross Network (DCNet), WSCN, WM-PeleeNet. The comparison results of the average accuracy are shown in Table 5, indicating the good generalization of the method proposed in the present invention, and the detection accuracy of single defect, double defect and triple defect is higher than that of the other methods. The detection accuracy of single defect is 99.1%, the detection accuracy of double defect is 98.3%, the detection accuracy of triple defect is 97.9%, and the detection accuracy of quadruple defect also reaches 96.5%. The average classification accuracy of the model proposed in the present invention on the MixedWM38 dataset is 98.2%, which is better than all other comparison models, indicating the advantage of the MK-DANet model in single wafer defect classification.

[0137] Table 6 is the comparison of the number of parameters of each model and the accuracy on the WM-811K dataset and the MixedWM38 dataset. The comparison models used include LeNet, AlexNet, VGG16, MobileNetV2, Residual Network (ResNet18), WM-PeleeNet, PeleeNet, Dense Connection Network (DenseNet121), Lightweight Compression Network (Squeezenet), WSCN. Table 6 shows that while improving the generalization ability and detection ability of the model proposed in the present invention, the number of parameters of the model is reduced as much as possible.

[0138] Table 7 is the comparison of the intersection over union and Dice coefficient of each model on the WM-811K dataset and the MixedWM38 dataset. The comparison models used include U-Net, DeepLabV3+ (based on Residual Network-50 backbone) and WSCN. It can be seen from Table 7 that the MK-DANet model of the present invention has good defect segmentation ability compared with other models.

[0139] Table 7

[0140]

Claims

1. A wafer defect detection method based on multi-core attention fusion and dynamic large-core jumping, characterized in that Including the following steps: Obtain the wafer image to be detected and perform image preprocessing; Input the preprocessed image into the first MKCS-SE module for feature extraction, and then perform max pooling on the extracted feature map to obtain the first feature map; Input the first feature map into the second MKCS-SE module for feature extraction, and then perform max pooling on the extracted feature map to obtain the second feature map; Input the second feature map into the third MKCS-SE module for feature extraction, and then perform max pooling on the extracted feature map to obtain the third feature map; Input the third feature map into the fourth MKCS-SE module for feature extraction, and then perform average pooling on the extracted feature map to obtain the fourth feature map; Input the fourth feature map into the fifth MKCS-SE module for feature extraction, and then perform average pooling on the extracted feature map to obtain the fifth feature map; Input the first to fourth feature maps into DALKConv-SC module to obtain four feature maps of different sizes, denoted as the sixth, seventh, eighth, and ninth feature maps respectively; Use the fifth to ninth feature maps for fusion decoding to obtain the defect segmentation result; Input the fifth feature map into the global average pooling layer and the activation function layer in sequence to obtain the defect classification result.

2. The wafer defect detection method based on multi-core attention fusion and dynamic large-core jumping according to claim 1, wherein Input the first to fourth feature maps into DALKConv-SC The steps of the module include: Take one of the first, second, third, or fourth feature maps as the input feature map, input the input feature map into the first depthwise separable convolution and the second depthwise separable convolution respectively for large kernel feature extraction, obtain the first large kernel feature map and the second large kernel feature map respectively, after fusing the first large kernel feature map and the second large kernel feature map, perform average pooling and max pooling respectively to obtain the third large kernel feature map and the fourth large kernel feature map; Input the third largest kernel feature map and the fourth largest kernel feature into the third depthwise separable convolution for channel fusion, and after passing the result of the channel fusion through Sigmod function activation, the fifth largest kernel feature is obtained; Perform an operation on the fifth largest kernel feature map and the first largest kernel feature map Scale to obtain the seventh largest kernel feature map; Perform an operation on the fifth largest kernel feature map and the second largest kernel feature map Scale to obtain the eighth largest kernel feature map; Perform a merging operation on the seventh largest kernel feature map, the eighth largest kernel feature map, and the input feature map to obtain DALKConv-SC the feature map finally output by the module.

3. The wafer defect detection method based on multi-core attention fusion and dynamic large-core jumping according to claim 1, wherein The steps of using the fifth to ninth feature maps for fusion decoding to obtain the defect segmentation result include: After fusing the ninth feature map and the fifth feature map, use Transpose-Conv module for decoding to obtain the tenth feature map; After fusing the eighth feature map and the tenth feature map, use Transpose-Conv module for decoding to obtain the eleventh feature map; After fusing the seventh feature map and the eleventh feature map, use Transpose-Conv module for decoding to obtain the twelfth feature map; Fuse the sixth feature map and the twelfth feature map, and use Transpose-Conv module for decoding to obtain the thirteenth feature map; Resize the thirteenth feature map through depthwise separable convolution and use Sigmod function to obtain the output segmentation result.

4. The wafer defect detection method based on multi-core attention fusion and dynamic large-core jump according to claim 2, characterized in that, First MKCS-SE module to the fifth MKCS-SE The structures of the modules are the same. The process of feature extraction includes: Perform preliminary feature extraction on the input feature map through depthwise separable convolution, and then adjust the channels of the obtained preliminary feature map through depthwise separable convolution to adjust the channels to a multiple of four; Then perform channel segmentation of the adjusted feature map into four equal parts; Use four depthwise separable convolutions with different sizes to perform feature extraction of different scales on the feature maps obtained by dividing the channels into four equal parts; Merge the feature maps extracted at four different scales, perform a residual connection with the preliminary feature map, and input the feature map after the residual connection into SE the attention module; The feature map obtained by processing through SE the attention module is input into the depthwise separable convolution for channel adjustment, and the channels are adjusted to a preset number of channels for output.

5. The wafer defect detection method based on multi-core attention fusion and dynamic large-core jumping according to any one of claims 1 to 4, characterized in that, And the steps of performing image preprocessing include: Adjust the size of the defect image to a preset size, perform median filtering, and perform data augmentation for unbalanced categories.

6. The wafer defect detection method based on multi-core attention fusion and dynamic large-core jumping according to claim 5, wherein, Data augmentation includes vertically or horizontally flipping the image, vertically or horizontally moving the image, and rotating or scaling the image.

Images