Wafer defect detection method based on improved convolutional neural network with integrated attention mechanism

Abstract

The application provides an improved convolutional neural network wafer defect detection method with integrated attention mechanism, and relates to the technical field of wafer defect detection.S1, a Gaussian kernel is used for convolution operation with an original image;S2, the image is divided into a plurality of non-overlapping sub-blocks, histogram equalization is performed on each sub-block, and then the results of the sub-blocks are spliced through a bilinear interpolation method;S3, a channel attention module and a spatial attention module are used to simultaneously perform adaptive weighting on features in two dimensions of channels and spaces to obtain a feature map;S4, the feature map after attention enhancement is flattened into a one-dimensional vector, then feature fusion is performed through a fully connected layer, and finally the probability of each category is output by a softmax layer.The application has high practical application value and can provide strong technical support for intelligent quality detection of semiconductor manufacturing enterprises.

Description

Technical Field

[0001] This invention relates to the field of wafer defect detection technology, and more particularly to an improved convolutional neural network wafer defect detection method integrating an attention mechanism. Background Technology

[0002] Wafers are the core substrate material for manufacturing semiconductor chips. As chip manufacturing processes advance to 7nm, 5nm, and even more advanced nodes, the linewidth of circuits on wafers is becoming increasingly narrow, placing unprecedented demands on wafer surface quality. Even micron-level scratches, stains, cracks, or material defects can cause the entire chip to malfunction, directly reducing production yield and causing significant economic losses to companies. Therefore, efficient and accurate wafer defect detection has become an indispensable and critical process in semiconductor manufacturing.

[0003] Traditional wafer defect inspection relies primarily on two methods: manual visual inspection and machine vision inspection based on manual features. Manual inspection depends entirely on the operator's experience and condition, resulting in extremely low efficiency, high labor intensity, and susceptibility to subjective factors such as fatigue and mood, leading to high false positive and false negative rates, making it completely unsuitable for the demands of modern large-scale semiconductor production. While machine vision methods based on manual features improve inspection efficiency to some extent, they require engineers to manually design feature extraction operators based on experience, such as edge detection and texture features. This method is effective for defects with simple shapes and obvious features, but it has poor adaptability and limited feature extraction capabilities when faced with complex, varied, and irregularly shaped micro-defects, failing to meet the requirements of high-precision inspection.

[0004] In recent years, the rapid development of deep learning technology has provided new ideas for solving this problem. Convolutional Neural Networks (CNNs) can automatically learn feature representations of defects from large amounts of labeled data without the need for manual feature design, demonstrating performance far exceeding traditional methods in image recognition tasks. However, existing deep learning-based wafer defect detection methods still have many problems in practical industrial applications: First, ordinary CNN models treat features of all regions equally, failing to automatically focus on key regions containing defects, causing the features of small defects to be easily submerged by large areas of background information, resulting in a high false negative rate; second, most methods only focus on feature enhancement in the channel dimension, ignoring the importance of spatial location information for accurate defect localization; third, wafer defect datasets generally suffer from severe class imbalance, with the number of normal wafers far exceeding that of defective wafers. When training with the traditional cross-entropy loss function, the model is dominated by the overwhelmingly numerous normal samples, resulting in insufficient ability to identify defective samples; finally, the complex environment of industrial production sites, where acquired wafer images are often affected by Gaussian noise and uneven illumination, further increases the difficulty of detection.

[0005] In summary, current mainstream wafer defect detection technologies have the following main shortcomings:

[0006] Traditional machine learning methods (such as support vector machines (SVM) and random forests) rely excessively on manually designed features, which have limited expressive power. They have low accuracy in detecting defects with complex and irregular shapes, and poor generalization ability. Once the production line process or defect type changes, the features need to be redesigned, resulting in high maintenance costs.

[0007] While methods based on ordinary convolutional neural networks achieve automatic feature extraction, they lack effective attention mechanisms and cannot adaptively highlight key defect features. When detecting minute defects, features are easily submerged by the background, leading to missed detections. Furthermore, most of these methods do not consider the class imbalance of wafer defect data, and the model training tends to predict normal samples, resulting in generally low recall rates for defective samples. This is fatal for industrial production, as missing even one defective wafer can lead to the scrapping of a large number of subsequent chips. In addition, most methods lack targeted image preprocessing steps, failing to effectively suppress noise and lighting interference in industrial environments, affecting the stability and robustness of the detection system. Summary of the Invention

[0008] To address the aforementioned problems, this invention provides an improved convolutional neural network method for wafer defect detection integrating an attention mechanism. This invention aims to achieve the following objectives:

[0009] 1. Significantly improves the detection accuracy of minute and irregular defects on the wafer surface, and reduces the false detection rate and missed detection rate;

[0010] 2. Enhance the model's ability to generalize to different types of defects, adapting to the detection needs of different processes and production lines;

[0011] 3. Effectively solves the severe class imbalance problem in wafer defect datasets, significantly improving the recall rate of defect samples;

[0012] 4. Improve the detection system's resistance to interference from industrial noise and uneven lighting, and ensure the stability of the detection results;

[0013] 5. While ensuring high precision, it meets the speed requirements of real-time online testing in semiconductor production lines.

[0014] To solve the above problems, the technical solution adopted by the present invention is as follows:

[0015] An improved convolutional neural network method for wafer defect detection with integrated attention mechanism includes the following steps: S1, performing convolution operation with the original image using a Gaussian kernel; S2, dividing the image into multiple non-overlapping sub-blocks, performing histogram equalization on each sub-block, and then concatenating the results of each sub-block using bilinear interpolation; S3, using channel attention module and spatial attention module to adaptively weight the features in both channel and spatial dimensions to obtain a feature map. S4. Flatten the attention-enhanced feature map into a one-dimensional vector, then perform feature fusion through a fully connected layer, and finally output the probability of each category by a softmax layer.

[0016] Preferably, the expression for the Gaussian kernel function is:

[0017] ;

[0018] In the formula, Image pixel coordinates, and These are the mean values ​​of the Gaussian kernel in the x and y directions, respectively.

[0019] Preferably, step S2 uses the cumulative distribution function of the sub-block histogram for calculation. The cumulative distribution function of the sub-block histogram is as follows:

[0020] ;

[0021] In the formula, Image grayscale level (0~255). For the sub-block, the gray level is The number of pixels, This represents the total number of pixels in the sub-block.

[0022] Preferably, based on the traditional convolutional neural network, a channel attention module and a spatial attention module are introduced to construct the IA-CNN model. This model consists of four convolutional blocks, one CBAM attention module, two fully connected layers, and one softmax classification layer. Each convolutional block contains a convolutional layer, a batch normalization layer, and a ReLU activation function. The convolutional layers are responsible for extracting features from different levels of the image, from shallow edges and textures to deep semantic features. The batch normalization layer accelerates model convergence and effectively prevents overfitting. The ReLU activation function introduces non-linearity into the model. The basic formula for convolution operation is:

[0023] ;

[0024] In the formula, For the first Each channel is located in Output feature map at that location, The kernel size is [size]. Input the number of channels. For convolution kernel weights, For the input feature map, The bias term; the ReLU activation function is defined as: .

[0025] Preferably, the formula for calculating channel attention weights is:

[0026] ;

[0027] In the formula, For the input feature map, and These represent global average pooling and global max pooling, respectively. We use the Sigmoid activation function; we set the number of hidden layer nodes in the MLP to... To balance model performance and computational cost; the feature map after channel attention weighting is ;in, This indicates element-wise multiplication.

[0028] Preferably, the formula for calculating spatial attention weights is as follows:

[0029] ;

[0030] In the formula, This represents the channel splicing operation; the final feature map after spatial attention weighting is... This data is then fed into subsequent fully connected layers for classification.

[0031] Preferably, the wafer defect dataset uses Focal Loss as the model's loss function; Focal Loss introduces a modulation factor to automatically reduce the weight of easily classified samples while increasing the weight of difficult-to-classify samples, making the model pay more attention to defect samples and effectively improving the recall rate.

[0032] Preferably, the formula for Focal Loss is as follows:

[0033] ;

[0034] In the formula, The total number of samples, To balance the weights across categories, we set the weight of defective samples to 0.25 and the weight of normal samples to 0.75. For the model to predict the first The probability that a sample belongs to the target category; The focus parameter is set to 2, which adjusts the level of attention given to difficult-to-classify samples.

[0035] The beneficial effects of this invention are as follows:

[0036] Compared with existing technologies, this method solves the problems of low detection accuracy, high false negative rate of minute defects, and sample imbalance. It boasts high detection accuracy, high speed, and strong robustness, making it highly valuable for practical applications and providing strong technical support for intelligent quality inspection in semiconductor manufacturing enterprises. Attached Figure Description

[0037] Figure 1 : Overall framework diagram of the IA-CNN wafer defect detection algorithm of this invention.

[0038] Figure 2 Internal structure diagram of the CBAM channel-spatial attention module.

[0039] Figure 3 Confusion matrix of IA-CNN model on test set (normal vs. defective).

[0040] Figure 4 Comparison of ROC curves for different detection models.

[0041] Figure 5 Visual comparison of defect detection results between Base-CNN and IA-CNN models (including central defects, edge localization defects, edge loop defects, localization defects, random defects, and scratch defects). Detailed Implementation

[0042] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0043] The technical solution adopted in this invention is divided into three core modules: an image preprocessing module, an improved convolutional neural network feature extraction module (IA-CNN), and a defect classification module. First, the acquired wafer images are preprocessed to remove noise and enhance defect contrast. Then, the preprocessed images are input into the IA-CNN model for feature extraction, and key defect features are enhanced by introducing an attention mechanism. Finally, an improved loss function is used for classification to solve the problem of imbalanced samples.

[0044] See attached document Figure 1 - Appendix Figure 5 An improved convolutional neural network method for wafer defect detection integrating an attention mechanism is proposed, which uses an image preprocessing module, an improved convolutional neural network feature extraction module, and a defect classification module.

[0045] (a) Image preprocessing module

[0046] Wafer images acquired in industrial settings often suffer from high noise and uneven lighting, which can severely impact inspection results if directly input into the model. Therefore, we designed a two-step preprocessing workflow:

[0047] The first step is Gaussian noise reduction. Gaussian noise is the most common type of noise in wafer images. We use a Gaussian kernel to convolve with the original image to effectively suppress noise while preserving as much edge and detail information of defects as possible.

[0048] The Gaussian kernel function is expressed as follows, where we set the standard deviation σ to 0.8, achieving a good balance between denoising effect and detail preservation.

[0049]

[0050] In the formula, Image pixel coordinates, and These are the mean values ​​of the Gaussian kernel in the x and y directions, respectively.

[0051] The second step is Adaptive Histogram Equalization (AHE). To address the issue of low contrast between defects and the background caused by uneven illumination, we divide the image into multiple non-overlapping sub-blocks, perform histogram equalization on each sub-block separately, and then stitch the results together using bilinear interpolation to avoid block artifacts. This effectively enhances the contrast of local areas, making previously inconspicuous defect features stand out. The cumulative distribution function (CDF) of the sub-block histogram is calculated as follows:

[0052]

[0053] In the formula, Image grayscale level (0~255). For the sub-block, the gray level is The number of pixels, This represents the total number of pixels in the sub-block.

[0054] (ii) Improved Convolutional Neural Network Feature Extraction Module (IA-CNN)

[0055] To address the issue of insufficient attention to key features in ordinary CNNs, we introduced a Channel-Spatial Attention Module (CBAM) to build the IA-CNN model based on traditional convolutional neural networks. This model mainly consists of four convolutional blocks, one CBAM attention module, two fully connected layers, and one softmax classification layer.

[0056] Each convolutional block contains a convolutional layer, a batch normalization (BN) layer, and a ReLU activation function. The convolutional layer extracts features from different levels of the image, from shallow edges and textures to deep semantic features; the batch normalization layer accelerates model convergence and effectively prevents overfitting; the ReLU activation function introduces non-linearity into the model, enhancing its ability to express complex features. The basic formula for convolution operations is:

[0057]

[0058] In the formula, For the first Each channel is located in Output feature map at that location, The kernel size is [size]. Input the number of channels. For convolution kernel weights, For the input feature map, This is a bias term.

[0059] The ReLU activation function is defined as:

[0060]

[0061] The CBAM attention module is one of the core innovations of this invention. It is composed of a cascaded channel attention module (CAM) and a spatial attention module (SAM), which can adaptively weight features in both channel and spatial dimensions.

[0062] The channel attention module automatically learns the importance of each channel, enhancing channels containing defective information and suppressing irrelevant channels. Specifically, we perform global average pooling and global max pooling on the input feature map, obtaining two 1×1×C feature vectors. These two vectors are then fed into a shared multilayer perceptron (MLP) for processing. Finally, the two processed vectors are summed and activated using the sigmoid function to obtain the final channel attention weights. The calculation formula is as follows:

[0063]

[0064] In the formula, For the input feature map, and These represent global average pooling and global max pooling, respectively. We use the Sigmoid activation function. We set the number of hidden layer nodes in the MLP to... To balance model performance and computational cost, the feature map after channel attention weighting is as follows:

[0065]

[0066] in This indicates element-wise multiplication.

[0067] The spatial attention module, building upon channel attention, further focuses on the spatial location of defects. We then examine the feature map output by the channel attention module. Global average pooling and global max pooling are performed along the channel dimension to obtain two H×W×1 feature maps. These two maps are then concatenated along the channel dimension, and a 1×1 convolutional layer is applied to reduce the number of channels to 1. Finally, the model is activated using the sigmoid function to obtain spatial attention weights. This allows the model to automatically focus on regions containing defects in the image and suppress interference from background information. The calculation formula is as follows:

[0068]

[0069] In the formula, This indicates the channel splicing operation. The final feature map after spatial attention weighting is... This data is then fed into subsequent fully connected layers for classification.

[0070] (III) Defect Classification Module

[0071] The classification module consists of two fully connected layers and one softmax layer. First, the attention-enhanced feature maps are... The vector is flattened into a one-dimensional vector, then features are fused through a fully connected layer, and finally the probability of each class is output by a softmax layer.

[0072] To address the severe class imbalance problem in wafer defect datasets, we did not use the traditional cross-entropy loss function, but instead adopted Focal Loss as the model's loss function. Focal Loss, by introducing a modulation factor, automatically reduces the weight of easily classified samples (mainly normal samples) while increasing the weight of difficult-to-classify samples (mainly defect samples), thus making the model focus more on defect samples and effectively improving recall. The formula for Focal Loss is as follows:

[0073]

[0074] In the formula, The total number of samples, To balance the weights across categories, we set the weight of defective samples to 0.25 and the weight of normal samples to 0.75. For the model to predict the first The probability that a sample belongs to the target category; The focus parameter is set to 2, which adjusts the level of attention given to difficult-to-classify samples.

[0075] Model training employs the Adam optimizer to minimize the loss function and update the model parameters. The Adam optimizer combines the advantages of momentum and RMSprop methods, adaptively adjusting the learning rate, exhibiting fast convergence and good stability. Its parameter update formula is as follows:

[0076]

[0077] In the formula, and These are the first and second moments of the gradient, respectively. and The exponential decay rate is set to 0.9 and 0.999, respectively. For a moment The gradient of the loss function; and This is the moment estimate after bias correction; The learning rate is set to 0.001. It is a very small constant used to avoid the denominator being zero, and is set to 1e-8; For a moment Model parameters.

[0078] The present invention achieves significant technical effects through the organic combination of the above technical solutions, mainly reflected in the following aspects:

[0079] First, detection accuracy is significantly improved. We conducted comprehensive testing on the widely used public WM-811K wafer defect dataset. The results show that the IA-CNN algorithm proposed in this invention achieves a detection accuracy of 98.76%, a recall rate of 97.92%, and an F1 score of 98.34%. Compared with traditional machine learning methods such as SVM and Random Forest, all indicators are improved by 6% to 8%; compared with mainstream deep learning models such as ResNet18 and SE-CNN, the accuracy is improved by 1.32% to 2.53%, and the recall rate is improved by 2.79% to 5.36%, which is very significant. In particular, for minor defects such as scratches and localization defects, the false negative rate is reduced by more than 60%.

[0080] Secondly, it effectively solves the problem of imbalanced samples. By introducing the Focal Loss function, the model is no longer dominated by a large number of normal samples, and its ability to identify defective samples is significantly enhanced. In the test set, the recall rate of defective samples increased from 89.24% for ordinary CNNs to 97.92%, which is crucial for industrial production, as it can minimize the flow of defective wafers into the next process and reduce subsequent losses.

[0081] Third, the model exhibits strong generalization ability and robustness. The introduction of the CBAM attention mechanism enables the model to simultaneously focus on key features in both channel and spatial dimensions, resulting in excellent recognition performance for defects of different types and shapes. Simultaneously, the preprocessing module effectively suppresses the effects of noise and uneven illumination, allowing the model to maintain stable detection performance even in complex industrial environments.

[0082] Finally, the detection speed meets real-time requirements. Although an attention mechanism was introduced, we optimized the model structure and controlled the computational load. Test results show that the model's average inference time for a single image is only 28.3 milliseconds, which translates to processing more than 35 images per second, fully meeting the needs of high-speed online inspection in semiconductor production lines.

[0083] The present invention will be further described below with reference to specific embodiments.

[0084] (I) Experimental Environment and Dataset

[0085] This experiment is based on the PyTorch deep learning framework, and all code is written in Python. The experiment runs on a high-performance workstation with an Intel Core U7 CPU, 32GB of RAM, and an NVIDIA RTX 5060 dedicated graphics card.

[0086] The dataset used in the experiment is WM-811K, which is currently the largest and most authoritative publicly available wafer defect dataset in the semiconductor field. This dataset contains 811,457 wafer images, covering six common defect types (center defect, edge-localized defect, edge-ring defect, localized defect, random defect, and scratch defect) and one normal category. The sample size and percentage for each category are shown in the table below. It can be seen that the normal sample percentage is as high as 70.37%, indicating a very serious class imbalance problem. Specific data are shown in Table 1.

[0087] Table 1

[0088]

[0089] To ensure the objectivity and reproducibility of the experimental results, we randomly divided the dataset into training, validation, and test sets in a 7:2:1 ratio. Meanwhile, to prevent overfitting and improve generalization ability, we performed data augmentation on the training set, including random rotation (0°~360°), horizontal flipping, and vertical flipping.

[0090] (II) Model Training Process

[0091] The main training parameters for the IA-CNN model are set as follows: a total of 100 epochs, a batch size of 32, an initial learning rate of 0.001, an Adam optimizer, and Focal Loss (α=0.25, γ=2). During training, a validation set is used to monitor the model's performance. If the validation set loss does not decrease for 10 consecutive epochs, the learning rate is automatically reduced to 1 / 10 of its original value. After training is complete, the best-performing model on the validation set is saved for final testing.

[0092] (III) Experimental Results and Analysis

[0093] To comprehensively evaluate the performance of this invention, we compared IA-CNN with several mainstream methods, including traditional machine learning methods (SVM, Random Forest) and deep learning methods (Base-CNN, ResNet18, SE-CNN). All comparison methods were trained and tested using the same dataset and experimental environment, and the results are shown in Table 2.

[0094] Table 2

[0095]

[0096] The comparison results clearly show that the IA-CNN algorithm proposed in this invention significantly outperforms other methods in all evaluation metrics. In particular, the recall rate is 2.79 percentage points higher than that of the second-best performing SE-CNN, indicating that our method is highly effective in reducing false negatives.

[0097] To verify the individual contributions of each module, we also conducted ablation experiments, the results of which are shown in the table below. It can be seen that the image preprocessing module improved the accuracy by 1.92%, the CBAM attention module further improved it by 1.23%, and the FocalLoss loss function improved it by 0.70%. This indicates that the three core modules we proposed all play an important role in performance improvement, and their combination achieves the best detection results, as detailed in Table 3.

[0098] Table 3

[0099]

[0100] In terms of inference speed, IA-CNN has an average inference time of 28.3 milliseconds per image. Although it is slightly slower than the simplest Base-CNN, it is much faster than ResNet18's 35.6 milliseconds, which fully meets the requirements of industrial real-time detection.

[0101] We also performed a visual analysis of the test results. From Figure 5It can be seen that the Base-CNN model has significant false negatives and false positives for minor scratches and localization defects, while the IA-CNN model can accurately identify all types of defects, including those that are very subtle and difficult to detect. This fully demonstrates the effectiveness of the attention mechanism and Focal Loss.

[0102] (iv) Conclusion

[0103] This embodiment verifies in detail the effectiveness of the wafer defect detection method proposed in this invention. Experimental results show that by combining image preprocessing, the CBAM attention mechanism, and the Focal Loss loss function, we successfully solve the problems of low detection accuracy, high false negative rate of small defects, and sample imbalance in existing technologies. This method has high detection accuracy, fast speed, and strong robustness, and has high practical application value, providing strong technical support for intelligent quality inspection in semiconductor manufacturing enterprises.

[0104] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An improved convolutional neural network method for wafer defect detection integrating an attention mechanism, characterized in that, Includes the following steps: S1. Perform a convolution operation between the original image and a Gaussian kernel; S2. Divide the image into multiple non-overlapping sub-blocks, perform histogram equalization on each sub-block, and then stitch the results of each sub-block together using bilinear interpolation. S3. Use the channel attention module and the spatial attention module to adaptively weight the features in both the channel and spatial dimensions to obtain the feature map. ; S4. Flatten the attention-enhanced feature map into a one-dimensional vector, then perform feature fusion through a fully connected layer, and finally output the probability of each category by a softmax layer.

2. The improved convolutional neural network wafer defect detection method with integrated attention mechanism according to claim 1, characterized in that, The expression for the Gaussian kernel function is: ； In the formula, Image pixel coordinates, and These are the mean values ​​of the Gaussian kernel in the x and y directions, respectively.

3. The improved convolutional neural network wafer defect detection method with integrated attention mechanism according to claim 1, characterized in that, Step S2 uses the cumulative distribution function of the sub-block histogram for calculation. The cumulative distribution function of the sub-block histogram is as follows: ； In the formula, Image grayscale level (0~255). For the sub-block, the gray level is The number of pixels, This represents the total number of pixels in the sub-block.

4. The improved convolutional neural network wafer defect detection method with integrated attention mechanism according to claim 1, characterized in that, Based on traditional convolutional neural networks, channel attention modules and spatial attention modules are introduced to construct the IA-CNN model; the model consists of 4 convolutional blocks, 1 CBAM attention module, 2 fully connected layers and 1 softmax classification layer; Each convolutional block contains a convolutional layer, a batch normalization layer, and a ReLU activation function. The convolutional layer is responsible for extracting features at different levels from the image, from shallow edges and textures to deep semantic features. The batch normalization layer can accelerate model convergence and effectively prevent overfitting. The ReLU activation function introduces non-linearity into the model. The basic formula for convolution operation is: ； In the formula, For the first Each channel is located in Output feature map at that location, The kernel size is [size]. Input the number of channels. For convolution kernel weights, For the input feature map, For bias terms; The ReLU activation function is defined as: 。 5. The improved convolutional neural network wafer defect detection method with integrated attention mechanism according to claim 4, characterized in that, The formula for calculating channel attention weights is: ； In the formula, For the input feature map, and These represent global average pooling and global max pooling, respectively. We use the Sigmoid activation function; we set the number of hidden layer nodes in the MLP to... To balance model performance and computational cost; the feature map after channel attention weighting is ； in, This indicates element-wise multiplication.

6. The improved convolutional neural network wafer defect detection method with integrated attention mechanism according to claim 4, characterized in that, The formula for calculating spatial attention weights is as follows: ； In the formula, This indicates a channel splicing operation; The final feature map after spatial attention weighting is This data is then fed into subsequent fully connected layers for classification.

7. The improved convolutional neural network wafer defect detection method with integrated attention mechanism according to claim 1, characterized in that, The wafer defect dataset uses Focal Loss as the model's loss function. Focal Loss introduces a modulation factor to automatically reduce the weight of easily classified samples while increasing the weight of difficult-to-classify samples, making the model pay more attention to defect samples and effectively improving the recall rate.

8. The improved convolutional neural network wafer defect detection method with integrated attention mechanism according to claim 7, characterized in that, The formula for Focal Loss is as follows: ； In the formula, The total number of samples, To balance the weights across categories, we set the weight of defective samples to 0.25 and the weight of normal samples to 0.

75. For the model to predict the first The probability that a sample belongs to the target category; The focus parameter is set to 2, which adjusts the level of attention given to difficult-to-classify samples.

Images