Wafer defect detection method based on PeleeNet and CNN

Abstract

The invention discloses a dual-path defect detection method based on PeleeNet and CNN, belongs to the field of semiconductor manufacturing, and particularly relates to a wafer defect detection technology. In order to solve the problems of insufficient precision, limited feature extraction capability and the like when an existing wafer defect detection method is used for processing complex geometric morphology and multi-scale defect information, the invention provides a dual-path detection model which combines a PeleeNet classification network, a convolutional neural network (CNN) and an SE attention layer (SE attention layer), and the dual-path detection model is called a DPSE-WDDN network. According to the method, the multi-scale features are respectively extracted through a double-path structure, and the feature channel weight is adaptively adjusted, so that the accuracy and efficiency of wafer defect detection are remarkably improved. The method is suitable for scenes such as quality control in semiconductor manufacturing, automatic detection of a production line, quality evaluation in a wafer supply chain and the like.

Description

Technical Field

[0001] The present invention relates to the field of semiconductor manufacturing technology, and in particular to a wafer defect detection method based on PeleeNet and CNN. Background Art

[0002] In recent years, with the rapid development of the semiconductor industry, the importance of wafer defect detection technology has become increasingly prominent. Wafer defect detection is a key step to ensure chip production quality and improve production yield. However, there may be a variety of complex defect types on the wafer surface, such as center defects, edge defects, and random defects. These defects are not only numerous in variety, but also in various forms, which brings great challenges to the detection task.

[0003] Traditional wafer inspection methods usually rely on experienced semiconductor engineers to analyze the size, shape, and location of defects in wafer inspection images through visual recognition. However, this method is not only time-consuming and labor-intensive, but also costly when faced with large amounts of data. In addition, the accuracy of manual inspection is relatively low, which is particularly insufficient on large-scale data sets. In addition, as chip size continues to shrink, the difficulty of manual inspection is also increasing. Therefore, the industry has begun to explore the use of available technologies to reduce costs and improve detection accuracy.

[0004] With the advancement of hardware and algorithms, shallow machine learning methods have been widely used in the field of wafer defect detection. Researchers extract Radon features and geometric features of wafers, and then classify them through support vector machines (SVM) to identify wafer defects. Although shallow machine learning has achieved certain results in certain defect recognition tasks, its limitations lie in the low effectiveness of feature extraction and its difficulty in dealing with complex and diverse image features. With the development of deep learning technology, researchers have begun to explore the application of deep learning in the field of wafer image defect detection. Deep learning models can identify and classify complex defect types in large-scale data by automatically learning features in the data, thereby improving the automation and accuracy of detection.

[0005] At present, deep learning-based wafer defect detection methods focus on convolutional neural networks (CNNs). These methods can capture the details of wafer surface defects by extracting multi-level features of images. However, traditional CNN models still have certain limitations when dealing with wafer defects with complex geometric morphology. Specifically, CNN models may lose important spatial features when processing defect information at different scales. In addition, since the distribution of defects in wafer images is usually sparse and irregular, traditional CNN models are insufficient in capturing local structural information. To overcome these challenges, many improved methods have been proposed in recent years, such as multi-scale feature extraction and attention mechanism. Multi-scale feature extraction helps capture the global and local information of defects by combining features at different scales. The attention mechanism enhances the model's sensitivity to key defects by weighting important features. However, these methods still have the problems of high computing resource consumption and long training time when processing large-scale wafer data.

[0006] The above method solves the problem of wafer defect detection to a certain extent, but there are still many shortcomings. For example, wafer defects still have problems such as data imbalance and large differences in the spatial and positional distribution of feature information of different types of wafer images. The recognition effect of using the above model on defects of different types of wafer images will be different, and the recognition and classification accuracy of defect patterns of certain types of wafer images is still low. Therefore, how to extract accurate defect feature information for different types of wafer images and further improve the overall recognition and classification accuracy of wafer images is the research focus in this field. Summary of the invention

[0007] In view of the shortcomings of the prior art described above, the present invention provides a wafer defect detection method based on PeleeNet and CNN. The present invention is based on defect detection of the DPSE-WDDN network, which includes defect detection and classification. By combining the dual-path model constructed by PeleeNet and convolutional neural network (CNN), the defect features in the wafer image are effectively extracted and analyzed, which is suitable for quality control in the wafer production process, production line automation detection, and defect identification and classification in wafer reuse.

[0008] To achieve the above effects, the technical solution of the present invention is as follows:

[0009] The present invention provides a wafer defect detection method based on PeleeNet and CNN, comprising the following steps:

[0010] Step 1: Image preprocessing: adjust the input wafer image to be inspected into a three-channel RGB image, and perform median filtering on the wafer image to be inspected;

[0011] Step 2: construct a DPSE-WDDN network, and use the DPSE-WDDN network to perform wafer defect detection on the wafer to be detected image; the DPSE-WDDN network includes a PeleeNet branch, a Conv-SE branch, and a first fully connected layer, an SE attention layer, a Spatial-Dropout layer, a maximum pooling layer, a Flatten layer, a Dense feature extraction layer, and a SoftMax layer connected in sequence; the output ends of the PeleeNet branch and the Conv-SE branch are connected to the input end of the first fully connected layer;

[0012] The PeleeNet branch includes a Stem feature extraction layer, a Dense feature extraction layer, a Transition layer, and an average pooling layer connected in sequence; the Conv-SE branch includes four Conv-SE layers connected in sequence;

[0013] Step 3: Use the PeleeNet branch and the Conv-SE branch to extract features of the wafer image to be inspected, and obtain the fourth feature map and the fifth feature map respectively, wherein the PeleeNet branch includes:

[0014] Step 3.1: Input the pre-processed wafer image to be inspected into the Stem feature extraction layer in the Peleenet network to obtain the first feature map;

[0015] Step 3.2: Input the first feature map into the Dense feature extraction layer for feature extraction to obtain the second feature map; the second feature map is input into the Transition layer in the Peleenet network for transition processing to obtain the third feature map;

[0016] The Dense feature extraction layer in step 3.2 is a three-branch feature extraction module, including a Conv layer, a maximum pooling layer, and a Concatenate layer;

[0017] Step 3.3: Perform average pooling on the third feature map that passes through the Transition layer in the Peleenet network, adjust the size of the feature map, and output the fourth feature map;

[0018] The Conv-SE layer is composed of a first 3x3Conv layer, a RELU activation function layer, a normalization layer BN, a maximum pooling layer, a second 3x3Conv layer, a normalization layer BN, and a SE attention layer connected in sequence;

[0019] Input the preprocessed wafer image to be inspected into four sequentially connected Conv-SE layers for feature extraction to obtain a fifth feature map;

[0020] Step 4: Dual-path feature fusion:

[0021] Step 4.1: The fourth feature map obtained by the output of the PeleeNet branch and the fifth feature map obtained by the output of the Conv-SE branch are fused through the first fully connected layer to obtain the sixth feature map;

[0022] Step 4.2: Input the sixth feature map into the Spatial-Dropout layer for regularization, and then input the output of the Spatial-Dropout layer into the maximum pooling layer of the DPSE-WDDN network to obtain the seventh feature map;

[0023] Step 4.3: Input the seventh feature map into the Flatten layer to flatten the obtained eighth feature map, input the eighth feature map into the Dense feature extraction layer of the DPSE-WDDN network to obtain the ninth feature map, input the ninth feature map into the SoftMax layer of the DPSE-WDDN network for classification, and obtain the wafer defect detection result.

[0024] Furthermore, step 1 also includes:

[0025] The data of the unbalanced categories is enhanced for the wafer inspection images. The data enhancement process includes vertically or horizontally flipping the images, vertically or horizontally moving the images, and rotating or scaling the images.

[0026] Furthermore, the Stem feature extraction layer in step 3.1 is composed of a Conv layer, a maximum pooling layer and a Concatenate layer.

[0027] Furthermore, the Transition layer in step 3.2 is composed of a normalization layer BN, a RELU activation function layer, a Conv layer and an average pooling layer.

[0028] Furthermore, the Conv-SE layer is composed of a Conv layer, a SE attention layer, a normalization layer BN, a maximum pooling layer, a Conv layer, and a normalization layer BN connected in sequence.

[0029] Furthermore, the SE attention layer includes a first branch and a second branch, and the second branch includes a global pooling layer, a second fully connected layer, a RELU activation function, a third fully connected layer, and a Sigmod function connected in sequence.

[0030] Furthermore, the regularization process of the Spatial Dropout layer in step 4.2 is as follows:

[0031] The probability of discarding the feature map is set to p, and the probability of retaining it is 1-p. For each channel c (c = 1, 2, ..., C), the retained channel is randomly selected; it is represented as generating a binary mask:

[0032]

[0033] Among them, M C is the value of the mask at the Cth channel.

[0034] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0035] (1) The present invention adopts a dual-path structure, and extracts multi-scale image features through two paths, PeleeNet and convolutional neural network (CNN), respectively, so that the model performs better when processing wafer defects of different scales and shapes, and significantly improves the accuracy of defect detection. Due to the use of the dual-path structure and attention mechanism, the model of the present invention has good generalization ability, can adapt to different types of wafer defect detection tasks, and has strong practical value.

[0036] (2) The present invention integrates the lightweight architecture of PeleeNet and utilizes its densely connected structure and multi-channel feature fusion technology to efficiently extract and process rich features in wafer images, thereby reducing computational complexity and ensuring high-quality feature extraction.

[0037] (3) By introducing the SE attention layer, the present invention can adaptively adjust the weight of each feature channel, so that the model pays more attention to key defect information and suppresses irrelevant features, thereby enhancing the feature representation ability and detection performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is the overall structural diagram of the DPSE-WDDN network in the specific implementation scheme of the present invention.

[0039] Figure 2 It is the category image of the WM-811K dataset.

[0040] Figure 3 It is the result of median filtering of the category images of the WM-811K dataset.

[0041] Figure 4 This is the specific structure diagram of the Stem feature extraction layer in the PeleeNet branch.

[0042] Figure 5 This is the specific structure diagram of the Dense Block module in the PeleeNet branch.

[0043] Figure 6 This is the specific structure diagram of the Conv-SE branch.

[0044] Figure 7 This is the specific structure diagram of the SE attention layer.

[0045] Figure 8This is the verification result of the DPSE-WDDN network on the WM-811K dataset. DETAILED DESCRIPTION

[0046] The following will describe the embodiments of the present invention with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention, not for limiting the scope of protection of the present invention.

[0047] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and thus the drawings only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0048] Example

[0049] This embodiment proposes a wafer defect detection method based on PeleeNet and CNN. Figure 1 , including the following steps:

[0050] Step 1: Image preprocessing: The input wafer image to be inspected is resized to a three-channel RGB image of 224×224×3, and the wafer image to be inspected is median filtered; this process helps train the model to have a high tolerance for changes in the size, position, and direction of defects in the image. Figure 3 It is the result of median filtering of the category images of the WM-811K dataset.

[0051] Step 2: construct a DPSE-WDDN network, and use the DPSE-WDDN network to perform wafer defect detection on the wafer to be detected image; the DPSE-WDDN network includes a PeleeNet branch, a Conv-SE branch, and a first fully connected layer, an SE attention layer, a Spatial-Dropout layer, a maximum pooling layer, a Flatten layer, a Dense feature extraction layer, and a SoftMax layer connected in sequence; the output ends of the PeleeNet branch and the Conv-SE branch are connected to the input end of the first fully connected layer;

[0052] The PeleeNet branch includes a Stem feature extraction layer, a Dense feature extraction layer, a Transition layer (i.e., a transition layer TL) and an average pooling layer (Average Pooling) connected in sequence; the Conv-SE branch includes four Conv-SE layers connected in sequence;

[0053] Step 3: Use the PeleeNet branch and the Conv-SE branch to extract features of the wafer image to be inspected, and obtain the fourth feature map and the fifth feature map respectively, wherein the PeleeNet branch includes:

[0054] Step 3.1: Input the preprocessed wafer image to be inspected with a size of 224×224×3 into the Stem feature extraction layer in the Peleenet network once to obtain the first feature map with a size of 56×56×32;

[0055] Step 3.2: Input the first feature map into the Dense feature extraction layer for feature extraction to obtain a second feature map of size 56×56×96; the second feature map of 56×56×96 is input into the Transition layer in the Peleenet network for transition processing to obtain a third feature map of size 28×28×128;

[0056] The Dense feature extraction layer (i.e., Dense Block module) in step 3.2 includes a Conv layer and a Concatenate layer; the Dense feature extraction layer is a three-branch feature extraction module, which inputs the 56×56×32 feature map obtained by the Stem feature extraction layer, and then concatenates the three branches to obtain a feature map of size 56×56×96;

[0057] The Transition layer in step 3.2 includes a normalization layer BN (Batch Normalization), an activation function layer RELU (Rectified Linear Unit), a Conv layer, and an average pooling layer (Average Pooling);

[0058] like Figure 5 As shown, the Dense feature extraction layer in step 3.2 is a three-branch feature extraction module, including a Conv layer, a maximum pooling layer, and a Concatenate layer; the three-branch feature extraction module process is as follows:

[0059] The first feature map of size 56×56×32 obtained in step 3.1 is input into a three-branch process: the first branch keeps the first feature map of size 56×56×32 unchanged; the second branch inputs the feature map into a Conv layer with a step size of 1, a padding method of the same, a convolution kernel size of 1×1, and a number of convolution kernels of 64, outputs a feature map of 56×56×64, and then inputs it into a Conv layer with a step size of 1, a padding method of the same, a convolution kernel size of 3×3, and a number of convolution kernels of 32, outputs a feature map of 56×56×32; the third branch inputs the feature map into a step size of 1, a padding method of the same, a convolution kernel size of 3×3, and a number of convolution kernels of 32, outputs a feature map of 56×56×32. The method is the same, the convolution kernel size is 1×1, the number of convolution kernels is 64, and the Conv layer outputs a 56×56×64 feature map. It is then input to a Conv layer with a step size of 1, a padding method of the same, a convolution kernel size of 3×3, and a number of convolution kernels of 32, and outputs a 56×56×32 feature map. It is then input to a Conv layer with a step size of 1, a padding method of the same, a convolution kernel size of 3×3, and a number of convolution kernels of 32, and outputs a 56×56×32 feature map. The three branches are concatenated to obtain an output feature map of 56×56×96.

[0060] Step 3.3: Perform average pooling on the third feature map of 28×28×128 that passes through the Transition layer in the Peleenet network, adjust the size of the feature map, and output the fourth feature map of size 13×13×128;

[0061] Conv-SE branch:

[0062] The Conv-SE layer is composed of a first 3x3Conv layer, a RELU activation function layer, a normalization layer BN, a maximum pooling layer, a second 3x3Conv layer, a normalization layer BN, and a SE attention layer connected in sequence;

[0063] The preprocessed wafer image to be inspected with a size of 224×224×3 is input into four sequentially connected Conv-SE layers for feature extraction. The number of channels of each Conv-SE layer is 16, 32, 64, and 128 respectively, and the fifth feature map with an output size of 13×13×128 is obtained;

[0064] Step 4: Dual-path feature fusion:

[0065] Step 4.1: The 13×13×128 fourth feature map output by the PeleeNet branch and the 13×13×128 fifth feature map output by the Conv-SE branch are fused through the first fully connected layer to obtain the sixth feature map with an output of 13×13×256;

[0066] Step 4.2: Input the sixth feature map of the first fully connected layer with a size of 13×13×256 into the Spatial-Dropout layer for regularization, and then input the output of the Spatial-Dropout layer into the maximum pooling layer of the DPSE-WDDN network to obtain the seventh feature map with an output of 6×6×256;

[0067] Step 4.3: Input the seventh feature map of 6×6×256 into the Flatten layer for flattening to obtain the eighth feature map of size 9216, input the eighth feature map into the Dense feature extraction layer of the DPSE-WDDN network to obtain the ninth feature map of size 512, input the ninth feature map into the SoftMax layer of the DPSE-WDDN network for classification, and obtain the wafer defect detection result.

[0068] It can be understood that PeleeNet (Faster Region-based Convolutional Neural Network) is a deep learning architecture for target detection. This architecture has the characteristics of high accuracy and real-time performance in the field of target detection while having a small model size. It can improve the accuracy of feature extraction of dynamic targets, thereby improving the detection accuracy of dynamic targets.

[0069] As a preferred technical solution, in this embodiment, step 1 also includes:

[0070] The data of the unbalanced categories is augmented for the wafer inspection images. The data augmentation process includes vertically or horizontally flipping the images, vertically or horizontally moving the images, and slightly rotating or scaling the images.

[0071] As a preferred technical solution, in this embodiment, the Stem feature extraction layer in step 3.1 includes a Conv layer (Convlution), a maximum pooling layer (Max Pooling) and a Concatenate layer.

[0072] The Stem feature extraction layer in step 3.1 is a two-branch feature extraction module, such as Figure 4As shown in the figure, it includes a Conv layer, a maximum pooling layer and a Concatenate layer. The process of the two-branch feature extraction module is as follows: First, the preprocessed defect image of size 224×224×3 is input into the Conv layer with a step size of 2, a padding method of the same, a convolution kernel of 3×3, and a number of convolution kernels of 32, and a feature map of 112×112×32 is output. The feature map is respectively input into the Conv layer with a step size of 2, a convolution kernel size of 1×1, and a number of convolution kernels of 16 and the maximum pooling layer with a step size of 2 and a convolution kernel size of 2×2, and the output feature maps of 112×112×16 and 56×56×32 are obtained respectively; secondly, the feature map of 112×112×16 is input into the Conv layer with a step size of 2, a padding method of the same, a convolution kernel of 3×3, and a number of convolution kernels of 32, and a feature map of 56×56×32 is output. At this time, the two sets of 56×56×32 feature maps are concatenated to obtain an output feature map of 56×56×64, and the fused feature map is input into the Conv layer with a step size of 2, a convolution kernel size of 1×1, and the number of convolution kernels of 32 to obtain an output feature map of 56×56×32.

[0073] The expression of the Conv layer (Convlution) in the step 3.1 Stem feature extraction layer is:

[0074] X CONV =Conv2D(X ReLU , k, k, f)

[0075] Among them, X CONV is the output of the Conv layer, Conv2D(X, k, k, f') represents the convolution operation, (k, k) is the convolution kernel size of the Conv layer, f is the number of output channels, and ReLU represents the LeakyReLu activation function:

[0076] LeakyReLu(X)=max(0,X)+leaky×min(0,X)

[0077] Among them, X is the input data and leaky is a constant.

[0078] The expression of the maximum pooling layer (Max Pooling) in the step 3.1 Stem feature extraction layer is:

[0079] X MaxPool (i,j,c)=max{X(m,n,c)|m=i·s+p, n=j·s+q, 0≤p, q<k}

[0080] Among them, i, j represent the position index of the output feature map, c represents the channel index, (m, n) represents the position of the pooling window on the input feature map, s, p, q all represent real numbers; max{·} represents the maximum value in the pooling window, and the result of the maximum pooling operation is to output a feature map with a reduced spatial size and the same number of channels. The maximum pooling layer in other network modules can also use the above expression.

[0081] As a preferred technical solution, in this embodiment, the Transition layer in step 3.2 is composed of a normalization layer BN, a RELU activation function layer (Rectified Linear Unit), a Conv layer, and an average pooling layer;

[0082] In step 3.2, for the input feature map X (shape is H×W×C), the normalization BN process is as follows:

[0083] X BN =BatchNorm(X)

[0084] Among them, Batch(·) represents the normalization operation, X BN is the normalized feature map;

[0085] The normalized feature map will be transformed nonlinearly through the ReLU activation function:

[0086] X ReLU =ReLU(X BN )

[0087] Among them, ReLU(·) represents the ReLU activation function, X ReLU is the feature map after the activation function;

[0088] The activated feature map passes through a 1×1 Conv layer, and the number of output channels is controlled by the reduction coefficient:

[0089] X CONV =Conv2D(X ReLU , 1, 1, C')

[0090] Among them, Conv2D(X, k, k, C') represents the convolution operation, (k, k) is the convolution kernel size, C' is the number of output channels, and the calculation formula is:

[0091] C'=int(C×reduction)

[0092] Where, C is the number of input channels;

[0093] The convolutional feature map passes through an average pooling layer with a stride of 2 and a convolution kernel size of 2×2:

[0094] X pool =AvgPool2D(X conv , 2, 2)

[0095] Among them, AvgPool2D(X, k, s) represents the pooling operation, k is the pooling window size, and s is the step size;

[0096] Output feature map X out The dimensions are (H'×W'×C'), expressed as:

[0097]

[0098] Among them, H' and W' are the spatial dimensions, and C' is the number of output channels;

[0099] The role of the Transition layer is to reduce the number of channels and spatial size of the feature map, thereby reducing the computational complexity of subsequent layers while maintaining the ability to extract important features.

[0100] As a preferred technical solution, in this embodiment, Figure 6 As shown, the Conv-SE layer is composed of a Conv layer, a SE attention layer, a normalization layer BN, a maximum pooling layer, a Conv layer, and a normalization layer BN which are connected in sequence.

[0101] As a preferred technical solution, in this embodiment, Figure 7 As shown, the SE attention layer includes a first branch and a second branch, and the second branch includes a global pooling layer, a second fully connected layer, a RELU activation function, a third fully connected layer, and a Sigmod function connected in sequence; the specific processing process is as follows:

[0102] The input is processed in two ways; the first branch keeps the original structure unchanged; the second branch is input into the global pooling layer for pooling operation, and then the pooling result is input into the second fully connected layer for full connection to obtain the second fully connected result; the second fully connected result is activated by the RELU activation function and then input into the third fully connected layer for full connection to obtain the third fully connected result; the third fully connected result is then input into the Sigmod function for operation to obtain the first branch feature map of the same size as the original input, and its content is the specific weight of each channel; the obtained first branch feature map and the first branch input are scaled to obtain a compressed and excited feature map;

[0103] Among them, the Sigmoid function is one of the commonly used activation functions in machine learning and deep learning, especially in binary classification problems; it maps the input to the interval (0, 1), so it is often used in the output layer to predict probability values, and is defined as follows:

[0104]

[0105] Where x is the input value, which is an arbitrary real number, and e is the base of the natural logarithm.

[0106] As a preferred technical solution, in this embodiment, the regularization process of the Spatial Dropout layer in step 4.2 is as follows:

[0107] The probability of discarding the feature map is set to p, and the probability of retaining it is 1-p. For each channel c (c = 1, 2, ..., C), the retained channel is randomly selected; it is represented as generating a binary mask:

[0108]

[0109] Among them, M C is the value of the mask at the Cth channel.

[0110] The flattening operation performed by the Flatten layer in step 4.3 is to flatten the seventh feature map of 6×6×256 into a one-dimensional vector with a length of H·W·C; let the seventh feature map of 6×6×256 be X, and its flattened output is X Flat ,but:

[0111] X Flat =Flatten(X)

[0112] The seventh feature map of 6×6×256 is input into the Flatten layer for flattening, and then the feature map of the Flatten layer output size of 9216 is input into the fully connected layer (FC, Fully Connected) of the DPSE-WDDN network, and then the ninth feature map of size 512 obtained by the fully connected layer is input into the SoftMax layer for classification to obtain the detection result.

[0113] The output of the fully connected layer in step 4.3 is expressed as:

[0114] y=Wx+b

[0115] Among them, x is the input vector (usually the flattened feature map), W is the weight matrix, b is the bias vector, and y is the output vector.

[0116] The SoftMax layer in step 4.3 is an activation function layer commonly used in classification problems. It converts the output of the fully connected layer into a probability distribution. The output value of the SoftMax layer ranges from 0 to 1, and the sum of all outputs is 1, indicating the probability of each class. The SoftMax layer formula is expressed as:

[0117]

[0118] Among them, σ(z) j is the output probability of the jth class, z j is the jth output of the fully connected layer, m is the number of output categories, and the SoftMax function ensures that the output value is a probability distribution, so that the output of the DPSE-WDDN network can be interpreted as the probability of each category, which is convenient for classification decision.

[0119] Furthermore, the method of the present invention is evaluated by the following steps to obtain the data in Tables 1 and 2; Table 1 is a comparison of the accuracy, precision, recall and F1-score of different methods on the wafer defect dataset WM-811K. Table 2 is a comparison of the FLOPS and parameter quantity of different methods on the wafer defect dataset WM-811K. According to the results in Table 2, it can be concluded that the evaluation effect of the DPSE-WDDN network of the present invention is better.

[0120] The evaluation indicators are accuracy, precision, recall and F1-score, and their expressions are:

[0121]

[0122] Among them, TP is the number of samples predicted as positive that are actually positive; FP is the number of samples predicted as positive that are actually negative; TN is the number of samples predicted as negative that are actually negative; FN is the number of samples predicted as negative that are actually positive. The F1 score is a weighted average of precision and recall, and the larger the value, the better the model.

[0123] Table 1 Comparison of accuracy, precision, recall and F1 scores of different methods on the wafer defect dataset WM-811K:

[0124]

[0125]

[0126] Table 2 Comparison of FLOPS and parameters of different methods on wafer defect dataset WM-811K:

[0127] Network Model Total params(M) FLOPS(G) Accuracy (%) DenseNet 6.963 2.865 80.8 VGG16 134297 18.48 80.1 SVM - - 70.4 MobileNetV2 3.4 0.313 90.8 WM-PeleeNet 0.169 0.316 93.6 DPSE-WDDN 3.173 0.775 96.8

[0128] This paper proposes a defect recognition and classification method based on DPSE-WDDN (Dual Path Attention-Wafer Defect Detection Network), which effectively extracts wafer image features by combining the advantages of SE attention layer, PeleeNet classification network and convolutional neural network (CNN), thereby improving the wafer defect detection capability and efficiency.

[0129] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. The wafer defect detection method based on PeleeNet and CNN is characterized by: The following steps are involved: Step 1: Image preprocessing: adjust the input wafer image to be inspected into a three-channel RGB image, and perform median filtering on the wafer image to be inspected; Step 2: construct a DPSE-WDDN network, and use the DPSE-WDDN network to perform wafer defect detection on the wafer to be detected image; the DPSE-WDDN network includes a PeleeNet branch, a Conv-SE branch, and a first fully connected layer, an SE attention layer, a Spatial-Dropout layer, a maximum pooling layer, a Flatten layer, a Dense feature extraction layer, and a SoftMax layer connected in sequence; the output ends of the PeleeNet branch and the Conv-SE branch are connected to the input end of the first fully connected layer; The PeleeNet branch includes a Stem feature extraction layer, a Dense feature extraction layer, a Transition layer, and an average pooling layer connected in sequence; the Conv-SE branch includes four Conv-SE layers connected in sequence; Step 3: Use the PeleeNet branch and the Conv-SE branch to extract features of the wafer image to be inspected, and obtain the fourth feature map and the fifth feature map respectively, wherein the PeleeNet branch includes: Step 3.1: Input the pre-processed wafer image to be inspected into the Stem feature extraction layer in the Peleenet network to obtain the first feature map; Step 3.2: Input the first feature map into the Dense feature extraction layer for feature extraction to obtain the second feature map; the second feature map is input into the Transition layer in the Peleenet network for transition processing to obtain the third feature map; The Dense feature extraction layer in step 3.2 is a three-branch feature extraction module, including a Conv layer, a maximum pooling layer, and a Concatenate layer; Step 3.3: Perform average pooling on the third feature map that passes through the Transition layer in the Peleenet network, adjust the size of the feature map, and output the fourth feature map; The Conv-SE layer is composed of a first 3x3Conv layer, a RELU activation function layer, a normalization layer BN, a maximum pooling layer, a second 3x3Conv layer, a normalization layer BN, and a SE attention layer connected in sequence; Input the preprocessed wafer image to be inspected into four sequentially connected Conv-SE layers for feature extraction to obtain a fifth feature map; Step 4: Dual-path feature fusion: Step 4.1: The fourth feature map obtained by the output of the PeleeNet branch and the fifth feature map obtained by the output of the Conv-SE branch are fused through the first fully connected layer to obtain the sixth feature map; Step 4.2: Input the sixth feature map into the Spatial-Dropout layer for regularization, and then input the output of the Spatial-Dropout layer into the maximum pooling layer of the DPSE-WDDN network to obtain the seventh feature map; Step 4.3: Input the seventh feature map into the Flatten layer to flatten the obtained eighth feature map, input the eighth feature map into the Dense feature extraction layer of the DPSE-WDDN network to obtain the ninth feature map, input the ninth feature map into the SoftMax layer of the DPSE-WDDN network for classification, and obtain the wafer defect detection result.

2. The method according to claim 1, characterized in that Step 1 also includes: The data of the unbalanced categories is enhanced for the wafer inspection images. The data enhancement process includes vertically or horizontally flipping the images, vertically or horizontally moving the images, and rotating or scaling the images.

3. The method according to claim 1, characterized in that The Stem feature extraction layer in step 3.1 is composed of a Conv layer, a maximum pooling layer and a Concatenate layer.

4. The method according to claim 3, characterized in that The Transition layer in step 3.2 is composed of a normalization layer BN, a RELU activation function layer, a Conv layer, and an average pooling layer.

5. The method according to claim 3, characterized in that: The Conv-SE layer is composed of a Conv layer, a SE attention layer, a normalization layer BN, a maximum pooling layer, a Conv layer, and a normalization layer BN connected in sequence.

6. The method according to claim 1, characterized in that The SE attention layer includes a first branch and a second branch, and the second branch includes a global pooling layer, a second fully connected layer, a RELU activation function, a third fully connected layer, and a Sigmod function connected in sequence.

7. The method according to claim 1, characterized in that The regularization process of the Spatial Dropout layer in step 4.2 is as follows: The probability of discarding the feature map is set to p, and the probability of retaining it is 1-p. For each channel c (c = 1, 2, ..., C), the retained channel is randomly selected; it is represented as generating a binary mask: Among them, M C is the value of the mask at the Cth channel.

Images