Wafer graph defect mode detection method based on FPRN-CBAM and related device

Abstract

The invention discloses a wafer graph defect mode detection method based on FPRN-CBAM and a related device. The method comprises the following steps: acquiring a wafer graph to be identified; inputting the wafer graph to be identified into a pre-trained defect mode detection model, and outputting a detection result; wherein the defect mode detection model is obtained by training a feature pyramid residual network based on a convolution block attention module by using training data, and the training data comprises wafer graphs of different defect modes and corresponding defect mode labels; according to the feature pyramid residual network based on the convolution block attention module, an attention mechanism module is integrated in each convolution layer of a ResNet18 backbone network, meanwhile, a feature pyramid module is embedded in the ResNet18 backbone network, and the attention mechanism module is used for adaptively learning the importance weight of each channel and space. And the feature pyramid module is used for performing multi-scale feature extraction on the wafer graph. The purpose of the invention is to accurately identify the type of the mixed defect mode of the wafer graph through multi-scale feature extraction and adaptive learning importance weight.

Description

Technical Field

[0001] The present invention belongs to the field of semiconductor manufacturing and detection, and specifically relates to a wafer image defect mode detection method based on FPRN-CBAM and a related device. Background Art

[0002] Semiconductors, as the core of electronic devices, are widely used in various fields. Wafers made mainly of silicon materials are important semiconductor substrates and are widely used in the manufacture of integrated circuit chips and other semiconductor devices. Environmental pollution, abnormal processes, substandard material quality, or the presence of impurities during the manufacturing process can all lead to wafer defects. Circuit probe testing can convert a wafer into a fixed-size wafer map composed of die test results. Due to the complexity of the semiconductor manufacturing process, wafer maps often produce random and different defect characteristics. These defects not only affect the performance of the wafer, but may also reduce the reliability and service life of the entire semiconductor device.

[0003] Deep learning can automatically learn from raw data and extract effective feature representations, and has become the main method for wafer defect detection. Among them, the deformable convolution unit designed by the deformable convolutional network (DC-Net) can selectively sample from mixed defects, and then extract high-quality features from the wafer map. The multi-label output layer is improved by a single-click encoding mechanism to decompose the extracted mixed features into each basic single defect. However, this method does not take into account the characteristics of the size variation of wafer map defects at different scales, so the detection results of mixed wafer defects fluctuate greatly. Summary of the invention

[0004] In view of the problems existing in the prior art, the present invention provides a wafer image defect pattern detection method and related devices based on FPRN-CBAM, which aims to accurately identify the mixed defect pattern categories of wafer images through multi-scale feature extraction and adaptive learning of importance weights.

[0005] In order to solve the above technical problems, the present invention is implemented by the following technical solutions: According to a first aspect of the present invention, a wafer image defect mode detection method based on FPRN-CBAM is provided, comprising: Obtaining a wafer image to be identified; Inputting the wafer image to be identified into a pre-trained defect mode detection model, and outputting a detection result; Among them, the defect mode detection model is obtained by training a feature pyramid residual network based on a convolutional block attention module using training data, and the training data includes wafer images of different defect modes and their corresponding defect mode labels; the feature pyramid residual network based on the convolutional block attention module is: integrating an attention mechanism module in each convolution layer of the ResNet18 backbone network, and embedding the feature pyramid module in the ResNet18 backbone network, the attention mechanism module is used to adaptively learn the importance weights of each channel and space, and the feature pyramid module is used to perform multi-scale feature extraction on the wafer image.

[0006] In a possible implementation manner of the first aspect, the attention mechanism module includes a channel attention module and a spatial attention module; The ResNet18 backbone network includes a first convolutional layer, a second convolutional layer, a third convolutional layer, a fourth convolutional layer and a fifth convolutional layer; The attention mechanism module is integrated into each convolutional layer of the ResNet18 backbone network, specifically: The channel attention module and the spatial attention module are sequentially inserted after the residual block of each convolutional layer.

[0007] In a possible implementation manner of the first aspect, the feature pyramid module is used to extract multi-scale features from the wafer image, specifically: The fifth convolutional layer first undergoes 1×1 convolution to obtain a fifth feature, and the fifth feature is up-sampled and added to the feature obtained by the fourth convolutional layer through 1×1 convolution to obtain a fourth feature; The fourth feature is up-sampled and added with the feature obtained by the third convolution layer through 1×1 convolution to obtain the third feature; The third feature is up-sampled and added with the feature obtained by the second convolution layer through 1×1 convolution to obtain the second feature; The second feature is used as the input of the classifier in the ResNet18 backbone network.

[0008] In a possible implementation manner of the first aspect, a convolution kernel size of the first convolution layer is 3×3, which is used to extract small-size defect features.

[0009] According to a first aspect of the present invention, a wafer image defect mode detection device based on FPRN-CBAM is provided, characterized in that it includes: Obtaining a wafer image to be identified; Inputting the wafer image to be identified into a pre-trained defect mode detection model, and outputting a detection result; Among them, the defect mode detection model is obtained by training a feature pyramid residual network based on a convolutional block attention module using training data, and the training data includes wafer images of different defect modes and their corresponding defect mode labels; the feature pyramid residual network based on the convolutional block attention module is: integrating an attention mechanism module in each convolution layer of the ResNet18 backbone network, and embedding the feature pyramid module in the ResNet18 backbone network, the attention mechanism module is used to adaptively learn the importance weights of each channel and space, and the feature pyramid module is used to perform multi-scale feature extraction on the wafer image.

[0010] In a possible implementation manner of the first aspect, the attention mechanism module includes a channel attention module and a spatial attention module; The ResNet18 backbone network includes a first convolutional layer, a second convolutional layer, a third convolutional layer, a fourth convolutional layer and a fifth convolutional layer; The attention mechanism module is integrated into each convolutional layer of the ResNet18 backbone network, specifically: The channel attention module and the spatial attention module are sequentially inserted after the residual block of each convolutional layer.

[0011] In a possible implementation manner of the first aspect, the feature pyramid module is used to extract multi-scale features from the wafer image, specifically: The fifth convolutional layer first undergoes 1×1 convolution to obtain a fifth feature, and the fifth feature is up-sampled and added to the feature obtained by the fourth convolutional layer through 1×1 convolution to obtain a fourth feature; The fourth feature is up-sampled and added with the feature obtained by the third convolution layer through 1×1 convolution to obtain the third feature; The third feature is up-sampled and added with the feature obtained by the second convolution layer through 1×1 convolution to obtain the second feature; The second feature is used as the input of the classifier in the ResNet18 backbone network.

[0012] In a possible implementation manner of the first aspect, a convolution kernel size of the first convolution layer is 3×3, which is used to extract small-size defect features.

[0013] According to a first aspect of the present invention, there is provided a device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of a wafer image defect mode detection method based on FPRN-CBAM when executing the computer program.

[0014] According to a first aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the wafer image defect mode detection method based on FPRN-CBAM are implemented.

[0015] Compared with the prior art, the present invention has at least the following beneficial effects: The present invention provides a wafer image defect pattern detection method based on FPRN-CBAM, which realizes efficient and accurate detection of different defect patterns in wafer images by adopting a feature pyramid residual network architecture based on a convolutional block attention module. Specifically, the feature pyramid residual network based on the convolutional block attention module cleverly integrates the feature pyramid structure with the ResNet18 backbone network. By fusing feature maps at different levels, the network can simultaneously capture the detail information and semantic information in the wafer image. This feature is crucial for detecting defect patterns of different scales in the wafer image, and can ensure that the network can still maintain high detection accuracy when processing complex and changeable wafer images. In the feature pyramid residual network based on the convolutional block attention module, an attention mechanism module is integrated, and the attention mechanism module can adaptively learn the importance weights of each channel and spatial position, that is, when the network extracts features, it can automatically focus on those channels and spatial information that are important for defect detection, thereby further improving the accuracy of wafer image defect detection.

[0016] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the specific implementation modes of the present invention, the drawings required for use in the description of the specific implementation modes will be briefly introduced below. Obviously, the drawings described below are some implementation modes of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 It is a flow chart of a wafer image defect mode detection method based on FPRN-CBAM of the present invention; Figure 2 It is FPN target detection mode; Figure 3 The schematic diagram of the CBAM submodules, where the upper figure is the channel attention module and the lower figure is the spatial attention module; Figure 4 The specific structure of FPRN-CBAM proposed in the present invention; Figure 5It is the Resnet18 model structure; Figure 6 An overview of the MixedWM38 wafer map dataset; Figure 7 is the confusion matrix for the binary classification task; Figure 8 The comparative experimental results of different models are shown in Figure 2. Fig. 9 Comparison of different model indicators under different mixture categories. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0020] like Figure 1 As shown, an embodiment of the present invention provides a wafer image defect mode detection method based on FPRN-CBAM, which is optimized according to the characteristics of wafer image defects with different scale size changes, and specifically includes the following steps: Step 1: Obtain a wafer image to be identified.

[0021] Specifically, wafer images to be identified are acquired by scanning or photographing, and these wafer images may contain defects of various types, sizes, and locations. The acquired wafer images will be used as input data for subsequent defect pattern detection.

[0022] Step 2: Input the wafer image to be identified into a pre-trained defect pattern detection model and output the detection result.

[0023] That is to say, the wafer image to be identified is input into the defect mode detection model, which will automatically extract the multi-scale features of the wafer image, perform weighted processing on the key information through the attention mechanism module, and finally output the detection results.

[0024] The defect mode detection model described in the present invention is obtained by training a feature pyramid residual network (FPRN) based on a convolutional block attention module (CBAM) using training data. The feature pyramid residual network (FPRN) based on a convolutional block attention module (CBAM) is named FPRN-CBAM.

[0025] The training data includes wafer images of different defect modes and their corresponding defect mode labels. Specifically, a large number of wafer images containing different defect modes are collected as training data. These wafer images should cover various possible defect types, sizes and locations. Each wafer image is annotated to generate corresponding defect mode labels. These labels will be used to guide the learning direction of the model during the training process.

[0026] Combination Figures 2 to 5 As shown, the feature pyramid residual network based on the convolutional block attention module is: an attention mechanism module is integrated in each convolutional layer of the ResNet18 backbone network, and the feature pyramid module is embedded in the ResNet18 backbone network. The attention mechanism module is used to adaptively learn the importance weights of each channel and space, and the feature pyramid module is used to perform multi-scale feature extraction on the wafer image.

[0027] That is to say, the present invention adopts FPRN-CBAM as the core architecture of the defect mode detection model, and integrates the attention mechanism module (Convolutional Block Attention Module, CBAM) in each convolutional layer of the ResNet18 backbone network. The attention mechanism module can adaptively learn the importance weights of each channel and space, thereby enhancing the model's ability to capture key information. At the same time, the Feature Pyramid Network (FPN) is embedded in the ResNet18 backbone network to achieve multi-scale feature extraction of wafer images. Specifically, FPN is a feature pyramid with strong semantics at all scales. Through multi-scale feature fusion, the strong semantic information of high-level features is combined with the high-resolution information of low-level features, which enhances the feature representation capability, especially for the identification and positioning of tiny wafer defects. In addition, FPN avoids the problem of ultra-high computational complexity and improves detection efficiency.

[0028] When training the defect mode detection model, the training data and the corresponding defect mode labels are input into the constructed FPRN-CBAM for training. During the training process, the model will learn how to extract effective features based on the input wafer image and accurately determine the defect mode to which it belongs.

[0029] By introducing an attention mechanism module and a feature pyramid module, the method of the present invention can more effectively extract multi-scale features of wafer images and adaptively learn key information, thereby improving the accuracy of defect recognition.

[0030] In one possible implementation, Figure 3 As shown, the attention mechanism module includes a channel attention module (Channel Attention Module) and a spatial attention module (Spatial Attention Module).

[0031] The ResNet18 backbone network includes a first convolutional layer, a second convolutional layer, a third convolutional layer, a fourth convolutional layer and a fifth convolutional layer, and each convolutional layer includes a residual block for extracting feature information of the wafer image.

[0032] The attention mechanism module is integrated into each convolutional layer of the ResNet18 backbone network, combined with Figure 4 As shown in the lower right corner, specifically: the channel attention module and the spatial attention module are inserted in sequence after the residual block of each convolutional layer, that is, the CBAM module is inserted after the second convolutional layer and the batch normalization layer. The CBAM module receives the output feature map of the residual block as input and applies channel attention and spatial attention to enhance the features. Finally, the modified residual block is integrated into the overall structure of ResNet18 with feature pyramid to form the proposed FPRN-CBAM model.

[0033] In detail, the channel attention module calculates the importance weight of each channel and performs channel-level weighted processing on the feature map, thereby enhancing the model's ability to capture key channel information. Based on the channel attention module, the spatial attention module further calculates the importance weight of each spatial position and performs spatial-level weighted processing on the feature map, thereby enhancing the model's ability to capture key spatial information. Through this integration method, the model can adaptively focus on key information in the wafer image during feature extraction and improve the accuracy of defect recognition.

[0034] In one possible implementation, the feature pyramid module is used to extract multi-scale features from the wafer image, specifically: The fifth convolutional layer first undergoes 1×1 convolution to obtain the fifth feature, and the fifth feature is upsampled and added to the feature obtained by the fourth convolutional layer through 1×1 convolution to obtain the fourth feature; the fourth feature is upsampled and added to the feature obtained by the third convolutional layer through 1×1 convolution to obtain the third feature; the third feature is upsampled and added to the feature obtained by the second convolutional layer through 1×1 convolution to obtain the second feature; the second feature is used as the input of the classifier in the ResNet18 backbone network.

[0035] Specifically, in order to realize multi-scale feature extraction of wafer images, the present invention embeds the feature pyramid module into the ResNet18 backbone network. Figure 4 As shown in the figure, the specific implementation method is as follows: the fifth convolution layer C5 is first subjected to a 1×1 convolution kernel for dimensionality reduction processing to obtain the fifth feature (denoted as M5). This feature map has high semantic information but low spatial resolution. M5 is upsampled to match its spatial resolution with the output feature map of the fourth convolution layer C4. Then, the upsampled M5 is added element by element to the feature (denoted as C4') obtained by the 1×1 convolution of the fourth convolution layer C4 to obtain the fused fourth feature (denoted as M4). This step realizes the combination of high-level semantic information and middle-level features. M4 is upsampled to match its spatial resolution with the output feature map of the third convolution layer C3. Then, the upsampled M4 is added element by element to the feature (denoted as C3') obtained by the 1×1 convolution of the third convolution layer C3 to obtain the fused third feature (denoted as M3). This step further integrates the middle-level and low-level features. M3 is upsampled to match its spatial resolution with the output feature map of the second convolution layer C2. Then, the upsampled M3 is added element by element with the features obtained by 1×1 convolution of the second convolutional layer C2 (denoted as C2') to obtain the fused second feature (denoted as M2). This step realizes the comprehensive fusion of low-level detail information and high-level semantic information. The fused second feature M2 is used as the input of the classifier in the ResNet18 backbone network. In other words, the M2 with the most feature information is convolved with 3×3 to obtain the P2 layer feature. Finally, the P2 layer converts the previously learned feature map into a fixed-size representation through the adaptive average pooling layer, and completes the image classification task of the specified category through the fully connected layer.

[0036] It should be understood that the classifier is typically composed of one or more fully connected layers for further processing and classification of the extracted multi-scale features to identify mixed defects in wafer images.

[0037] Preferably, the convolution kernel size of the first convolution layer is 3×3, that is, a convolution kernel of 3×3 size is set in the first convolution layer, and the 3×3 convolution kernel can effectively extract small-size defect features in the wafer image while maintaining low computational complexity. Since some defects in the wafer image may be very small, using a smaller convolution kernel can better capture these subtle features, thereby improving the accuracy of detection.

[0038] Compared with many methods, the present invention shows better performance in wafer image classification tasks, with higher average recognition accuracy, providing strong support for the reliability of the wafer production process. Next, the technical solution described in the present invention is described in full and detail using the MixedWM38 public wafer dataset as an example, and a comparative experiment is designed to verify the feasibility and superiority of the proposed FPRN-CBAM model. Figure 6 shown.

[0039] Step 1: Collect a large amount of wafer image data in a wafer manufacturing plant. The wafer images of the MixedWM38 dataset are obtained by testing the electrical performance of each chip on the wafer with a test probe. They are expressed in binary, with 0 indicating no wafer, 1 indicating the test passed, and 2 indicating the test failed. The number of various types of silicon wafer images actually collected varies greatly. In order to maintain a balance between various types of data, some wafer images are generated using a generative adversarial network to maintain a balance between the number of samples between types. Finally, 38,015 wafer images with a data shape of (52, 52) were generated.

[0040] Step 2: Divide the defect modes of the wafer images in the data set. The MixedWM38 data set contains 38 defect modes, including 1 normal mode C1 [Normal], 8 single defect modes C2-C9 [Center (C), Donut (D), Edge_Loc (EL), Edge_Ring (ER), Loc (L), Near_Full (NF), Scratch (S), Random (R)] and 29 mixed defect modes C10-C38. The 29 mixed defect modes include 13 dual mixed modes C10-C22 [C+EL, C+ER, C+L , C+S, D+EL, D+ER, D+L, D+S, EL+L, EL+S, ER+L, ER+S, L+S], 12 types of three-mixed modes C23-C34 [C+EL+L, C+EL+S, C+ER+L, C+ER+S, C+L+S, D+EL+L, D+EL+S, D+ER+L, D+ER+S, D+L+S, EL+L+S, ER+L+S] and 4 types of four-mixed modes C35-C38 [C+L+EL+S, C+L+ER+S, D+L+EL+S, D+L+ER+S]. Based on the above classification, the number of wafer images for each defect mode is shown in Table 1. The [NF] class has 866 wafer images, the [R] class has 149 wafer images, the [C+EL+S] class has 2000 wafer images, and the remaining 35 categories have 1000 wafer images each. Together, they constitute the 38015 wafer images of the MixedWM38 dataset.

[0041] Table 1 Number of wafer images for each defect mode in the MixedWM38 dataset

[0042] Step 3: Introduce four evaluation indicators, namely Precision, Recall, F1-Score and Accuracy. Figure 7 The confusion matrix of the binary classification task is shown. For the multi-classification task, the precision, recall and F1 of each class are calculated and their average is used as the final indicator. The accuracy is calculated by the ratio of the number of correctly classified samples to the total data. The evaluation indicator is calculated as follows:

[0043]

[0044]

[0045]

[0046] Where n represents the number of categories. The higher the precision, recall, and accuracy, or the closer the F1-Score value is to 1, the better the model performance.

[0047] Step 4: The MixedWM38 dataset is randomly divided into training set and test set in a ratio of 8:2. The loss function is the binary cross entropy loss function. The learning rate is set to 0.01 and the weight decay is set to 0.001. Construct the FPRN-CBAM model and use the training data to train the constructed FPRN-CBAM model. The detailed structure of the constructed FPRN-CBAM model is as follows: Figure 4 As shown, the parameter settings of the entire network are shown in Table 2, where k represents the convolution kernel size, s represents the step size, and p represents padding.

[0048] Table 2 Network structure settings of FPRN-CBAM model

[0049] Step 4.1: Integrate the feature pyramid network (FPN) into the Resnet structure. FPN is a feature pyramid with strong semantics at all scales. Through multi-scale feature fusion, the strong semantic information of high-level features is combined with the high-resolution information of low-level features, which enhances the feature representation capability, especially for the identification and location of tiny wafer defects. In addition, FPN avoids the problem of ultra-high computational complexity and improves detection efficiency. Figure 2 As shown in the figure, FPN consists of three parts: a bottom-up path, a top-down path, and a lateral connection.

[0050] The bottom-up path is the forward propagation process of CNN. In CNN, the input image is processed by multiple convolutional layers and pooling layers, and the size of the feature map gradually decreases. The feature map of the low layer is large in size and contains more small target information, while the feature map of the high layer is small in size but contains less small target information. FPN selects the output of the last layer of each stage as the reference set of feature mapping. The top-down path gradually enlarges the high-level feature map, and the upsampled feature map is fused with the low-level feature map of the corresponding scale through lateral connections. Figure 2 As shown in the dotted box, the low-level feature map is usually first subjected to a 1×1 convolution to adjust the number of channels, and then combined with the upsampled feature map. Figure 2 Multiples perform element-by-element addition or concatenation operations.

[0051] Step 4.2: Integrate the convolutional block attention module (CBAM) in the Resnet convolutional block. CBAM consists of two attention modules: channel and spatial. The details of the two attention modules are described below: like Figure 3 As shown above, firstly, the input feature map Perform global average pooling and global maximum pooling operations respectively to generate two feature maps and . Input them into the same shared network to get and , the shared network structure is a multi-layer perceptron (MLP), where the number of neurons in the hidden layer of MLP is ( r is the reduction ratio), and the activation function is ReLU. Finally, and Sum and merge, and then output the final feature vector through the Sigmoid activation function The channel attention module is calculated as follows:

[0052] in, is the sigmoid function, To calculate the MLP weights.

[0053] The spatial attention module and the channel attention module complement each other, such as Figure 3 As shown below, the output feature map of the channel attention module As the input of this module, first perform global maximum pooling and global average pooling operations. Then concatenate the two resulting feature maps. Next, they are reduced in dimension through a 7×7 convolution operation to highlight the effective information. Finally, pass the Sigmoid function and output the spatial attention feature vector The spatial attention module is calculated as:

[0054] Step 4.3: Integrate the first two steps to design a complete FPRN-CBAM structure, such as Figure 4 The core of ResNet is to add a residual structure to solve the degradation phenomenon caused by the deepening of the neural network layer, and retain the original structure of ResNet18 as the backbone, as shown in Figure 5As shown, the convolutional layers of the ResNet18 model are assumed to be C1-C5. The present invention first designs a feature pyramid network with the ResNet18 structure as the backbone. Specifically, the C5 layer first undergoes a 1×1 convolution to obtain the M5 feature, and M5 is upsampled and then added with the C4' feature obtained by the C4 layer through a 1×1 convolution to obtain the M4 feature. Similarly, the M3 and M2 features can be obtained through lateral connection. The M2 layer containing the most feature information is subjected to a 3×3 convolution to obtain the P2 layer feature. Finally, the P2 layer converts the previously learned feature map into a fixed-size representation through an adaptive average pooling layer, and completes the image classification task of the specified category through a fully connected layer.

[0055] Next, the channel attention and spatial attention modules of CBAM are integrated into the residual unit of ResNet18 to recalibrate the feature map, so that ResNet18 can capture key features more accurately when processing wafer image defects. Specifically, the CBAM module is inserted at the end of the residual block of each convolutional layer of ResNet18, that is, after the second convolutional layer and batch normalization layer, as shown in Figure 4 As shown in the lower right corner, the CBAM module receives the output feature map of the residual block as input and applies channel attention and spatial attention to enhance the features. Finally, the modified residual block is integrated into the overall structure of ResNet18 with feature pyramid to form the proposed FPRN-CBAM model.

[0056] The proposed FPRN-CBAM model not only integrates the fine texture information in the shallow features of the wafer image, but also makes full use of the rich semantic content contained in the deep features, thereby enhancing the detection accuracy of the model across defects of different sizes in wafer image classification.

[0057] Step 5: Input the test data into the trained FPRN-CBAM model to obtain the diagnosis results.

[0058] Step 5.1: In order to demonstrate the superiority of the proposed method, it is compared with various benchmark methods. The present invention uses three other SOTA methods for testing, and the three comparison methods are Deformable Convolutional Networks (DCNet), Composite wafer defect recognition framework (CWDRNet) and AttentionSpatial Pyramid Pooling Network (ESPPNet). DCNet uses deformable convolutional units to selectively sample mixed defects and can accurately detect defect areas; CWDRNet selectively extracts defect information and applies a classifier based on the attention mechanism to enhance feature expression; ESPPNet incorporates attention spatial pyramid pooling technology and has good performance in mixed type defect recognition. To ensure the fairness of the experimental comparison, all methods set the same hyperparameters. The MixedWM38 dataset is randomly divided into training and test sets in a ratio of 8:2. The loss function is a binary cross entropy loss function, the learning rate is set to 0.01, and the weight decay is set to 0.001. In order to avoid errors caused by factors such as initialization weights and data randomness, each method is run 10 times and the average value is taken as the final result, and each run is 300 rounds. Figure 8 and Table 3 show the specific results.

[0059] The results show that: (1) The FPRN-CBAM model shows excellent performance in the wafer defect pattern recognition task, with an accuracy rate of up to 98.71%. This figure means that among all the test samples, the model correctly identifies the defective and non-defective states in a near-perfect ratio, with only a very small number of samples being misjudged, which greatly improves the quality control efficiency of the production line; (2) The model’s accuracy reached 98.65%. This high-level indicator proves that among the defect samples identified by FPRN-CBAM, the proportion of real defects is extremely high, and there is almost no case of misjudging normal wafers as defects, which is of great significance for reducing unnecessary subsequent processing costs; (3) In terms of recall rate, the FPRN-CBAM model also performed well, reaching 98.58%. This result directly reflects the model's ability to capture actual defects, ensuring that the vast majority of defects on the production line can be discovered in a timely and accurate manner, effectively reducing the risk of defective products being discharged due to missed inspections, and maintaining the overall quality of the wafers. (4) The F1-Score of the FPRN-CBAM model reached 98.60%. This indicator combines precision and recall, which not only demonstrates the model's superb skills in balancing the two, but also reflects the model's high stability and adaptability in complex and changing actual production environments. The high value of F1-Score means that FPRN-CBAM can maintain stable recognition performance when facing different types and forms of wafer defects, which is of immeasurable value for improving the level of intelligence and automation in the semiconductor manufacturing industry; (5) In summary, the four evaluation indicators of FPRN-CBAM are higher than those of the other three models, and the accuracy, precision and F1-Score of FPRN-CBAM are more than 0.15% higher than the highest values ​​of other methods. All of the above show the effectiveness and superiority of the proposed FPRN-CBAM model in wafer defect pattern recognition.

[0060] Table 3 Classification results of different network models on MixedWM38 dataset

[0061] Step 5.2: To continue studying the recognition performance of the FPRN-CBAM model for mixed defects, the average precision, recall rate, and F1-Score of the four methods for single defects C1-C9, two mixed defects C10-C22, three mixed defects C23-C34, and four mixed defects C35-C38 are compared. The results are shown in Fig. 9 As shown in Table 4. The results show that FPRN-CBAM has higher precision, recall and F1-Score than other models in the recognition of two-mixed, three-mixed and four-mixed defect patterns, which shows that FPRN-CBAM performs well in the recognition of mixed defect patterns and the model has good generalization ability and can be applied to different types of defect pattern recognition tasks. It should be noted that since the number of C7 and C9 in a single defect is smaller than that of other defect patterns, the average recall and F1-Score of the proposed model for a single defect are lowered.

[0062] Table 4 Comparison of different model indicators under different mixing categories

[0063] In yet another embodiment of the present invention, a wafer image defect mode detection device based on FPRN-CBAM is provided, comprising: Obtain a wafer image to be identified.

[0064] The wafer image to be identified is input into a pre-trained defect mode detection model, and the detection result is output; wherein the defect mode detection model is obtained by training a feature pyramid residual network based on a convolutional block attention module using training data, and the training data includes wafer images of different defect modes and their corresponding defect mode labels; the feature pyramid residual network based on the convolutional block attention module is: an attention mechanism module is integrated in each convolution layer of the ResNet18 backbone network, and the feature pyramid module is embedded in the ResNet18 backbone network, the attention mechanism module is used to adaptively learn the importance weights of each channel and space, and the feature pyramid module is used to perform multi-scale feature extraction on the wafer image.

[0065] All relevant contents of each step involved in the embodiment of the aforementioned wafer image defect mode detection method based on FPRN-CBAM can be referred to the functional description of the functional module corresponding to the wafer image defect mode detection device based on FPRN-CBAM in the embodiment of the present invention, and will not be repeated here. The division of modules in the embodiment of the present invention is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional module in each embodiment of the present invention can be integrated into a processor, or it can exist physically separately, or two or more modules can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of software functional modules.

[0066] In another embodiment of the present invention, a computer device is provided, the computer device including a processor and a memory, the memory is used to store a computer program, the computer program includes program instructions, and the processor is used to execute the program instructions stored in the computer storage medium. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc., which are the computing core and control core of the terminal, which are suitable for implementing one or more instructions, and are specifically suitable for loading and executing one or more instructions in a computer storage medium to implement the corresponding method flow or corresponding function; the processor described in the embodiment of the present invention can be used for the operation of a wafer image defect mode detection method based on FPRN-CBAM.

[0067] In another embodiment of the present invention, the present invention also provides a storage medium, specifically a computer-readable storage medium (Memory), which is a memory device in a computer device for storing programs and data. It can be understood that the computer-readable storage medium here can include both built-in storage media in a computer device and, of course, extended storage media supported by the computer device. The computer-readable storage medium provides a storage space, which stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by a processor are also stored in the storage space, and these instructions can be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium here can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the corresponding steps of the above-mentioned embodiment regarding a wafer image defect mode detection method based on FPRN-CBAM.

[0068] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0069] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0070] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0071] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0072] In the present invention, the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" etc. mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the described specific features, structures, materials or characteristics may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0073] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A wafer image defect mode detection method based on FPRN-CBAM, characterized in that: include: Obtaining a wafer image to be identified; Inputting the wafer image to be identified into a pre-trained defect mode detection model, and outputting a detection result; Among them, the defect mode detection model is obtained by training a feature pyramid residual network based on a convolutional block attention module using training data, and the training data includes wafer images of different defect modes and their corresponding defect mode labels; the feature pyramid residual network based on the convolutional block attention module is: integrating an attention mechanism module in each convolution layer of the ResNet18 backbone network, and embedding the feature pyramid module in the ResNet18 backbone network, the attention mechanism module is used to adaptively learn the importance weights of each channel and space, and the feature pyramid module is used to perform multi-scale feature extraction on the wafer image.

2. The wafer image defect mode detection method based on FPRN-CBAM according to claim 1, characterized in that: The attention mechanism module includes a channel attention module and a spatial attention module; The ResNet18 backbone network includes a first convolutional layer, a second convolutional layer, a third convolutional layer, a fourth convolutional layer and a fifth convolutional layer; The attention mechanism module is integrated into each convolutional layer of the ResNet18 backbone network, specifically: The channel attention module and the spatial attention module are sequentially inserted after the residual block of each convolutional layer.

3. The wafer image defect mode detection method based on FPRN-CBAM according to claim 2, characterized in that: The feature pyramid module is used to extract multi-scale features from the wafer image, specifically: The fifth convolutional layer first undergoes 1×1 convolution to obtain a fifth feature, and the fifth feature is up-sampled and added to the feature obtained by the fourth convolutional layer through 1×1 convolution to obtain a fourth feature; The fourth feature is up-sampled and added with the feature obtained by the third convolution layer through 1×1 convolution to obtain the third feature; The third feature is up-sampled and added with the feature obtained by the second convolution layer through 1×1 convolution to obtain the second feature; The second feature is used as the input of the classifier in the ResNet18 backbone network.

4. The wafer image defect mode detection method based on FPRN-CBAM according to claim 2, characterized in that: The convolution kernel size of the first convolution layer is 3×3, which is used to extract small-size defect features.

5. A wafer image defect mode detection device based on FPRN-CBAM, characterized in that: include: Obtaining a wafer image to be identified; Inputting the wafer image to be identified into a pre-trained defect mode detection model, and outputting a detection result; Among them, the defect mode detection model is obtained by training a feature pyramid residual network based on a convolutional block attention module using training data, and the training data includes wafer images of different defect modes and their corresponding defect mode labels; the feature pyramid residual network based on the convolutional block attention module is: integrating an attention mechanism module in each convolution layer of the ResNet18 backbone network, and embedding the feature pyramid module in the ResNet18 backbone network, the attention mechanism module is used to adaptively learn the importance weights of each channel and space, and the feature pyramid module is used to perform multi-scale feature extraction on the wafer image.

6. The wafer image defect mode detection device based on FPRN-CBAM according to claim 5, characterized in that: The attention mechanism module includes a channel attention module and a spatial attention module; The ResNet18 backbone network includes a first convolutional layer, a second convolutional layer, a third convolutional layer, a fourth convolutional layer and a fifth convolutional layer; The attention mechanism module is integrated into each convolutional layer of the ResNet18 backbone network, specifically: The channel attention module and the spatial attention module are sequentially inserted after the residual block of each convolutional layer.

7. The wafer image defect mode detection device based on FPRN-CBAM according to claim 6, characterized in that: The feature pyramid module is used to extract multi-scale features from the wafer image, specifically: The fifth convolutional layer first undergoes 1×1 convolution to obtain a fifth feature, and the fifth feature is up-sampled and added to the feature obtained by the fourth convolutional layer through 1×1 convolution to obtain a fourth feature; The fourth feature is up-sampled and added with the feature obtained by the third convolution layer through 1×1 convolution to obtain the third feature; The third feature is up-sampled and added with the feature obtained by the second convolution layer through 1×1 convolution to obtain the second feature; The second feature is used as the input of the classifier in the ResNet18 backbone network.

8. The wafer image defect mode detection device based on FPRN-CBAM according to claim 6, characterized in that: The convolution kernel size of the first convolution layer is 3×3, which is used to extract small-size defect features.

9. A device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the wafer image defect mode detection method based on FPRN-CBAM as described in any one of claims 1 to 4 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of a wafer image defect mode detection method based on FPRN-CBAM are implemented as described in any one of claims 1 to 4.

Images