Improved ResNet18-based wafer graph defect pattern recognition method and related device

Abstract

The invention discloses a wafer graph defect pattern recognition method based on improved ResNet18 and a related device. The method comprises the following steps: acquiring a wafer graph to be recognized; inputting the to-be-identified wafer graph into a pre-trained wafer graph defect mode identification model, and outputting an identification result; wherein the wafer graph defect mode recognition model is obtained by training improved ResNet18 by using training data, and the training data comprises wafer graphs of different defect modes and corresponding defect mode labels; the improved ResNet18 comprises a selective convolution kernel module, the selective convolution kernel module is used for replacing a first convolution layer of a ResNet18 backbone network, and the selective convolution kernel module is used for adaptively extracting defect features of different sizes of a wafer graph. The invention aims to enhance the capability of extracting defect features of different sizes and types in a wafer graph, so as to improve the accuracy of wafer defect mode recognition.

Description

Technical Field

[0001] The present invention belongs to the field of semiconductor manufacturing, and specifically relates to a wafer image defect pattern recognition method based on improved ResNet18 and related devices. Background Art

[0002] Semiconductors are an important foundation of the electronic information industry and the "heart" of electronic information products. With the development and application of semiconductors in various fields, it has promoted the development of new energy, artificial intelligence, autonomous driving and other industries, and played an important role in promoting economic development. Semiconductor manufacturing mainly includes chip design, wafer manufacturing, integrated circuit manufacturing, wafer testing, chip packaging and final testing. Wafers are silicon-based substrates for manufacturing integrated circuits. However, during the wafer manufacturing process, defects often occur in wafers due to some factors. When defects accumulate to a certain extent, they will affect the pass rate and process stability of integrated circuits.

[0003] Traditional wafer defect pattern detection methods mainly rely on manual feature extraction, which is then fed into a classifier for training and classification. This method is not only complex to operate, but also highly dependent on the operator's experience and skill level. Errors are prone to occur in the manual feature extraction process, and when processing high-dimensional data, the amount of calculation is huge, resulting in low training efficiency. In addition, traditional methods are prone to overfitting, and the model's generalization ability is insufficient, making it difficult to cope with complex and changing wafer defect patterns.

[0004] In order to overcome the limitations of traditional methods, deep learning technology has made significant progress in the field of wafer defect pattern recognition (WMDPR) in recent years with its powerful feature extraction capabilities. Among them, wafer defect pattern recognition methods based on convolutional neural networks (CNNs) are widely used. CNNs can automatically learn and extract rich features, avoiding the incompleteness and feature loss problems caused by manual feature extraction, and significantly improving the accuracy of the model. Despite this, the use of classic convolutional neural networks still faces challenges in identifying mixed defects in wafer images. Since the features of wafer image defects are often relatively weak, and the distribution of feature information of different types of defects in spatial positions is quite different, it is easy for the model to ignore some weak but important feature information during training. This not only reduces the accuracy of the model, but also limits its performance in practical applications. Summary of the invention

[0005] In view of the problems existing in the prior art, the present invention provides a wafer image defect pattern recognition method and related devices based on improved ResNet18, the purpose of which is to enhance the ability to extract defect features of different sizes and types in wafer images, thereby improving the accuracy of wafer defect pattern recognition.

[0006] In order to solve the above technical problems, the present invention is implemented by the following technical solutions:

[0007] According to a first aspect of the present invention, a wafer image defect pattern recognition method based on improved ResNet18 is provided, comprising:

[0008] Obtaining a wafer image to be identified;

[0009] The wafer image to be identified is input into a pre-trained wafer image defect pattern recognition model, and a recognition result is output; wherein the wafer image defect pattern recognition model is obtained by training an improved ResNet18 using training data, and the training data includes wafer images of different defect patterns and their corresponding defect pattern labels; the improved ResNet18 includes a selective convolution kernel module, and the selective convolution kernel module is used to replace the first convolution layer of the ResNet18 backbone network, and the selective convolution kernel module is used to adaptively extract defect features of different sizes of wafer images.

[0010] In a possible implementation manner of the first aspect, the selective convolution kernel module combines 3×3 and 5×5 convolution kernels.

[0011] In a possible implementation of the first aspect, the adaptively extracting defect features of different sizes of the wafer image specifically includes:

[0012] 1) Wafer image features Two conversions are performed, the first conversion is The second conversion is

[0013] in, A wafer map feature representing a channel C′, a width W′, and a height H′; For the first conversion process; For the second conversion process; is the wafer map feature after the first conversion; is the wafer image feature after the second conversion;

[0014] 2) In order to adaptively adjust the size of the convolution kernel according to the defect content of the wafer image, the wafer image features after two conversions are integrated:

[0015]

[0016] Among them, U is the feature map after processing with different convolution kernels;

[0017] 3) Embedding global information is achieved by using global average pooling and generating data divided by channels

[0018]

[0019] Among them, s c is the characteristic of channel c; is the global average pooling operation; U c is the cth channel of the feature map U after convolution kernel processing; U c (i,j) is the value of the cth channel at position (i,j);

[0020] 4) Create a feature using a fully connected layer To achieve precision and adaptive selection guidance

[0021]

[0022] Among them, δ is the RELU function; is batch normalization; is a fully connected layer; z is a compact feature descriptor used to guide selective attention; s is the channel statistical feature obtained from U through global average pooling;

[0023] 5) Use cross-channel soft attention mechanism to adaptively select information of different spatial scales:

[0024]

[0025] in, is the cth row of matrix A, a c is the cth element of a; is the cth row of matrix B, b c is the cth element of b; a and b represent and Soft attention vector of ;

[0026] 6) Apply the softmax function to the soft attention vector

[0027]

[0028] 7) By focusing on each core, we get the defect features V of different sizes, where V c Features of the cth channel of the feature map.

[0029] In a possible implementation of the first aspect, an attention mechanism module is integrated in each convolutional layer of the ResNet18 backbone network, and the attention mechanism module is used to adaptively learn the importance weights of each channel and space.

[0030] In a possible implementation of the first aspect, an attention mechanism module is integrated in each convolutional layer of the ResNet18 backbone network, specifically:

[0031] The attention mechanism module includes a channel attention module and a spatial attention module.

[0032] The ResNet18 backbone network includes a first convolutional layer, a second convolutional layer, a third convolutional layer, a fourth convolutional layer and a fifth convolutional layer, and a channel attention module and a spatial attention module are inserted in sequence after the residual blocks of the second convolutional layer, the third convolutional layer, the fourth convolutional layer and the fifth convolutional layer.

[0033] According to a second aspect of the present invention, a wafer image defect pattern recognition device based on improved ResNet18 is provided, comprising:

[0034] An acquisition module, used for acquiring a wafer image to be identified;

[0035] A recognition module, used for inputting the wafer image to be recognized into a pre-trained wafer image defect pattern recognition model and outputting a recognition result;

[0036] Among them, the wafer image defect pattern recognition model is obtained by training the improved ResNet18 with training data, and the training data includes wafer images of different defect patterns and their corresponding defect pattern labels; the improved ResNet18 includes a selective convolution kernel module, and the selective convolution kernel module is used to replace the first convolution layer of the ResNet18 backbone network, and the selective convolution kernel module is used to adaptively extract defect features of different sizes of wafer images.

[0037] In a possible implementation manner of the second aspect, the selective convolution kernel module combines 3×3 and 5×5 convolution kernels.

[0038] In a possible implementation manner of the second aspect, the adaptively extracting defect features of different sizes of the wafer image specifically includes:

[0039] 1) Wafer image features Two conversions are performed, the first conversion is The second conversion is

[0040] in, A wafer map feature representing a channel C′, a width W′, and a height H′; For the first conversion process; For the second conversion process; is the wafer map feature after the first conversion; is the wafer image feature after the second conversion;

[0041] 2) In order to adaptively adjust the size of the convolution kernel according to the defect content of the wafer image, the wafer image features after two conversions are integrated:

[0042]

[0043] Among them, U is the feature map after processing with different convolution kernels;

[0044] 3) Embedding global information is achieved by using global average pooling and generating data divided by channels

[0045]

[0046] Among them, s c is the characteristic of channel c; is the global average pooling operation; U c is the cth channel of the feature map U after convolution kernel processing; U c (i,j) is the value of the cth channel at position (i,j);

[0047] 4) Create a feature using a fully connected layer To achieve precision and adaptive selection guidance

[0048]

[0049] Among them, δ is the RELU function; is batch normalization; is a fully connected layer; z is a compact feature descriptor used to guide selective attention; s is the channel statistical feature obtained from U through global average pooling;

[0050] 5) Use cross-channel soft attention mechanism to adaptively select information of different spatial scales:

[0051]

[0052] in, is the cth row of matrix A, a c is the cth element of a; is the cth row of matrix B, b c is the cth element of b; a and b represent and Soft attention vector of ;

[0053] 6) Apply the softmax function to the soft attention vector

[0054]

[0055] 7) By focusing on each core, we get the defect features V of different sizes, where V c Features of the cth channel of the feature map.

[0056] According to a third aspect of the present invention, there is provided a device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the method for identifying wafer image defect patterns based on improved ResNet18 are implemented.

[0057] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the wafer image defect pattern recognition method based on the improved ResNet18 are implemented.

[0058] Compared with the prior art, the present invention has at least the following beneficial effects:

[0059] The present invention provides a wafer image defect pattern recognition method based on improved ResNet18, which uses a selective convolution kernel module to replace the first convolution layer of the ResNet18 backbone network. The selective convolution kernel module is used to adaptively extract defect features of different sizes in the wafer image. The selective convolution kernel module can adaptively extract defect features of different sizes and types in the wafer image, thereby achieving accurate recognition of various defect patterns. The present invention introduces a selective convolution kernel module in ResNet18, which can dynamically adjust the parameters of the convolution kernel according to the input data, so that the model can better adapt to different input conditions, such as wafer defects of different sizes and types. In summary, by introducing the selective convolution kernel module, the flexibility and generalization ability of the model are enhanced, the ability to extract defect features of different sizes and types in the wafer image is enhanced, and the accuracy of wafer defect pattern recognition is improved.

[0060] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the technical solutions in the specific implementation modes of the present invention, the drawings required for use in the description of the specific implementation modes will be briefly introduced below. Obviously, the drawings described below are some implementation modes of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0062] Figure 1 This is a flow chart of a wafer image defect pattern recognition method based on improved ResNet18 of the present invention;

[0063] Figure 2 This is a diagram of the selective convolution kernel module of the present invention;

[0064] Figure 3 This is a structural diagram of the CAM of the present invention;

[0065] Figure 4 It is the structural diagram of SAM of the present invention;

[0066] Figure 5 This is a structural diagram of the CBAM-Residual block of the present invention;

[0067] Figure 6 The specific structure diagram of the SCKR-Net network of the present invention;

[0068] Figure 7 38 types of images in the mixed-mode wafer defect dataset;

[0069] Figure 8 Error histograms for mixed-mode wafer defect pattern recognition using different models;

[0070] Fig. 9 The precision detection results of different models on various defect types are shown in the figure;

[0071] Fig.10 The recall detection results of different models on various defect types are shown in the figure;

[0072] Fig.11 The following is a graph showing the F1 Score detection results of different models on various defect types. DETAILED DESCRIPTION

[0073] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0074] like Figure 1 As shown, an embodiment of the present invention provides a wafer image defect pattern recognition method based on an improved ResNet18, which mainly enhances the ability to extract defect features of different sizes and types in wafer images by introducing an improved ResNet18 network structure, especially a selective kernel convolution module (SKM), thereby improving the accuracy of wafer defect pattern recognition. The specific steps of the recognition method are as follows:

[0075] Step 1: Obtain a wafer image to be identified. Exemplarily, wafer image data collection is completed based on a test probe.

[0076] Step 2: Input the wafer image to be identified into a pre-trained wafer image defect pattern recognition model. The wafer image defect pattern recognition model will automatically extract features from the image and output the final defect pattern recognition result through a classifier. The recognition result may include the specific type of the defect.

[0077] Combination Figures 2 to 6 As shown, the wafer image defect pattern recognition model of the present invention is obtained by training an improved ResNet18 (SCKR-Net model) using training data, and the training data includes wafer images of different defect patterns and their corresponding defect pattern labels. The improved ResNet18 includes a selective convolution kernel module, which is used to replace the first convolution layer of the ResNet18 backbone network, and the selective convolution kernel module is used to adaptively extract defect features of different sizes of wafer images.

[0078] Specifically, the present invention uses ResNet18 as the basic network architecture because it has excellent feature extraction capabilities and relatively light computational burden, and is suitable for processing high-dimensional image data such as wafer images. In order to improve the sensitivity of the model to the defect features of wafer images, especially for weak and different-sized features, the present invention introduces a selective convolution kernel module (Selective Kernel Convolution, SKC) on the basis of ResNet18. The core idea of ​​SKC is to dynamically adjust the size and number of convolution kernels according to the different input features, so as to achieve adaptive extraction of defect features of different sizes and complexities. The first convolution layer of the ResNet18 backbone network is replaced by SKC. SKC contains multiple branches, each branch uses convolution kernels of different sizes (such as 3x3, 5x5, etc.), calculates the weight of each branch through the attention mechanism, and finally weights and fuses the outputs of each branch to obtain a more comprehensive and accurate feature representation.

[0079] The wafer image defect pattern recognition method proposed in the present invention realizes accurate recognition of various defect patterns in wafer images by constructing and applying an improved ResNet18 model. By introducing a selective convolution kernel module, the model can dynamically adjust the convolution kernel configuration according to the characteristics of different defects, effectively improving the recognition ability of weak and complex defect features.

[0080] Preferably, the selective convolution kernel module combines 3×3 and 5×5 convolution kernels. That is, the selective convolution kernel module combines two different sizes of convolution kernels, 3×3 and 5×5, and can adaptively and dynamically select convolution kernels of different sizes for feature extraction according to different input features.

[0081] Specifically, the 3×3 convolution kernel is suitable for capturing local detail features in wafer images, such as small scratches or particle contamination, while the 5×5 convolution kernel is more suitable for capturing larger defect features or global context information, such as cracks or stains.

[0082] In one possible implementation, a selective convolution kernel is used to aggregate information from multiple convolution kernels for different defect sizes in wafer images, thereby completing feature extraction. The selective convolution kernel consists of three parts: split, fuse, and select, which are used to generate branches with different convolution kernel sizes, aggregate different information to obtain a global representation of the selection weight, and aggregate feature maps of kernels of different sizes according to the selection weight.

[0083] The adaptive extraction of defect features of different sizes of wafer images specifically includes:

[0084] 1) Wafer image features Two conversions are performed, the first conversion is The second conversion is Both conversions are composed of deep convolution, batch normalization, and RELU function in sequence;

[0085] in, A wafer map feature representing a channel C′, a width W′, and a height H′; For the first conversion process; For the second conversion process; is the wafer map feature after the first conversion; It is the wafer image feature after the second conversion.

[0086] 2) In order to adaptively adjust the size of the convolution kernel according to the defect content of the wafer image, the wafer image features after two conversions are integrated:

[0087]

[0088] Among them, U is the feature map after being processed by different convolution kernels.

[0089] 3) Embedding global information is achieved by using global average pooling and generating data divided by channels

[0090]

[0091] Among them, sc is the characteristic of channel c; is the global average pooling operation; U c is the cth channel of the feature map U after convolution kernel processing; U c (i,j) is the value of the cth channel at position (i,j).

[0092] 4) Create a feature using a fully connected layer To achieve precision and adaptive selection guidance

[0093]

[0094] Among them, δ is the RELU function; is batch normalization; is a fully connected layer; z is a compact feature descriptor used to guide selective attention; s is the channel statistical feature obtained from U through global average pooling.

[0095] 5) Use cross-channel soft attention mechanism to adaptively select information of different spatial scales:

[0096]

[0097] in, is the cth row of matrix A, a c is the cth element of a; is the cth row of matrix B, b c is the cth element of b; a and b represent and The soft attention vector of .

[0098] 6) Apply the softmax function to the soft attention vector

[0099]

[0100] 7) By focusing on each core, we can obtain defect features V of different sizes, where V = [V1, V2, ..., V C ], V c Features of the cth channel of the feature map.

[0101] In one implementable manner, an attention mechanism module is integrated in each convolutional layer of the ResNet18 backbone network, and the attention mechanism module is used to adaptively learn the importance weights of each channel and space. An attention mechanism module is integrated in each convolutional layer of the ResNet18 backbone network, specifically: the attention mechanism module includes a channel attention module and a spatial attention module, the ResNet18 backbone network includes a first convolutional layer, a second convolutional layer, a third convolutional layer, a fourth convolutional layer, and a fifth convolutional layer, and the channel attention module and the spatial attention module are sequentially inserted after the residual blocks of the second convolutional layer, the third convolutional layer, the fourth convolutional layer, and the fifth convolutional layer.

[0102] That is to say, in order to further improve the model's ability to recognize wafer image defects, the attention mechanism module is integrated in the second convolutional layer, the third convolutional layer, the fourth convolutional layer and the fifth convolutional layer of the ResNet18 backbone network. The attention mechanism module can adaptively learn the importance weights of each channel and space, thereby enhancing the model's attention to key features and suppressing the interference of irrelevant or noise information. This implementation method designs two attention mechanism modules: a channel attention module (CAM) and a spatial attention module (SAM). The channel attention module (CAM) calculates the global average pooling and global maximum pooling of each channel, and then uses a simple fully connected layer and a Sigmoid activation function to generate the weight of each channel. These weights represent the contribution of different channels to the final feature representation. The spatial attention module (SAM) first average pools and max pools the feature map of each channel, then splices the two pooling results together, and generates a spatial attention map through a convolutional layer and a Sigmoid activation function. This attention map represents the importance of different spatial positions to the final feature representation. In the ResNet18 backbone network, the channel attention module and the spatial attention module are inserted successively after the residual blocks of the second convolutional layer, the third convolutional layer, the fourth convolutional layer, and the fifth convolutional layer. This design enables the model to adaptively adjust the feature representation after each convolutional layer, thereby better capturing the key defect features in the wafer image.

[0103] Next, the technical solution described in the present invention is fully and detailedly described by taking the Mixed-38 mixed mode wafer defect dataset as an example.

[0104] Step 1: Perform electrical performance tests on wafers collected from real wafer manufacturing plants to obtain wafer images. 0 represents no grain, 1 represents a grain that has passed the test, and 2 represents a grain that has not passed the test. The data size is 52×52. Since there are large differences in the number distribution of various types of wafer images actually collected, in order to maintain a balance between different types of data, a generative adversarial network is used to generate some wafer images to maintain a balance in the number of samples of each type.

[0105] Step 2: Simulate actual wafer defect data and set the data to normal samples, single defects, and mixed defects.

[0106] Step 3: On the basis of the commonly used accuracy indicators, three evaluation indicators Precision, Recall and F1 Score are added to evaluate the true expressiveness of the model on mixed-mode wafer image defect data. After ten experiments, the arithmetic mean is calculated to obtain the macro Precision, Recall and F1 Score.

[0107]

[0108]

[0109]

[0110]

[0111] Step 4: Build the SCKR-Net model and use the training data to train the built SCKR-Net model. The detailed structure of the built SCKR-Net model is as follows: Figure 6 The parameter settings of the entire network are shown in Table 1.

[0112] Table 1 SCKR-Net network structure settings

[0113]

[0114]

[0115] Add spatial attention module and channel attention module to the model. As input, CBAM sequentially derives 1D channel attention and 2D spatial attention The whole attention process can be summarized as

[0116]

[0117]

[0118] CBAM consists of two sub-modules: CAM (Channel Attention Module) and SAM (Spartial Attention Module), which perform channel and spatial attention respectively, saving parameters and computing power while ensuring that its plug-and-play module can be integrated into the existing network architecture.

[0119] In CAM, the channel dimension remains unchanged and the spatial dimension is compressed. This module focuses on the meaningful information of the input image. The specific calculation process is as follows:

[0120]

[0121] Among them, σ is the sigmoid function, and is the shared weight of MLP.

[0122] In SAM, the spatial dimension remains unchanged and the channel dimension is compressed. This module focuses on the target location information. The specific calculation process is as follows:

[0123]

[0124] Among them, σ is the sigmoid function, f 7×7 The padding size of the representative convolution is 7×7.

[0125] Finally, the CBAM module is inserted into the convolutional layer of the residual block to increase the attention to the meaningful information of the input features by compressing the spatial dimension and the attention to the target position information by compressing the channel dimension.

[0126] Use Residual Net as the backbone network to extract features from wafer images. Use the residual structure to build an ultra-deep network structure, and use BN to accelerate training.

[0127] 1) Residual learning: The residual is the difference between the observer and the estimated value. After a series of processing of the input X, the residual F(X) is obtained. If the downsampling process is not performed on the shortcut branch, the residual mapping function obtained in the end is F(X)+X. If downsampling is required on the shortcut branch, conv1×1 convolution should be used for dimensionality increase.

[0128] 2) Residual block: The residual block of ResNet has two 3×3 convolutional layers with the same number of output channels. Each convolutional layer is followed by a BN layer and a RELU activation function. In the residual block, the input can be propagated forward faster through the cross-layer data line. At the same time, through jump connections, the signal can also be transmitted without attenuation during back propagation. It can alleviate the gradient vanishing or gradient exploding phenomenon caused by deepening the layer. At the same time, it can also reduce the occurrence of overfitting in deep networks.

[0129] Step 5: Input the test data into the trained SCKR-Net model to obtain the diagnosis results. In order to demonstrate the superiority of the proposed method, it is compared with various benchmark methods. When using the Mixed-38 dataset in step 1 for verification, 80% of the samples are randomly selected as the training set and the other samples are used as the test set. The training set is randomly flipped and randomly rotated. In the test phase, since the probability value output by the model is between 0 and 1, a threshold needs to be given to determine whether the defect exists. If the probability value is greater than the threshold, it means that the defect exists, otherwise the defect does not exist. We set the threshold to 0.5. In the present invention, the proposed framework is implemented by PyTorch and Python 3.6, using RTX 3090GPU and Inter(R)Xeon(R)Silver 4210R CPU@2.40GHz. The optimization algorithm is SGD, the learning rate is 0.01, the weight decay rate is 0.001, the batch size is 64, and the epoch is 300. Three benchmark deep learning methods are selected to compare with the proposed SCKR-Net model, and a comprehensive evaluation is performed through four evaluation indicators to verify the effectiveness of SCKR-Net. Several comparison methods are DCNet, CWDR, and ESPPNet. The experimental parameters of the three comparison methods are set the same. In addition, we repeat each method 10 times to reduce the influence of chance. The comparison results of the four models are as follows:

[0130] 1) Classification accuracy and experimental results Figure 8 As shown in Table 2, it is obvious that SCKR-Net produces the best results on all datasets. The proposed SCKR-Net has an accuracy of 98.69%, a precision of 98.63%, a recall of 98.61%, and an F1 score of 98.61%, achieving the best performance in all four evaluation indicators, outperforming other models.

[0131] 2) To further illustrate the effectiveness of the proposed SCKR-Net model, the detection results of the four models on various defects are shown in the following table. Fig. 9 shown.

[0132] Table 2 Recognition results of different methods on Mixed-38 dataset.

[0133]

[0134] Through the above comparative tests, the following results can be further obtained:

[0135] 1. On the Mixed-38 wafer dataset, the proposed SCKR-Net has an accuracy of 98.69%, a precision of 98.63%, a recall of 98.61%, and an F1 score of 98.61%, achieving the best performance in all four evaluation indicators, outperforming other models

[0136] 2. From Fig. 9 It can be concluded that although the recognition accuracy of the SCKR-Net proposed by the present invention is relatively low in some types, the recognition performance of most types has been significantly improved, especially after the increase of mixed defect types, it can still maintain its superior performance. From the overall classification of Precision, Recall, and F1 Score indicators, the SCKR-Net model has a better classification effect.

[0137] In order to verify the effectiveness of the proposed channel attention mechanism and selective convolution kernel, an ablation experiment is conducted. As in the comparative experiment, in order to ensure the fairness of the experiment, the parameters of each model experiment are set the same.

[0138] Through the above ablation test, the following results can be further obtained:

[0139] 1. The results are shown in Table 3. After adding CBAM, the model accuracy is improved compared with the baseline model. After adding the selective convolution kernel, the model accuracy is improved by 0.35% to 98.69%. The results show the effectiveness of CBAM and selective convolution kernels.

[0140] 2. The use of channel and spatial attention modules improves the feature extraction capability while improving the model performance and network representation capabilities.

[0141] 3. The use of selective convolution kernel structure improves the problem of low recognition accuracy caused by different wafer defect sizes.

[0142] Table 3. Comparison results of ablation experiments

[0143]

[0144] In another embodiment of the present invention, a wafer image defect pattern recognition device based on improved ResNet18 is provided, comprising:

[0145] The acquisition module is used to acquire the wafer image to be identified.

[0146] The recognition module is used to input the wafer image to be identified into a pre-trained wafer image defect pattern recognition model and output a recognition result; wherein the wafer image defect pattern recognition model is obtained by training an improved ResNet18 using training data, and the training data includes wafer images of different defect patterns and their corresponding defect pattern labels; the improved ResNet18 includes a selective convolution kernel module, and the selective convolution kernel module is used to replace the first convolution layer of the ResNet18 backbone network, and the selective convolution kernel module is used to adaptively extract defect features of different sizes of wafer images.

[0147] All relevant contents of each step involved in the embodiment of the aforementioned wafer image defect pattern recognition method based on improved ResNet18 can be referred to the functional description of the functional module corresponding to the wafer image defect pattern recognition device based on improved ResNet18 in the embodiment of the present invention, and will not be repeated here. The division of modules in the embodiment of the present invention is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional module in each embodiment of the present invention can be integrated into a processor, or it can exist physically separately, or two or more modules can be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware or in the form of software functional modules.

[0148] In another embodiment of the present invention, a computer device is provided, the computer device including a processor and a memory, the memory is used to store a computer program, the computer program includes program instructions, and the processor is used to execute the program instructions stored in the computer storage medium. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc., which are the computing core and control core of the terminal, which are suitable for implementing one or more instructions, and are specifically suitable for loading and executing one or more instructions in a computer storage medium to implement the corresponding method flow or corresponding function; the processor described in the embodiment of the present invention can be used for an operation of a wafer image defect pattern recognition method based on an improved ResNet18.

[0149] In another embodiment of the present invention, the present invention also provides a storage medium, specifically a computer-readable storage medium (Memory), which is a memory device in a computer device for storing programs and data. It is understandable that the computer-readable storage medium here may include a built-in storage medium in a computer device, and of course may also include an extended storage medium supported by the computer device. The computer-readable storage medium provides a storage space, which stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by a processor are also stored in the storage space, and these instructions may be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium here may be a high-speed RAM memory, or a non-volatile memory (non-volatile memory), such as at least one disk memory. The processor may load and execute one or more instructions stored in the computer-readable storage medium to implement the corresponding steps of the above-mentioned embodiment regarding a wafer image defect pattern recognition method based on an improved ResNet18.

[0150] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0151] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0152] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0153] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0154] In the present invention, the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" etc. mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the described specific features, structures, materials or characteristics may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0155] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A wafer image defect pattern recognition method based on improved ResNet18, characterized in that: include: Obtaining a wafer image to be identified; The wafer image to be identified is input into a pre-trained wafer image defect pattern recognition model, and a recognition result is output; wherein the wafer image defect pattern recognition model is obtained by training an improved ResNet18 using training data, and the training data includes wafer images of different defect patterns and their corresponding defect pattern labels; the improved ResNet18 includes a selective convolution kernel module, and the selective convolution kernel module is used to replace the first convolution layer of the ResNet18 backbone network, and the selective convolution kernel module is used to adaptively extract defect features of different sizes of wafer images.

2. According to a wafer image defect pattern recognition method based on improved ResNet18 according to claim 1, it is characterized in that: The selective convolution kernel module combines 3×3 and 5×5 convolution kernels.

3. According to a wafer image defect pattern recognition method based on improved ResNet18 according to claim 2, it is characterized in that: The adaptive extraction of defect features of different sizes of wafer images specifically includes: 1) Wafer image features Two conversions are performed, the first conversion is The second conversion is in, Indicates that the channel is C ′, width is W ′, height is H ′ wafer map features; For the first conversion process; For the second conversion process; is the wafer map feature after the first conversion; is the wafer image feature after the second conversion; 2) In order to adaptively adjust the size of the convolution kernel according to the defect content of the wafer image, the wafer image features after two conversions are integrated: Among them, U is the feature map after processing with different convolution kernels; 3) Embedding global information is achieved by using global average pooling and generating data divided by channels Among them, s c is the characteristic of channel c; is the global average pooling operation; U c is the cth channel of the feature map U after convolution kernel processing; U c (i,j) is the value of the cth channel at position (i,j); 4) Create a feature using a fully connected layer To achieve precision and adaptive selection guidance Among them, δ is the RELU function; is batch normalization; is a fully connected layer; z is a compact feature descriptor used to guide selective attention; s is the channel statistical feature obtained from U through global average pooling; 5) Use cross-channel soft attention mechanism to adaptively select information of different spatial scales: in, is the cth row of matrix A, a c is the cth element of a; is the cth row of matrix B, b c is the cth element of b; a and b represent and Soft attention vector of ; 6) Apply the softmax function to the soft attention vector 7) By focusing on each core, we get the defect features V of different sizes, where V c Features of the cth channel of the feature map.

4. According to a wafer image defect pattern recognition method based on improved ResNet18 according to claim 1, it is characterized in that: An attention mechanism module is integrated in each convolutional layer of the ResNet18 backbone network, which is used to adaptively learn the importance weights of each channel and space.

5. According to a wafer image defect pattern recognition method based on improved ResNet18 according to claim 4, it is characterized in that: The attention mechanism module is integrated into each convolutional layer of the ResNet18 backbone network, specifically: The attention mechanism module includes a channel attention module and a spatial attention module. The ResNet18 backbone network includes a first convolutional layer, a second convolutional layer, a third convolutional layer, a fourth convolutional layer and a fifth convolutional layer, and a channel attention module and a spatial attention module are inserted in sequence after the residual blocks of the second convolutional layer, the third convolutional layer, the fourth convolutional layer and the fifth convolutional layer.

6. A wafer image defect pattern recognition device based on improved ResNet18, characterized in that: include: An acquisition module, used for acquiring a wafer image to be identified; A recognition module, used for inputting the wafer image to be recognized into a pre-trained wafer image defect pattern recognition model and outputting a recognition result; Among them, the wafer image defect pattern recognition model is obtained by training the improved ResNet18 with training data, and the training data includes wafer images of different defect patterns and their corresponding defect pattern labels; the improved ResNet18 includes a selective convolution kernel module, and the selective convolution kernel module is used to replace the first convolution layer of the ResNet18 backbone network, and the selective convolution kernel module is used to adaptively extract defect features of different sizes of wafer images.

7. The wafer image defect pattern recognition device based on improved ResNet18 according to claim 6 is characterized in that: The selective convolution kernel module combines 3×3 and 5×5 convolution kernels.

8. According to claim 7, a wafer image defect pattern recognition device based on improved ResNet18 is characterized in that: The adaptive extraction of defect features of different sizes of wafer images specifically includes: 1) Wafer image features Two conversions are performed, the first conversion is The second conversion is in, Indicates that the channel is C ′, width is W ′, height is H ′ wafer map features; For the first conversion process; For the second conversion process; is the wafer map feature after the first conversion; is the wafer image feature after the second conversion; 2) In order to adaptively adjust the size of the convolution kernel according to the defect content of the wafer image, the wafer image features after two conversions are integrated: Among them, U is the feature map after processing with different convolution kernels; 3) Embedding global information is achieved by using global average pooling and generating data divided by channels Among them, s c is the characteristic of channel c; is the global average pooling operation; U c is the cth channel of the feature map U after convolution kernel processing; U c (i,j) is the value of the cth channel at position (i,j); 4) Create a feature using a fully connected layer To achieve precision and adaptive selection guidance Among them, δ is the RELU function; is batch normalization; is a fully connected layer; z is a compact feature descriptor used to guide selective attention; s is the channel statistical feature obtained from U through global average pooling; 5) Use cross-channel soft attention mechanism to adaptively select information of different spatial scales: in, is the cth row of matrix A, a c is the cth element of a; is the cth row of matrix B, b c is the cth element of b; a and b represent and Soft attention vector of ; 6) Apply the softmax function to the soft attention vector 7) By focusing on each core, we get the defect features V of different sizes, where V c Features of the cth channel of the feature map.

9. A device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of a wafer image defect pattern recognition method based on improved ResNet18 are implemented as described in any one of claims 1 to 5.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of a wafer image defect pattern recognition method based on improved ResNet18 are implemented as described in any one of claims 1 to 5.

Images