Wafer defect detection and positioning method based on cascade YOLO-GAN

Abstract

This invention belongs to the fields of deep learning computer machine vision and semiconductor technology inspection, proposing a wafer defect detection and localization algorithm based on cascaded YOLO-GAN. During the wafer manufacturing process, the original wafer image is fed into a wafer detection model based on an improved YOLOv5 and a wafer semantic segmentation model based on BiseNet to obtain the wafer target detection box positions and the wafer foreground mask. The original image is then input into a defect detection model based on an improved generative adversarial network to reconstruct the wafer image and locate the wafer defect region. Using the wafer target detection box positions as constraints, the connected components of the defect image are analyzed, and a Softmax classifier is introduced to achieve defect localization and wafer defect subdivision. This invention enables visual inspection of wafers of different sizes, is suitable for automatic wafer defect detection, and can greatly improve the accuracy and efficiency of defect detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of deep learning computer machine vision and semiconductor technology detection, and particularly relates to a wafer defect detection and positioning method based on cascaded YOLO-GAN. BACKGROUND

[0002] With the rapid development of China's semiconductor design and manufacturing technology, as the wafer needed for the production of semiconductor circuits, the production and manufacturing process of the wafer is particularly important. Because the micro-nano processing process of the wafer is relatively complex, it is divided into uniform glue, mold pressing, and imprinting processes, and various types of wafer defects such as incomplete imprinting pattern, bubble in transferred layer, surface scratch, and particle defect are easily generated in these processes, thereby directly reducing the yield of the produced wafer and greatly affecting the electrical characteristics of the chip. Therefore, it is necessary to position and detect defects of the wafer during the manufacturing process of the semiconductor to remove the defective wafer and ensure the yield of the wafer.

[0003] The core of detecting the wafer and its defects by using machine vision technology is how to accurately extract features related to the wafer and defect discrimination from the wafer image. The commonly used method relies on manually designed features, such as a wafer detection method and system in patent CN 114709146A, which has low feature robustness and is only suitable for detection tasks in simple scenes, and has insufficient detection capability for the wafer and its defects in complex environments, and cannot achieve the speed and accuracy required by engineering practice.

[0004] In recent years, deep learning has been widely applied in the field of computer vision, and especially in the fields of target detection and surface defect detection, and has made great progress. The present application proposes a wafer defect detection and positioning method based on cascaded YOLO-GAN, which integrates and improves the target detection, semantic segmentation, and image generation to form a joint system according to the characteristics of the wafer defects generated in the production process. SUMMARY

[0005] The present application aims to provide a wafer defect detection and positioning method based on cascaded YOLO-GAN, which proposes to use cascaded YOLO-GAN to integrate target detection, semantic segmentation, and image reconstruction technology to realize the wafer defect detection and positioning algorithm.

[0006] The technical scheme of the present application is as follows:

[0007] A wafer defect detection and positioning method based on cascaded YOLO-GAN, comprising the following steps:

[0008] 1) In the wafer production and manufacturing process, the original image wafer is sent into the wafer detection model based on improved YOLOv5 to obtain the wafer target detection frame position;

[0009] 2) input the original image wafer to the BiseNet-based wafer semantic segmentation model, perform wafer semantic segmentation, and obtain a foreground mask m of the wafer;

[0010] 3) input the original image T to the defect detection model based on the improved generative adversarial network, the defect detection model based on the improved generative adversarial network comprising a generator G, a discriminator D and a secondary encoder E; the original image is generated into a reconstructed image by the generator, and a content loss L con reflects the difference between the reconstructed image and the real image; the secondary encoder encodes the reconstructed image to obtain a secondary hidden variable, and a coding loss function L enc reduces the difference between the hidden variable of the original image and the secondary hidden variable of the reconstructed image; the original image and the reconstructed image are input into the discriminator, and the generator and the discriminator are iteratively optimized to reach Nash equilibrium until the generated adversarial loss of the original image and the reconstructed image in the game process meets the requirements, and the defect detection model based on the improved generative adversarial network is trained; the difference between the original image T and the reconstructed image in the discriminator is obtained to obtain a difference image; the foreground pixel number in the difference image ΔT is compared with a given threshold by using the foreground mask of the wafer as a constraint, and it is judged whether there is a defect or not;

[0011] 4) when the detection result of step 3) exists defects, the wafer target detection frame position in step 1) is used as a constraint to analyze the connected domain of the defect image, and a Softmax classifier is introduced to realize the positioning of the defects and the subdivision of the wafer defects; the Softmax classifier uses a cross-entropy loss function to evaluate and adjust whether the wafer defect classification is correct or not; when the detection result of step 3) does not exist defects, the wafer positioning and defect detection result are directly output.

[0012] The step 1) of improving the wafer detection model of YOLOv5 is improved based on single-stage YOLOv5; the wafer detection model of the application needs to consider the speed and accuracy of detection when applied to edge computing devices, therefore, the single-stage YOLOv5 detection algorithm is introduced and improved; the traditional YOLOv5 output module includes three different size feature maps, different feature maps are sent into the detection head at the same time to detect targets of different scales in the image, since the wafer image usually contains a single or multiple wafers of similar scales, the traditional YOLOv5 output module needs to be improved, the mobilenet convolution operator is used to replace the standard convolution to improve the feature fusion speed, and the wafer visual features are output to the YOLOv5 detection head to realize wafer detection; the output module of the improved wafer detection model of YOLOv5 uses the mobilenet convolution operator to replace the standard convolution of the single-stage YOLOv5 to improve the feature fusion speed; the improved wafer detection model of YOLOv5 includes a skeleton network, a feature enhancement network, an up-down sampling network layer, a mobilenet convolution layer and a detection head network layer;

[0013] The original wafer image is subjected to the skeleton network and the feature enhancement network to obtain multi-scale feature information streams from low-level feature scale 1 / 2x to high-level feature scale 1 / 16x; then the up-down sampling network layer and the 1x1 convolution kernel are used to integrate the weighted information in the depth direction of the feature map to obtain feature maps of different sizes; the feature maps of different sizes are simultaneously sent into the detection head network layer to detect targets of different scales in the feature map, so as to realize wafer detection and obtain the wafer target detection frame position.

[0014] In the step 2), the wafer semantic segmentation model based on BiseNet specifically adopts a bidirectional semantic segmentation network to realize target detection speed, and also pays more attention to spatial information; the network reserves spatial position information by designing a small-step spatial path to generate a high-resolution feature map; in addition, an objective receptive field is obtained by designing a context path containing a fast down-sampling rate; a new feature fusion module is integrated into the above two parts to fuse the feature maps of the two parts, so that the speed and accuracy are balanced; the original wafer image is input, and the features are extracted through the spatial path and the context path respectively; in the spatial path, the convolution batch processing is subjected to batch normalization and activation to reserve the spatial scale of the input wafer image and enrich the spatial information coding; the context path combines the attention optimization module to optimize the feature output of each section; the features extracted by the spatial path and the context path are input into the feature fusion module, and the BiseNet output is supervised by the main loss function, so that the foreground mask of the wafer is obtained.

[0015] The step 3) includes a generator G, a discriminator D and a secondary encoder E based on the improved generative adversarial network; the generator G is a generative network, which includes an encoder G E and a decoder G D ; an original image is input into the encoder G E and the decoder G D to generate a reconstructed image The encoder G E in the generator G encodes the original image T to obtain a hidden variable a, and the hidden variable a is input into the decoder G D in the generator G to decode and obtain the reconstructed image W GD , W GE respectively represent the learnable weights of the encoder G E and the decoder G D ; the secondary encoder E encodes the reconstructed image to obtain a secondary hidden variable as shown in the following formula: W E represents the learnable parameters of the decoder E; an encoding loss is obtained according to the difference between the hidden variable a and the secondary hidden variable ; the discriminator D is a discriminative network, which is a three-layer convolutional neural network; the original image and the reconstructed image are input into the discriminator D, which discriminates and outputs whether the reconstructed image is a real image, and outputs l D ,

[0016] The defect detection model based on the improved generative adversarial network adopts a weighted joint generation loss function; three loss functions, including an adversarial loss, a content loss and an encoding loss, are designed, and a weighted joint loss function is constructed; the adversarial loss function L adv represents the loss generated by the generator and the discriminator in the game process, which is calculated using an L2 norm loss function, as shown in the following formula: L adv =||f(r)-f(G(r)||2; the content loss L con reflects the difference between the reconstructed image and the real image T, which is calculated using an L1 norm loss function, as shown in the following formula: L con =||T-G(r)||1; the encoding loss L enc is used to reduce the gap between the input image feature vector T and the reconstructed image feature vector , which is calculated using an L2 norm loss function, as shown in the following formula: L enc =||G E (x)-E(G(x))||2, and finally, a weighted joint loss function L is generated, as shown in the following formula: L=ω adv Ladv +ω con L con +ω enc L enc ,ω adv ,ω con ,ω enc is a weighting parameter that adjusts the influence of individual losses on the overall objective function.

[0017] When the detection result has defects, the defect area obtained by training the generative adversarial network is enlarged by 2 times according to the width and the like to obtain a wafer defect mask M, M contains the defect area and its spatial context information, and the wafer defect mask M is used to extract the subdivided defect features from the target detection frame, and the process is as follows: D = F O M; wherein O represents point-by-point multiplication, F represents the feature map output by the mobilenet convolution operator, and D represents the wafer defect related target detection frame; a multilayer perceptron MLP is used to encode and reduce the dimension of D to obtain a reduced dimension feature encoding D'; after the reduced dimension feature encoding D' passes through a Softmax classifier, a cross-entropy loss function is used for classification, as shown in the following formula: k represents the number of subdivided defect types, represents one-hot encoding of the label, d i represents the output of the Softmax classifier.

[0018] The beneficial effects of the present application are:

[0019] (1) The multi-scale output features of YOLOv5 are integrated to realize visual detection of wafers of different scales;

[0020] (2) The foreground mask of the wafer is calculated by introducing a lightweight BiseNet, thereby suppressing the interference of irrelevant background noise;

[0021] (3) The wafer defect related features are extracted by combining the wafer defect area positioning and the visual features obtained by YOLOv5, thereby realizing the subdivision of the defects;

[0022] (4) The present application is suitable for automatic detection of wafer defects, and can greatly improve the accuracy and efficiency of defect detection. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1 is the overall algorithm flowchart of the wafer defect detection and positioning of the present application based on cascade YOLO-GAN;

[0024] Figure 2 is the wafer detection model structure diagram based on improved YOLOv5 proposed in the present application;

[0025] Figure 3is a wafer semantic segmentation model based on BiseNet proposed in the application;

[0026] Figure 4 is a defect detection model based on improved generative adversarial network proposed in the application. DETAILED DESCRIPTION

[0027] The application will be further described below with reference to the accompanying drawings, but the protection scope of the application is not limited thereto;

[0028] Figure 1 A wafer defect detection and positioning overall method flow chart based on cascade YOLO-GAN is given:

[0029] The application proposes a wafer defect detection and positioning method based on cascade YOLO-GAN, and combines an improved YOLOv5 target detection model, reconstructs a wafer region by using a defect detection model based on improved generative adversarial network, detects and positions a wafer defect region unsupervisedly through the difference between a reconstructed image and an original image, and simultaneously introduces a wafer semantic segmentation model based on BiseNet to calculate a foreground mask of the wafer, so that the defect is subdivided.

[0030] The application sends wafer pictures into the detection model of improved YOLOv5 and the semantic segmentation model based on BiseNet respectively by deep learning, obtains wafer target detection frame positions and corresponding foreground masks, positions a defect region from an original image according to a wafer detection result, introduces improved GAN for region reconstruction, removes irrelevant background interference by using the foreground mask of the wafer, judges whether there is a defect and realizes defect positioning according to a reconstruction result, and finally extracts visual features related to wafer defects from the improved YOLOv5 according to the defect positioning result, so that wafer defect subdivision is realized.

[0031] The specific operation steps of the application are as follows:

[0032] Figure 2 A wafer detection model structure diagram based on improved YOLOv5 is given:

[0033] 1) Framework of the YOLOv5 target detection model

[0034] The application introduces a single-stage YOLOv5 for improvement, which can balance the speed and accuracy of wafer detection, the framework of the YOLOv5 target detection model mainly includes a skeleton network, a feature enhancement network and an output module; and the wafer image usually contains a single or multiple scales of wafers, so the output module of the traditional YOLOv5 can be improved; the input wafer image x sequence is subjected to the skeleton network and the feature enhancement network of the YOLOv5, so as to obtain multi-scale feature information flow from low-level feature scale 1 / 2x to high-level feature scale 1 / 16x; then, the feature map is subjected to weighted information integration in the depth direction through upsampling and downsampling and 1x1 convolution kernel, so as to obtain the feature map x';

[0035] 2) MobileNet convolution operator replaces standard convolution

[0036] The basic unit of MobileNet is a depthwise separable convolution, which is actually a decomposable convolution operation. Unlike standard convolution, which uses a convolution kernel on all input channels, the depthwise separable convolution uses different convolution kernels for each input channel, i.e., one convolution kernel corresponds to one input channel. In the improved YOLOv5 target detection model framework, MobileNet is used instead of standard convolution, which can improve the feature fusion speed and greatly reduce the computational complexity and model parameter quantity. The output wafer visual features are sent to the YOLOv5 detection head to realize wafer detection, and the whole process is S=Head(MB(x', W B ), W H ), wherein S represents the wafer target detection frame position, Head() and MB() represent the YOLOv5 detection head and MobileNet convolution operator respectively, W H and W B represent learning parameters respectively;

[0037] Figure 3 A wafer semantic segmentation model based on BiseNet is shown in the schematic diagram:

[0038] 1) Introducing BiseNet real-time semantic segmentation network framework

[0039] The application adopts a bidirectional semantic segmentation network, which can realize target detection speed, and also pays more attention to spatial information. The network retains spatial position information by designing a small-step spatial path, thereby generating a high-resolution feature map. In addition, by designing a semantic path containing a fast downsampling rate, an objective receptive field is obtained. A new feature fusion module is integrated into the above two parts to fuse the feature maps of the two parts, so that the speed and accuracy are balanced.

[0040] Figure 4An improved generative adversarial network-based defect detection model is proposed:

[0041] 1) Improved generative adversarial network

[0042] The improved generative adversarial network (GAN) -based defect detection model proposed in the application comprises a generator G, a discriminator D and a secondary encoder E, wherein the generator G is implemented through a self-encoder. The encoder GE in G encodes the input image T to obtain the latent variable a, and then sends it to the decoder GD to obtain the reconstructed image The formula is as follows: Wherein, W GD , W GE respectively represent the learnable weights of the encoder G E and the decoder G D , the discriminator D is implemented using a three-layer convolutional neural network, and respectively receives the real image T and the reconstructed image and judges the authenticity, and the formula is as follows: Wherein, l D represents the output result of the discriminator D, and W D represents its learnable parameters; in order to improve the sample generation ability of the generator G, a secondary encoder E is introduced to encode the reconstructed image to obtain the encoding result and let approximate a, and the formula is as follows: Wherein, W E represents the learnable parameters of the decoder E;

[0043] 2) Design three loss functions

[0044] In order to train the improved generative adversarial network-based defect detection model, the following three loss functions are designed, and a joint loss function is constructed through weighting, as follows:

[0045] Adversarial loss: represents the loss generated by the generator and the discriminator in the game process, calculated using the L2 norm loss function, and the formula is as follows: L adv =||f(r)-f(G(r)||2;

[0046] Content loss: reflects the difference between the reconstructed image and the real image, calculated using the L1 norm loss function, and the formula is as follows: L con =||T-G(r)||1;

[0047] Encoding loss: used to reduce the gap between the input image feature vector and the reconstructed image feature vector, calculated using the L2 norm loss function, as follows: L enc =||G E (x)-E(G(x))||2;

[0048] Finally, the joint loss function is generated by weighting, and the following formula is used for calculation: L=w adv L adv +w con L con +w enc L enc , wherein w adv , w con and w enc are weighting parameters for adjusting the influence of individual loss on the overall objective function.

[0049] 3) Softmax classifier detects wafer defects

[0050] The Softmax classifier uses a cross-entropy loss function to evaluate and adjust the correctness of the classification; the Softmax classifier can expand the score gap, that is, make the score function score results have little difference, and through the Softmax classifier, the score gap can be further enlarged, so that the classification effect is more obvious. In the process of extracting defect-related visual features, the defect-related features are sent into the Softmax classifier to detect the defects of the wafer.

[0051] 4) Sub-defect features are extracted according to Mobile convolution

[0052] In the case of defects in the wafer, the defect area obtained by improving the wafer defect area positioning of the GAN is enlarged by 2 times according to the width and other proportions to obtain a wafer defect mask M, and M contains the defect area and its spatial context information. Then, the M is used to extract the sub-defect features from the feature map output by the Mobile convolution module in the improved YOLOv5, and the process is as follows: D=F⊙M;In the formula, ⊙ represents point-by-point multiplication, F represents the feature map output by the Mobile convolution, and D represents the wafer defect-related visual features. Further, the multi-layer perceptron (MLP) is used to encode and reduce the dimension of D to obtain the reduced feature encoding D'. In order to successfully train the wafer defect classifier, D' is subjected to the Softmax activation function, and the cross-entropy loss function is used for classification, and the following formula is used: In the formula, k represents the number of sub-defect types, represents the one-hot encoding of the label, and di represents the output of the Softmax activation function.

[0053] 1) In order to verify the effectiveness of the improved strategy, the present application uses common target detection algorithms for comparison, including Faster RCNN, SSD, YOLOv3, YOLOv4, YOLOv5, as shown in Table 1. At the same time, the present application compares the processing speed of different algorithms. The improved YOLOv5 proposed in the present application has the best performance in terms of average precision mean and processing speed. Compared with other commonly used target detection algorithms, YOLOv5 has better performance in terms of average precision mean and processing speed. In contrast, the processing speed of Faster RCNN, a two-stage algorithm, reaches 142.5 ms, and the mAP index is weaker than YOLOv5. Compared with the original YOLOv5, the present application improves the information aggregation method of the output module, thus improving the average precision mean by 0.34% and accelerating the average processing speed by 4.1 ms.

[0054] Table 1 Comparison of wafer detection algorithm performance

[0055]

[0056] Table 2 Wafer detection algorithm ablation experiment data table

[0057]

[0058] 2) As can be seen from Table 2, the cascade YOLO-GAN proposed in the present application has the best wafer defect detection performance. In addition, the improvement of GAN and YOLOv5 helps to improve the average recognition rate of defect detection. By comparing the results with and without BiseNet, it is not difficult to see that the wafer mask is of great help to the detection of wafer defects. By suppressing irrelevant background interference, the network can focus on the visual features of wafer defects, thereby better subdividing the wafer defect categories.

[0059] The above describes the present application simply and is not limited by the above working range. As long as the idea and working method of the present application are used for simple modification and application to other equipment, or improved and decorated without changing the main concept of the present application, they are within the protection scope of the present application.

Claims

1. A wafer defect detection and positioning method based on a cascaded YOLO-GAN, characterized in that, The steps include the following; 1) In the wafer generation manufacturing process, the original image wafer is sent into the wafer detection model based on improved YOLOv5 to obtain the wafer target detection frame position; 2) The original image wafer is input into the wafer semantic segmentation model based on BiseNet, wafer semantic segmentation is performed, and the foreground mask m of the wafer is obtained; 3), the original image T is input to the defect detection model based on the improved generative adversarial network, the defect detection model based on the improved generative adversarial network includes generator G, discriminator D and secondary encoder E;The original image is generated into a reconstructed image by the generator, and the content loss L con reflects the difference between the reconstructed image and the real image;The secondary encoder is used for secondary encoding of the reconstructed image to obtain secondary hidden variables, and the encoding loss function L enc reduce the difference between the hidden variables of the original image and the secondary hidden variables of the reconstructed image;The original image and the reconstructed image are input to the discriminator, and the generator and the discriminator are iteratively optimized to reach Nash equilibrium until the generated adversarial loss of the original image and the reconstructed image in the game process meets the requirements, and the defect detection model based on the improved generative adversarial network is trained;The difference between the original image T and the reconstructed image in the discriminator is obtained to obtain the difference image ΔT;The foreground mask of the wafer is used for constraint, and then the number of foreground pixels in the difference image ΔT is compared with the given threshold to determine whether there is a defect. 4) When the detection result of step 3) exists defects, the wafer target detection frame position in step 1) is used as a constraint to analyze the connected domain of the defect image, and a Softmax classifier is introduced to realize the positioning of the defect and the subdivision of the wafer defect; the Softmax classifier uses a cross-entropy loss function to evaluate, and adjusts whether the wafer defect classification is correct or not; when the detection result of step 3) does not exist defects, the wafer positioning and defect detection result are directly output.

2. The wafer defect detection and positioning method based on the cascade YOLO-GAN according to claim 1, characterized in that, The wafer detection model of the improved YOLOv5 in step 1) is improved based on single-stage YOLOv5, the output module of the wafer detection model of the improved YOLOv5 uses a mobilenet convolution operator to replace the standard convolution of the single-stage YOLOv5 to improve the feature fusion speed; the wafer detection model of the improved YOLOv5 includes a skeleton network, a feature enhancement network, an up-down sampling network layer, a mobilenet convolution layer and a detection head network layer; The original image wafer passes through the skeleton network and the feature enhancement network to obtain multi-scale feature information flow from low-level feature scale 1 / 2x to high-level feature scale 1 / 16x; Then, the up-down sampling network layer and the 1x1 convolution kernel are used to integrate the weighted information in the depth direction of the feature map to obtain feature maps of different sizes; the feature maps of different sizes are simultaneously input into the detection head network layer to detect different scale targets in the feature map, and wafer detection is realized to obtain the wafer target detection frame position.

3. The wafer defect detection and positioning method based on the cascade YOLO-GAN according to claim 1, characterized in that, In step 2), the wafer semantic segmentation model based on BiseNet is specifically The wafer semantic segmentation model based on BiseNet uses a bidirectional semantic segmentation network, inputs the original image wafer, extracts features through a space path and a context path respectively, and performs batch normalization and activation through convolution batch processing in the space path to retain the spatial scale of the input wafer image and enrich the spatial information coding; the context path combines an attention optimization module to optimize the feature output of each section; the features extracted by the space path and the context path are input into a feature fusion module, supervised by a main loss function, and the BiseNet output is obtained, so as to obtain the foreground mask of the wafer.

4. The wafer defect detection and positioning method based on the cascaded YOLO-GAN according to claim 1, characterized in that, The step 3) includes a generator G, a discriminator D and a secondary encoder E based on the improved defect detection model of the generative adversarial network; the generator G is a generative network, which includes an encoder G E and a decoder G D ; an original image is input into the encoder G E and the decoder G D to generate a reconstructed image The encoder G E in the generator G encodes the original image T to obtain a hidden variable a, the hidden variable a is input into the decoder G D in the generator G to decode and obtain the reconstructed image W GD , W GE respectively represent the learnable weights of the encoder G E and the decoder G D ; the secondary encoder E encodes the reconstructed image to obtain a secondary hidden variable as shown in the following formula: W E represents the learnable parameters of the decoder E; an encoding loss is obtained according to the difference between the hidden variable a and the secondary hidden variable ; the discriminator D is a discriminative network, which is a three-layer convolutional neural network; the original image and the reconstructed image are input into the discriminator D, which discriminates and outputs whether the reconstructed image is a real image, and outputs l D , 5. The wafer defect detection and positioning method based on the cascade YOLO-GAN according to claim 1 or 4, characterized in that, The defect detection model based on the improved generative adversarial network uses a weighted joint generation loss function; The three loss functions of adversarial loss, content loss and encoding loss are designed, and the joint loss function is constructed by using the weighting method; the adversarial loss function is used to represent L adv The loss generated by the generator and the discriminator in the game process is calculated using the L2 norm loss function, as shown in the following formula: L adv =||f(r)-f(G(r)||2; the content loss L con is used to reflect the difference between the reconstructed image and the real image T, which is calculated using the L1 norm loss function, as shown in the following formula: L con =||T-G(r)||1; the encoding loss L enc is used to reduce the gap between the input image feature vector T and the reconstructed image feature vector , which is calculated using the L2 norm loss function, as shown in the following formula: L enc =||G E (x)-E(G(x))||2, finally, the joint loss function L is generated by using the weighting method, as shown in the following formula: L=ω adv L adv +ω con L con +ω enc L enc , ω adv , ω con , ω enc is the weighting parameter, which is used to adjust the influence of a single loss on the overall objective function.

6. The wafer defect detection and positioning method based on the cascaded YOLO-GAN according to claim 1, characterized in that, When the detection result is defective, the defective area obtained by training the generative adversarial network is enlarged by 2 times according to the width and the like to obtain a wafer defect mask M, M contains the defect area and its spatial context information, and the wafer defect mask M is used to extract the subdivided defect features from the target detection frame, and the process is as follows: D = F O M; wherein O represents point-by-point multiplication, F represents the feature map output by the mobilenet convolution operator, and D represents the wafer defect related target detection frame; a multilayer perceptron MLP is used to encode and reduce the dimension of D to obtain a reduced dimension feature encoding D'; after the reduced dimension feature encoding D' passes through a Softmax classifier, a cross-entropy loss function is used for classification, as follows: k represents the number of subdivided defect types, represents one-hot encoding of the label, d i represents the output of the Softmax classifier.

Images