Wafer detection model training method, wafer detection method, wafer detection system, wafer detection equipment and medium

Abstract

The invention discloses a wafer detection model training method, a wafer detection method, a wafer detection system, wafer detection equipment and a medium, and the method comprises the steps: obtaining a wafer surface defect image set, carrying out the category marking of the wafer surface defect image set, constructing a wafer defect detection data set, constructing an improved RT-DETR model, and carrying out the training of the improved RT-DETR model according to the wafer defect detection data set. Training the to-be-detected image of the wafer until the training is finished to obtain a wafer detection model, inputting the to-be-detected image of the wafer into a pre-trained wafer detection model, outputting a detection result of the to-be-detected image of the wafer, outputting a defect probability and a mark of the to-be-detected image of the wafer, and obtaining the defect probability of the to-be-detected image of the wafer through an improved RT-DERT model. The detection performance on small particles, slender scratches and the like is effectively improved.

Description

Technical Field

[0001] This application belongs to the field of target detection algorithms in computer vision, especially to wafer detection model training methods, detection methods, systems, equipment and media Background Art Wafer surface defects refer to various imperfect or irregular defects that occur during the semiconductor manufacturing process, such as particles, scratches, cracks, etc. These defects may be caused by various factors such as impurities, process parameters, equipment failures, etc. The existence of defects may cause short circuits, leakage, and other problems in electronic devices, which have a direct impact on the performance and reliability of semiconductor devices.

[0002] In order to control and improve the yield of wafer manufacturing, wafer fabs focus on fine defect detection after key processes. However, the speed of fine inspection is too slow, only a few wafers per hour. To improve efficiency, a rapid inspection process is added before fine inspection to quickly detect wafers with obvious surface defects for rework. Currently in the field of wafer surface defect detection, the RT-DETR target detection model has very good detection performance for larger wafer surface defects such as missing corners, missing chips, and ripples. However, wafer surface defects are still common in some very small bad spots and slender scratches. The RT-DETR model is still not very ideal in detecting these tiny targets and slender objects. Summary of the invention

[0003] The purpose of this application is to provide a wafer inspection model training method, inspection method, system, equipment and medium that can solve at least one of the above problems.

[0004] The first aspect of the present application provides a wafer detection model training method, comprising the following steps: acquiring a wafer surface defect image set, classifying the wafer surface defect image set, and constructing a wafer surface defect image dataset; Construct an RT-DETR model, which includes a backbone network, an encoder and a decoder; The ResNet50 network is used as the backbone network, and the last feature layer P5 of the backbone network is replaced with a dynamic snake convolution layer to obtain an improved RT-DETR model; Obtain any wafer surface defect image in the data set, input the wafer surface defect image into the backbone network of the improved RT-DETR model, and obtain the first feature maps at different levels; The first feature map is obtained, and the second feature map is obtained after being extracted by the dynamic snake convolution layer and the encoder in sequence; Performing feature fusion on the first feature map and the second feature map to obtain image features; The image features are input into the decoder to obtain the detection results of wafer surface defects; According to the detection results, the improved RT-DETR model is iteratively trained until the training is completed to obtain the wafer detection model.

[0005] Therefore, the present application has at least the following beneficial effects: by introducing dynamic serpentine convolution, the extracted features can better process input data with complex structures and changing shapes, thereby more effectively extracting the shape information of input wafer features, and improving the feature extraction capability of slender objects. Compared with traditional convolution operations, dynamic serpentine convolution can better capture the details and shape information of slender objects, thereby improving the accuracy and robustness of wafer surface defect detection and improving the performance of wafer detection models.

[0006] According to some embodiments of the present application, the construction of an RT-DETR model, which includes a backbone network, an encoder and a decoder, further includes: Build DAE module and RFFPN module from RT-DETR model; The encoder includes an AIFI module and a CCFM module. The AIFI module is replaced by a DAE module, and the CCFM module in the encoder is replaced by an RFFPN module.

[0007] According to some embodiments of the present application, the first feature map is obtained, and the second feature map is obtained after sequentially extracting the first feature map through a dynamic snake convolution layer and an encoder, including: Obtain the first feature map, and extract it through the dynamic snake convolution layer to obtain a specific feature map; A specific feature map is obtained, and the specific feature map is input into the DAE module to obtain a second feature map.

[0008] According to some embodiments of the present application, obtaining a specific feature map and inputting the specific feature map into the DAE module to obtain a second feature map includes: The DAE module includes a DAttention layer, a Dropout layer, a fully connected layer, and an activation function GELU layer; Get a specific feature map, and use the DAttention layer to calculate the dynamic attention weight of the specific feature map; The dynamic attention weights are randomly inactivated to obtain the first output graph; Perform a residual connection between the specific feature map and the first output map, and normalize them to obtain a second output map; The second output graph is transformed through the fully connected layer, the activation function GELU layer, the Dropout layer, the fully connected layer and the Dropout layer in sequence to obtain the third output graph; The third output map is residually connected and then normalized to obtain the second feature map.

[0009] According to some embodiments of the present application, an image feature map is input into a decoder to obtain detection results of wafer surface defects, including: inputting a second feature map and an image feature map into a decoder, extracting and generating a bounding box and a confidence score through the decoder, and obtaining detection results of wafer surface defects.

[0010] According to some embodiments of the present application, the step of fusing the first feature map and the second feature map to obtain the image feature includes: Obtain a first feature map and a second feature map; The first feature map is expanded through a transposed convolution layer with a 3x3 convolution kernel and then upsampled using bilinear interpolation to obtain a unified feature map with the same dimension as the second feature map; The unified feature map is first multiplied and fused with the second feature map, and then added and fused with the unified feature map to obtain the image features.

[0011] The second aspect of the present application also provides a wafer detection method, comprising: Acquire the image to be inspected on the wafer surface; Inputting the wafer to-be-detected image into a pre-trained wafer detection model, wherein the wafer detection model is obtained by the wafer detection model training method as described above; After the inspection, the inspection result of the wafer image to be inspected is output, and the inspection result includes the surface defect mark and surface defect type of the wafer image to be inspected.

[0012] The third aspect of the present application also provides a wafer detection system, including: An acquisition module is used to acquire the image to be inspected on the wafer surface; A detection module, used for inputting the wafer to-be-detected image into a pre-trained wafer detection model, wherein the wafer detection model is obtained according to the wafer detection model training method as described above; The output module is used to output the detection result of the wafer image to be detected after detection, and the detection result includes the surface defect mark and surface defect type of the wafer image to be detected.

[0013] The fourth aspect embodiment of the present application also provides an electronic device, including a memory and a processor; the memory is used to store a computer program; the processor is used to implement the wafer inspection model training method as described above, or to implement the wafer inspection method as described above, when executing the computer program.

[0014] The fifth aspect embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the wafer inspection model training method as described above is implemented, or the wafer inspection method as described above is implemented.

[0015] Additional aspects and advantages of the present application will be given in part in the following description, and in part will become apparent from the following description, or learned through the practice of the present application. The creative aspects of the training methods, detection methods, systems, devices and media in the field of large models provided in this specification can be fully explained by practicing or using the methods, devices and combinations provided in the following detailed examples. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A schematic diagram of a process flow of a wafer inspection model training method provided in an embodiment of the present application; Figure 2 A schematic diagram of a process flow of a wafer inspection method provided in an embodiment of the present application; Figure 3 An improved RT-DETR model structure diagram of the wafer image detection model training method provided in an embodiment of the present application; Figure 4 A schematic diagram of the feature fusion process provided in an embodiment of the present application; Figure 5 A schematic diagram of the structure of a DAE module provided in an embodiment of the present application; Figure 6 A schematic diagram of the detection results of the wafer detection method provided in the embodiment of the present application; Figure 7 A schematic diagram of a wafer inspection model provided in an embodiment of the present application; Figure 8 A structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0017] The following description provides specific application scenarios and requirements of this specification, with the purpose of enabling those skilled in the art to make and use the contents of this specification. Various local modifications to the disclosed embodiments will be apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments and applications without departing from the spirit and scope of this specification. Therefore, this specification is not limited to the embodiments shown, but to the widest scope consistent with the claims.

[0018] The terms used herein are only used for the purpose of describing specific example embodiments and are not restrictive. For example, unless the context clearly indicates otherwise, as used herein, the singular forms "one", "an" and "the" may also include plural forms. When used in this specification, the terms "include", "comprise" and / or "contain" mean the existence of the associated integers, steps, operations, elements and / or components, but do not exclude the existence of one or more other features, integers, steps, operations, elements, components and / or groups or other features, integers, steps, operations, elements, components and / or groups may be added in the system / method. The terms "first", "second" and the like are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated.

[0019] First, the terms involved in one or more embodiments of the present specification are explained.

[0020] DETR: Detection Transformer, is a deep learning object detection model based on the Transformer architecture. Due to the sequence length limit of the Transformer, DETR may be affected by memory limitations when processing large images and has poor detection effects on small objects. In addition, DETR has a slow inference speed and is not suitable for real-time application scenarios.

[0021] RT-DETR: Real-Time Detection Transformer, a method proposed by Baidu's deep learning team, uses an improved model based on DETR to solve real-time problems and improve the performance and efficiency of the DETR model. From a structural point of view, RT-DETR mainly includes the backbone network, the neck network, and the head network.

[0022] Backbone network: Backbone network usually uses convolutional neural networks such as ResNet to extract feature representations of input images.

[0023] Feature layer: responsible for extracting features, such as extracting feature maps from a certain stage of the backbone network. These features can be low-level such as edges and textures, or high-level such as objects and scenes. In the RT-DETR model, feature layer P5 and feature layers of other scales such as P3 and P4 together form a multi-scale feature pyramid.

[0024] Decoder: Decoder iteratively optimizes the target query through the auxiliary prediction head to generate bounding boxes and confidence scores. The decoder supports flexible reasoning of different layers without retraining.

[0025] Neck network: An efficient hybrid encoder is used to convert multi-scale features into image feature sequences. The efficient hybrid encoder includes: AIFI module and CCFM module.

[0026] AIFI: Attention-based Intra-scale Feature Interaction, the AIFI module uses the attention mechanism to enhance the interaction ability of the neural network when processing multi-scale features, thereby improving the performance of target detection.

[0027] CCFM: Cross-Scale Feature Fusion Module, cross-scale feature fusion module. By fusing features of different scales, CCFM can effectively integrate detail features and contextual information, thereby improving the overall performance of the model.

[0028] ResNet50: A classic deep convolutional neural network that has shown high accuracy and efficiency in image processing and computer vision tasks such as object detection and classification.

[0029] Normalization: A data preprocessing method that scales the data to a certain range, usually [0, 1] or [-1, 1].

[0030] Residual connection: In a neural network, the input of a layer is directly added to the output of that layer to form a residual path, so that the output of that layer is no longer just a nonlinear transformation of the input, but a superposition of the input and the nonlinear transformation of the input.

[0031] The present application is described in detail below through specific embodiments.

[0032] Small targets such as tiny particles on the surface of the wafer usually have only a few pixels and detail information, resulting in insufficient feature representation and difficulty in accurately capturing target information in the feature map. This makes it difficult for the RT-DERT model to distinguish between small targets and backgrounds, thus affecting detection performance. The Backbone of RT-DETR uses a conventional 3*3 convolution kernel, which will be mixed with invalid background information when extracting slender defect information. Therefore, the RT-DETR model is not ideal for detecting slender scratch defects.

[0033] Based on the above, see Figure 5 , the embodiment of the present application provides a wafer image detection model training method, comprising the following steps: The first aspect of the present application provides a wafer inspection model training method, comprising the following steps: S100, obtaining a wafer surface defect image set, classifying the wafer surface defect image set, and constructing a wafer surface defect image dataset; the wafer surface defect image set includes slender scratches and tiny target images; S110, constructing an RT-DETR model, which includes a backbone network, an encoder and a decoder; wherein the encoder includes an AIFI module and a CCFM module; S120, using the ResNet50 network as the backbone network, and replacing the last feature layer P5 of the backbone network with a dynamic snake convolution layer to obtain an improved RT-DETR model; S130, obtaining any wafer surface defect image in the data set, inputting the wafer surface defect image into the backbone network of the improved RT-DETR model, and obtaining first feature maps of different levels; wherein, after the convolution layer operation in the first BasicBlock, the output is the P3 feature map. After the convolution layer operation in the second BasicBlock, the output is the P4 feature map, thereby obtaining first feature maps of different levels.

[0034] S140, obtaining a first feature map, and sequentially extracting the first feature map through a dynamic snake convolution layer and an encoder to obtain a second feature map; S150, performing feature fusion on the first feature map and the second feature map to obtain an image feature map; S160, inputting the image feature map into a decoder to obtain a detection result of wafer surface defects; S170, based on the detection results, iteratively train the improved RT-DETR model until the training is completed to obtain a wafer detection model.

[0035] Specifically, Figure 1 and Figure 3 As shown, a basic RT-DETR model is constructed. This application uses ResNet50 as the backbone, and replaces the last feature layer P5 of the backbone with a block using dynamic snake convolution. Further, a DAE module and an RFFPN module are constructed from the RT-DETR model; the AIFI module in the encoder is replaced by a DAE module, and the CCFM module in the encoder is replaced by an RFFPN module.

[0036] Among them, the dynamic snake convolution is designed based on the idea of ​​deformable convolution kernel. However, unlike the deformable convolution, the deformable convolution will cause roaming of the perception area when processing slender tubular structures, while the dynamic snake convolution takes into account the serpentine morphology of the tubular structure and enhances the perception of the tubular structure through the constrained learning process, thus avoiding this roaming phenomenon by introducing constraints.

[0037] For a standard 3x3 2D convolution kernel, let its coordinates be K and the center coordinates be Ki=(Xi,Yi), and its expression is:

[0038] Dynamic snake convolution is inspired by deformable convolution and introduces deformation offset , and then introduce constraints. Starting from the center coordinate, the position of each convolution grid depends on the position of the previous grid: Ki+1 adds an offset to Ki = {δ|δ ∈ [−1, 1]}, the offsets need to be accumulated to ensure that the convolution kernel conforms to the linear morphology structure.

[0039] Therefore, the coordinate of the dynamic snake convolution kernel on the x-axis is expressed as:

[0040] The coordinates on the y-axis are expressed as:

[0041] Finally, the coordinates are converted into integers through bilinear interpolation. That is, the convolution kernel will adjust the position of the convolution grid of the convolution kernel itself according to the learned feature form, so as to better perceive the key features.

[0042] Therefore, if Figure 3 As shown in the figure, the last feature layer P5 in the backbone is replaced by a dynamic snake convolution layer. By introducing dynamic snake convolution, the extracted features can better process input data with complex structures and changing shapes, thereby more effectively extracting the shape information of the input features and improving the feature extraction capability for slender objects. Compared with traditional convolution operations, dynamic snake convolution can better capture the details and shape information of slender objects, thereby improving the accuracy and robustness of wafer inspection and the performance of the inspection model.

[0043] According to some embodiments of the present application, since the AIFI module will over-focus on the overall semantic information when processing high-level features and ignore the detailed information of small targets, this may lead to the loss of key details in the detection of small targets, affecting the detection accuracy of wafer surface defects. In order to solve this problem, after constructing an RT-DETR model, the present application uses the DAE (Dynamic Attention Encoder) module for sequence conversion, and replaces the AIFI module with the DAE module to improve the ability to extract information from small targets. This module can effectively capture the contextual information in the sequence and extract representative features by relying on the DAttention dynamic attention mechanism.

[0044] According to some embodiments of the present application, a first feature map is obtained, and a second feature map is obtained after sequentially being extracted by a dynamic snake convolution layer and an encoder, including: Obtain the first feature map, and extract it through the dynamic snake convolution layer to obtain a specific feature map; A specific feature map is obtained, and the specific feature map is input into the DAE module to obtain a second feature map.

[0045] According to some embodiments of the present application, Figure 5 The structure of the DAE module is shown, which includes a DAttention layer, a Dropout layer, a fully connected layer, and an activation function GELU layer; the DAE module converts the input sequence into a high-dimensional representation to capture the features and context information of the input sequence. Therefore, the step of obtaining a specific feature map and inputting the specific feature map into the DAE module to obtain a second feature map includes: Get a specific feature map, and use the DAttention layer to calculate the dynamic attention weight. Specifically, DAttention introduces a deformable attention weight calculation method, which allows the attention weight to shift in the spatial dimension, so as to more flexibly adapt to irregular patterns and relationships in the input sequence.

[0046] The dynamic attention weights are randomly inactivated to obtain the first output graph; Perform a residual connection between the specific feature map and the first output map, and normalize them to obtain a second output map; The second output graph is transformed through the fully connected layer, the activation function GELU layer, the Dropout layer, the fully connected layer and the Dropout layer in sequence to obtain the third output graph; The third output map is residually connected and then normalized to obtain the second feature map.

[0047] Specifically, the attention weights are adjusted by introducing additional offset vectors, which can be learned based on the features and position information of the input sequence, and the attention patterns can be adjusted when calculating the attention weights. By introducing the offset vectors, deformable attention can better capture local patterns and long-range dependencies in the input sequence, while alleviating the limitation of self-attention on fixed patterns in the input sequence.

[0048] Therefore, DAttention can enhance the expressiveness and flexibility of the attention mechanism in the encoder module, so that the encoder can better capture the local relations and long-distance dependencies in the input sequence, thereby improving the performance and representation ability of the model. A series of modules after the DAttention layer perform feature conversion, nonlinear transformation and normalization on the output of the DAttention layer through operations such as convolution, activation function, layer normalization and random inactivation. The combination of these modules can extract higher-level and more representative feature representations and enhance the expressiveness, robustness and generalization performance of the model.

[0049] Therefore, by calculating the attention weights, we focus on the features that are particularly important for the current task, and then use these weights to adjust the representation and fusion of the features. In this way, the improved network can adaptively adjust the contribution of features at different levels, so as to better utilize the relationship and expression power between them.

[0050] Since the resolution of small targets in wafer images is usually low, low resolution may cause blurred or missing target boundaries, making detection difficult. Although the original CCFM module can effectively fuse features from different levels and obtain a better balance between semantics and detail information, operations such as upsampling, downsampling, convolution, weighting, etc. in the feature fusion process will inevitably lead to the loss of some detail information. If CCFM is used to perform the above operations on the feature map multiple times, the risk of loss of detail information is further increased. Therefore, in order to solve the problem of large morphological differences between the two target categories of small particles and slender scratches, the RFFPN module is constructed based on the benchmark model RT-DETR, and its feature fusion module CCFM is replaced by a new feature fusion network RFFPN (Residual Fusion Feature Pyramid Network) network.

[0051] According to some embodiments of the present application, the image feature map is input into the decoder to obtain the detection result of the wafer surface defects, which specifically includes: inputting the second feature map and the image feature map into the decoder, extracting and generating the bounding box and the confidence score through the decoder, and obtaining the detection result of the wafer surface defects. Among them, the RFFPN network adopts a more conservative strategy. First, ensure that the feature layer is directly input into the decoder after the feature is extracted by the DAE module to retain the semantic information of the small target to the greatest extent. In addition, the feature fusion method adopted by the RFFPN network is feature map multiplication and feature map addition, that is, see Figure 3 and Figure 4 , the first feature map and the second feature map are subjected to feature fusion to obtain image features, and the fusion process includes the following steps: Obtain a first feature map and a second feature map; The first feature map is expanded through a transposed convolution layer with a 3x3 convolution kernel and then upsampled using bilinear interpolation to obtain a unified feature map with the same dimension as the second feature map; The unified feature map is first multiplied and fused with the second feature map, and then added and fused with the unified feature map to obtain the image features.

[0052] In actual operation, the fusion step is to give a high-level feature converted by the DAE layer and a low-level feature extracted by the feature extraction layer, input the high-level feature And input low-level features , where C is the number of channels, H is the image height, and W is the image width. The high-level features are expanded through a transposed convolution (T-Conv) layer with a 3x3 convolution kernel, and then upsampled by bilinear interpolation to unify the dimensions with the low-level features to obtain , the entire fusion process is shown in the following equations (1) and (2), where T is the transposed convolution and BL is the bilinear interpolation: (1) (2) Among them, feature map multiplication fusion is to fuse two feature maps through element-by-element multiplication operation, and each element of the resulting feature map is the product of the elements of the corresponding position of the input feature map. Feature map addition fusion is to fuse two feature maps through element-by-element addition operation, and each element of the resulting feature map is the sum of the elements of the corresponding position of the input feature map. The multiplication-then-addition fusion method similar to the residual connection can effectively highlight the semantic information and detail information of important features, while also avoiding the information scaling problem that may be caused by performing feature map multiplication alone or the information accumulation problem that may be caused by performing feature map addition alone.

[0053] In this way, the low-level features obtained from other layers are only fused with the high-level features obtained after the DAE module to a certain extent, so as to balance the model's adaptability to two types of targets with large shape differences, namely, thin and long scratches on the wafer and tiny targets, while trying to avoid excessive loss of detail information. Therefore, in this way, the RFFPN network can better balance semantic information and detail information when dealing with the morphological differences of the two target categories of small particles and thin and long scratches on the wafer, so that the model can better perceive and detect small-sized targets. By effectively combining and utilizing multi-scale features, the accuracy and recall rate of small target detection can be improved, thereby improving detection performance.

[0054] The second aspect of the present application also provides a wafer detection method, such as Figure 2 As shown, including: S200, obtaining the image to be inspected on the wafer surface; S210, inputting the wafer image to be inspected into a pre-trained wafer inspection model, where the wafer inspection model is obtained by the above wafer inspection model training method; S220, outputting the detection result of the wafer image to be detected after the detection, the detection result includes the surface defect mark and surface defect type of the wafer image to be detected.

[0055] Specifically, an image to be inspected on the surface of a wafer is obtained; the image to be inspected on the wafer is input into a pre-trained wafer inspection model, and the inspection result of the image to be inspected is output, and the defect probability of the image to be inspected is output, wherein the inspection model is obtained according to the above wafer inspection model training method; the inspection result of the image to be inspected is determined according to the defect probability of the image to be inspected, and the inspection result includes marking the surface defects of the wafer image.

[0056] In summary, see Figure 1 and Figure 3 , the improved model inputs the features extracted by the last dynamic snake convolution layer of the backbone network into the dynamic attention encoder, and then converts the multi-scale features into a series of image features through cross-scale feature fusion. Subsequently, the encoder features embedded with the dynamic attention mechanism are used as the initial object query of the decoder. Finally, the decoder with auxiliary prediction head iteratively optimizes the object query to generate categories and label boxes. Therefore, after model detection, the following is obtained Figure 6 The wafer surface defect identification shown includes scratches on the wafer surface and particle spots on the wafer surface.

[0057] For tiny targets, in order to further improve the resolution of images captured by the camera, a high-resolution camera lens can be used to capture high-resolution images. Optionally, after the image is captured, the high-resolution image is segmented and input into the model for detection. In other words, the wafer image to be inspected is segmented into multiple pictures, and the multiple pictures are respectively input into the model for detection to avoid insufficient video memory due to high-resolution images. After the detection is completed, the image is reconstructed to the original image size.

[0058] In order to verify the reliability and superiority of the wafer detection method, various target detection methods were compared. This application uses AP (Average Precision) and mAP (mean Average Precision) as evaluation criteria. The AP in the table is the average value of the accuracy on the PR curve, which is used to measure the detection accuracy of a single category. mAP is the average of the average accuracy of each category. All evaluation indicators are calculated when the intersection over union (IoU) is equal to 0.5. The defect detection methods available for comparison include the DyRT-DETR, RT-DETR, Faster R-CNN, YOLOv5 and YOLOv8 models proposed in this application. The specific results are shown in Table 1 below: Table 1 Model performance comparison

[0059] Faster R-CNN and RT-DETR both use ResNet50 as the backbone, and the Yolo series uses the official CSPDarknet53 as the backbone. The method used in this application of DyRT-DETR is to add a dynamic snake convolution layer to the backbone of ResNet50.

[0060] From the above indicators, it can be seen that the mAP of the proposed method is 84.5%, which is better than all the comparison models and is 6.3%, 8.6%, 5.6% and 4.8% higher than FasterR-CNN, Yolov5, YOLOv8 and RT-DETR respectively. The results show that the proposed method can detect most of the wafer surface defects and provide accurate location information, meeting the requirements of intelligent manufacturing and high detection accuracy.

[0061] In order to verify the effectiveness of the introduced dynamic snake convolution DSConv and the DAE module and RFFPN module proposed in this application, this application also conducted an ablation experiment on this method. The specific experimental results are shown in Table 2 below: Table 2 Overall model ablation experiment

[0062] As shown in Table 2, the model using the DAE module improves the performance of detecting particles by 2.7% and the performance of detecting scratches by 1.9% compared to the basic model RT-DETR; The RT-DETR+DySnake model using the Dysnake convolutional layer has only a 0.2% improvement in particle detection performance compared to the basic model RT-DETR, which is almost no improvement, while the performance in scratch detection has increased by 4.7%, which is a very good effect; the RT-DETR+DAE+DySnake model that uses both the DAE module and the Dysnake convolutional layer has improved particle detection performance compared to the basic model, but is worse than the model that uses the DAE alone, and its scratch detection performance is almost the same as the model that uses the Dysnake alone.

[0063] The method of the present application not only uses the DAE module and the Dysnake convolutional layer, but also uses the RFFPN network. The detection performance on wafer particles is improved by 3.5% compared with the RT-DETR+DAE+DySnake model and 4.1% compared with the RT-DETR baseline model. The detection performance on scratches is still improved by 0.6% compared with the RT-DETR+DAE+DySnake model.

[0064] In summary, the DAE module can help the model better capture the global and local information of the input data, thereby improving the accuracy of target detection. The Dysnake convolution layer has a very good detection effect improvement for scratch targets. The RFFPN module has a positive effect when combining the DAE and Dysnake modules, which can promote the two modules to play their respective roles without causing performance degradation.

[0065] In addition, an ablation experiment was conducted to investigate the effect of the position of the DySnake convolutional feature layer in Backbone on feature extraction, that is, the last three feature layers of ResNet50 were replaced with DySnake convolutional layers one by one. P3, P4, and P5 represent feature layers scaled to 1 / 8, 1 / 16, and 1 / 32 of the input image size, respectively. The experimental results are shown in Table 3 below: Table 3 DySnake ablation experiment

[0066] As shown in Table 3, for particle targets, replacing P3 will slightly affect the detection performance of small targets, while replacing other positions will have little effect. For scratch targets, when replacing a single layer, replacing the P5 feature layer has the best effect. When replacing multiple layers, such as replacing P3, P4, and P5 at the same time, it will bring a slight improvement in detection performance, but compared with replacing the P5 layer alone, it only improves AP by 0.3%, but brings an additional computational cost of up to 9.4GFLOPs, which is not cost-effective. Therefore, the model proposed in this application only uses the DySnake convolution feature layer to enhance the Backbone.

[0067] The third aspect of the present application also provides a wafer detection system, see Figure 7 , including: an acquisition module 200, a detection module 210 and an output module 220.

[0068] Specifically, the acquisition module 200 is used to acquire the wafer surface image to be inspected; the detection module 210 is used to input the wafer image to be inspected into a pre-trained wafer detection model, and the wafer detection model is obtained according to the above wafer detection model training method; the output module 220 is used to output the detection result of the wafer image to be inspected after detection, and the detection result includes the surface defect mark and surface defect type of the wafer image to be inspected.

[0069] The fourth aspect embodiment of the present application also provides an electronic device, including a memory and a processor; the memory is used to store a computer program; the processor is used to implement the above-mentioned wafer inspection model training method, or implement the above-mentioned wafer inspection method when executing the computer program.

[0070] Figure 8 3 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. The electronic device 300 may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 310 (for example, one or more processors) and a memory 320, and one or more storage media 330 (for example, one or more mass storage devices) storing application programs 333 or data 332. Among them, the memory 320 and the storage medium 330 may be short-term storage or permanent storage. The program stored in the storage medium 330 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the electronic device 300. Furthermore, the processor 310 may be configured to communicate with the storage medium 330, and execute a series of instruction operations in the storage medium 330 on the electronic device 300 to implement the wafer detection model training method as described above, or to implement the wafer detection method as described above.

[0071] The electronic device 300 may also include one or more power supplies 340, one or more wired or wireless network interfaces 350, one or more input and output interfaces 360, and / or one or more operating systems 331, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. It will be appreciated by those skilled in the art that Figure 8 The structure of the electronic device shown does not constitute a limitation on the components, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0072] The fifth aspect embodiment of the present application also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium, or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions are executed on a computer, the computer executes the wafer inspection model training method as described above, or implements the steps of the wafer inspection method as described above.

[0073] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described system, device, or unit can refer to the corresponding process in the aforementioned method embodiment and will not be repeated here.

[0074] The above is a description of a specific embodiment of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require a specific order or a continuous order to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0075] In addition, certain terms in this specification have been used to describe embodiments of this specification. For example, "one embodiment", "an embodiment" and / or "some embodiments" mean that a particular feature, structure or characteristic described in conjunction with the embodiment may be included in at least one embodiment of this specification. Therefore, it can be emphasized and should be understood that two or more references to "an embodiment" or "one embodiment" or "an alternative embodiment" in various parts of this specification do not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics may be appropriately combined in one or more embodiments of this specification.

[0076] Finally, it should be understood that the embodiments of the application disclosed herein are explanations of the principles of the embodiments of this specification. Other modified embodiments are also within the scope of this specification. Therefore, the embodiments disclosed in this specification are only used as examples and not as limitations. Those skilled in the art can adopt alternative configurations according to the embodiments in this specification to implement the application in this specification. The embodiments of this specification are not limited to the embodiments accurately described in the application.

Claims

1. A wafer inspection model training method, characterized in that: The following steps are involved: Acquire a wafer surface defect image set, perform category labeling on the wafer surface defect image set, and construct a wafer surface defect image dataset; Construct an RT-DETR model, which includes a backbone network, an encoder and a decoder; The ResNet50 network is used as the backbone network, and the last feature layer P5 of the backbone network is replaced with a dynamic snake convolution layer to obtain an improved RT-DETR model; Obtain any wafer surface defect image in the data set, input the wafer surface defect image into the backbone network of the improved RT-DETR model, and obtain the first feature maps at different levels; The first feature map is obtained, and the second feature map is obtained after being extracted by the dynamic snake convolution layer and the encoder in sequence; Performing feature fusion on the first feature map and the second feature map to obtain an image feature map; The image feature map is input into the decoder to obtain the detection result of the wafer surface defects; According to the detection results, the improved RT-DETR model is iteratively trained until the training is completed to obtain the wafer detection model.

2. The wafer inspection model training method according to claim 1, characterized in that: The RT-DETR model is constructed, which includes a backbone network, an encoder and a decoder, and further includes: Build DAE module and RFFPN module from RT-DETR model; The encoder includes an AIFI module and a CCFM module. The AIFI module is replaced by a DAE module, and the CCFM module in the encoder is replaced by an RFFPN module.

3. The wafer inspection model training method according to claim 2, characterized in that: The first feature map is obtained, and the second feature map is obtained after being extracted by the dynamic snake convolution layer and the encoder in sequence, including: Obtain the first feature map, and extract it through the dynamic snake convolution layer to obtain a specific feature map; A specific feature map is obtained, and the specific feature map is input into a DAE module in the encoder to obtain a second feature map.

4. The wafer inspection model training method according to claim 3, characterized in that: The obtaining of a specific feature map and inputting the specific feature map into a DAE module in an encoder to obtain a second feature map includes: The DAE module includes a DAttention layer, a Dropout layer, a fully connected layer, and an activation function GELU layer; Get a specific feature map, and use the DAttention layer to calculate the dynamic attention weight of the specific feature map; The dynamic attention weights are randomly inactivated to obtain the first output graph; Perform a residual connection between the specific feature map and the first output map, and normalize them to obtain a second output map; The second output graph is transformed through the fully connected layer, the activation function GELU layer, the Dropout layer, the fully connected layer and the Dropout layer in sequence to obtain the third output graph; The third output map is residually connected and normalized to obtain the second feature map.

5. The wafer inspection model training method according to claim 3, characterized in that: The image feature map is input into the decoder to obtain the detection results of wafer surface defects, including: The second feature map and the image feature map are input into the decoder, and the decoder extracts and generates a bounding box and a confidence score to obtain the detection result of the wafer surface defects.

6. The wafer inspection model training method according to claim 2, characterized in that: The step of fusing the first feature map and the second feature map to obtain the image feature includes: Obtain a first feature map and a second feature map; The first feature map is expanded through a transposed convolution layer with a 3x3 convolution kernel and then upsampled using bilinear interpolation to obtain a unified feature map with the same dimension as the second feature map; The unified feature map is first multiplied and fused with the second feature map, and then added and fused with the unified feature map to obtain the image features.

7. A wafer inspection method, characterized in that: include: Acquire the image to be inspected on the wafer surface; Inputting the wafer to-be-detected image into a pre-trained wafer detection model, wherein the wafer detection model is obtained according to the wafer detection model training method according to any one of claims 1 to 6; After the inspection, the inspection result of the wafer image to be inspected is output, and the inspection result includes the surface defect mark and surface defect type of the wafer image to be inspected.

8. A wafer inspection system, characterized in that: include: An acquisition module is used to acquire the image to be inspected on the wafer surface; A detection module, used for inputting the wafer to-be-detected image into a pre-trained wafer detection model, wherein the wafer detection model is obtained according to the wafer detection model training method according to any one of claims 1 to 6; The output module is used to output the detection result of the wafer image to be detected after detection, and the detection result includes the surface defect mark and surface defect type of the wafer image to be detected.

9. An electronic device, characterized in that It comprises a memory and a processor; the memory is used to store a computer program; the processor is used to implement the wafer detection model training method as described in any one of claims 1 to 6, or implement the wafer detection method as described in claim 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that The storage medium stores a computer program, and when the computer program is executed by the processor, it implements the wafer inspection model training method as described in any one of claims 1 to 6, or implements the wafer inspection method as described in claim 7.

Images