A chip surface defect target detection network based on meta learning and a training method and application thereof

Abstract

The application discloses a chip surface defect target detection network based on meta learning and a training method and application thereof, and constructs a chip surface defect target detection network supporting meta learning, which comprises an encoder, a decoder and a detection head connected in sequence; the encoder adopts a Resnet network, the Resnet network comprises multiple layers to output feature maps of different sizes; the decoder is composed of multiple multi-scale feature fusion modules, performs multi-scale feature fusion on the feature maps to obtain a prediction feature map used for predicting a target position, and the detection head can realize regression prediction on center point position information and classification information and regression prediction on the distance between the center point and a target prediction box; and the trained chip surface defect target detection network structure can be used for detecting chip surface defects.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of chip surface defect target detection, and particularly relates to a chip surface defect target detection network based on meta-learning and a training method and application thereof. BACKGROUND

[0002] With the rapid development of the semiconductor industry, quality control in the chip manufacturing process is particularly important. Small defects on the surface of the chip can seriously affect its performance and service life, and therefore, accurate and efficient defect detection is a key link to ensure the quality of the chip.

[0003] At present, traditional detection methods mainly rely on regularized algorithms and manual detection, and have problems such as low detection efficiency, high cost, and being easily affected by human factors. With the progress of deep learning and computer vision technology, target detection methods based on convolutional neural networks (CNN) have gradually been applied to the automated detection of chip surface defects. However, due to the small sample size and complexity of chip surface defects, traditional deep learning models often show insufficient generalization ability when dealing with small sample problems, making it difficult to effectively detect multiple types of defects.

[0004] In view of the deficiencies in the prior art, there is an urgent need for a target detection method specifically for the characteristics of chip surface defects, which can quickly learn from a small amount of samples and effectively detect them to achieve efficient and accurate automated defect detection. SUMMARY

[0005] To solve the deficiencies in the prior art, the present application proposes a chip surface defect target detection network based on meta-learning and a training method and application thereof. In the scenario where chip surface defects are difficult to obtain, only a small amount of pictures and labeled data are needed for the defects to be detected, and the chip surface defect detection task can be completed without retraining, realizing accurate detection of chip surface defects under small sample conditions.

[0006] The technical solutions adopted by the application are as follows:

[0007] A training method for a chip surface defect target detection network based on meta-learning, comprising the following steps:

[0008] Step 1, a chip surface defect target detection network supporting meta-learning is constructed, the chip surface defect target detection network comprising an encoder, a decoder and a detection head connected in sequence; wherein the encoder adopts a Resnet network, the input of the Resnet network being a meta-support set and a meta-query set composed of pictures; the Resnet network comprises a plurality of Layers, each Layer outputting a feature map of different size;

[0009] The decoder is composed of a plurality of multi-scale feature fusion modules, the multi-scale feature fusion modules receive feature maps output by the encoder and perform multi-scale feature fusion to obtain a prediction feature map for predicting a target position, and the multi-scale feature fusion modules comprise a DGM module, a CEM module and a DAnA module.

[0010] The detection head comprises a CRD module, a MAD module, a center point regression prediction module and a point frame distance regression prediction module, wherein the CRD module performs feature enhancement on the prediction feature map output by the decoder, and then the center point regression prediction module performs regression prediction on the center point position information and the classification information;

[0011] The MAD module performs multi-scale feature transformation and depth convolution operation on the feature-enhanced feature map to enhance the spatial distance information in the feature map; and then the point frame distance regression prediction module performs regression prediction on the distance between the center point and the target prediction frame.

[0012] Step 2: Based on the chip surface defect target detection network structure built in step 1, a chip surface defect picture is obtained to construct a meta support set and a meta query set, and the meta support set and the meta query set are used to train the chip surface defect target detection network structure.

[0013] Further, the Resnet network comprises four sequentially connected Layers, which are respectively denoted as Layer1, Layer2, Layer3 and Layer4; the number of residual blocks contained in each Layer is different, and a feature map of a corresponding size is output after convolution processing of each Layer.

[0014] Further, the residual block is composed of a 1x1 convolution layer, a 3x3 convolution layer and a 1x1 convolution layer connected in sequence.

[0015] Further, the connection between each multi-scale feature fusion module in the decoder is as follows:

[0016] The feature maps after Layer2 and Layer3 in the encoder are spliced, and feature enhancement is performed by the first CRD module; the feature map output by the first CRD module is input into the CEM module together with the corresponding label heat map Support-label of the meta support set picture, to obtain a feature map containing contrast-enhanced features Figure 1 .

[0017] The feature maps after Layer2 and Layer3 in the encoder are spliced, and feature enhancement is performed by the second CRD module, to obtain a feature Figure 2 .

[0018] The feature map obtained by the meta-support set picture through Layer 3, the feature map obtained by the meta-query set picture through Layer 3 and the label heat map Support-label corresponding to the meta-support set picture are input into the DGM module, multi-scale difference fusion is performed, a feature map containing rich feature information is obtained, and then the feature map is input into the EDAM module to output features Figure 3 ;

[0019] The two feature maps obtained by the meta-support set picture through Layer 3 and Layer 4 in the encoder respectively, the two feature maps obtained by the meta-query set picture through Layer 3 and Layer 4 in the encoder respectively Figure 1 are input into the DAnA module to obtain a feature map containing multi-scale feature information Figure 4 ;

[0020] The feature Figure 1 , the feature Figure 2 , the feature Figure 3 and the feature Figure 4 are input into the Aggregation module for splicing to obtain a prediction feature map for predicting a target position.

[0021] Further, the DGM module is composed of multiple functional modules. First, two inputs enter the network through different paths, one of which is processed by feature differentiation, irrelevant features are eliminated through subtraction operation, and then combined with the features generated by the multi-scale feature fusion module MSFF, and passed through the Sigmoid activation function to limit the feature value between 0 and 1, which is used to adjust the feature weight; the support set label is also introduced, which is combined with the input features to strengthen the feature representation in few-shot learning; the generated weight is applied to the original input feature through the feature weight correlation module CWCB for weighting operation, and finally the high-level features are extracted through the convolution layer.

[0022] Further, the DAnA module uses the double-branch attention mechanism BA and the channel interaction spatial attention CISA to process and interact the features through multiple input sources; first, the input features are adaptively enhanced by the DAnA module, and the optimized features are output, which are refined and enhanced by the boundary information in BA, and the boundary optimized features are output, and the global context fusion is performed in CISA, the feature expression is enhanced by combining the global and local context information, and the rich information after adaptive enhancement, boundary optimization and global context fusion is contained.

[0023] Further, the CEM module combines the heat map and the original input, and the features are first input into the CAM channel attention module, the input features are weighted and processed through the channel attention mechanism, and the expression of important features is enhanced;The processed features are divided into two branches: one branch directly outputs the CAM processed features, and the other branch further extracts local and compressed features through 3x3 and 1x1 convolution operations;Finally, the features of the two branches are merged in the fusion module, and are output after being processed by the MLP multilayer perceptron.

[0024] Further, in the training process of the chip surface defect target detection network, the loss is calculated based on the results output by the detection head, and the weights of the encoder and the decoder are updated through back propagation.

[0025] A chip surface defect target detection network based on meta-learning is trained by the above method.

[0026] A chip surface defect target detection method, comprising the following steps:

[0027] S1, obtaining the RGB picture of the chip surface defect, and constructing the meta-support set and the meta-query set;

[0028] S2, in the preprocessing stage of the network, the picture size in the meta-support set and the meta-query set is unified;

[0029] S3, inputting the RGB picture of the chip surface defect processed in S2 into the encoder for feature extraction to generate a feature map;

[0030] S4, inputting the feature map into the decoder, and performing feature fusion through each multi-scale feature fusion module in the decoder to obtain a prediction feature map capable of predicting the target position;

[0031] S5, inputting the prediction feature map generated in the decoder into the detection head, and obtaining the center point position, point frame interval and category prediction result of the prediction target through the center point regression prediction module and the point frame interval regression prediction module;

[0032] S6, according to the prediction result, the non-maximum suppression is carried out and the prediction frame information is drawn, and the result is restored to the original picture to obtain the category and the prediction frame of the chip surface defect in the original image, and the prediction process of the network is completed.

[0033] The beneficial effects of the present application are:

[0034] 1. In order to solve the problem of insufficient generalization ability of the prior art in processing small sample problems, the feature fusion network structure is optimized when constructing the chip surface defect target detection network, only a small amount of pictures and labeled data are needed, and the chip surface defect detection task can be completed without retraining, realizing accurate detection of chip surface defects under small sample conditions.

[0035] 2. The present application combines meta-learning with target detection network, obtains a detector capable of efficiently identifying chip defects by using the method of meta-learning while training the target detection network to obtain the best loss function and model weight, and can still effectively identify defects under the condition of scarce chip defect data, significantly improving the detection ability under the condition of limited data.

[0036] 3. The DGM, DAnA and CEM modules are improved and used organically to form an efficient small sample learning feature processing framework. First, the DGM module effectively eliminates irrelevant features, strengthens the feature representation ability in small sample learning, and can extract more rich high-level features under different scale changes. Second, the DAnA module further refines and enhances the boundary information processing, uses the double-branch attention mechanism and channel interaction spatial attention mechanism to finely process and interact the features, combines the global and local context information, and significantly improves the accuracy and stability of feature expression. Finally, through the CEM module, the local features are extracted and compressed under the support of the attention mechanism, so that the local information is more detailed and complete, providing rich detail support for the fusion of global features. Through the synergistic effect of the three modules, the present application not only improves the recognition degree of the features, but also enhances the generalization ability of the model under the condition of small sample, effectively coping with the chip defect detection task in complex scenes. BRIEF DESCRIPTION OF DRAWINGS

[0037] Figure 1 is the overall structure diagram of the chip surface defect target detection network of the present application.

[0038] Figure 2 is the overall network structure diagram of the present application.

[0039] Figure 3 is the network structure diagram of the encoder of the present application.

[0040] Figure 4 is the DGM module diagram involved in the present application.

[0041] Figure 5 is the DAnA module diagram involved in the present application.

[0042] Figure 6 is the CEM module diagram involved in the present application.

[0043] Figure 7 is an improved MAD module diagram to which the present application relates.

[0044] Figure 8 is a center point regression prediction diagram to which the present application relates. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application will be further described in detail below in combination with the drawings and examples. It should be understood that the specific examples described herein are only used to explain the present application and do not limit the present application.

[0046] Example 1

[0047] A chip surface defect target detection network training method based on meta-learning includes the following steps:

[0048] Step 1, build a chip surface defect target detection network supporting meta-learning, the network structure is as shown in Figure 1 、 2 , including an encoder, a decoder and a detection head connected in turn; the network structure of each part is as follows:

[0049] 1. Encoder

[0050] In combination with the drawings Figure 2 , the encoder adopts a backbone feature extraction network, the backbone feature extraction network adopts a Resnet network, and the Resnet network is used for feature extraction on the input picture to generate a feature map.

[0051] As shown in Figure 3 , the Resnet network includes four Layers connected in turn, which are respectively Layer1, Layer2, Layer3 and Layer4, each Layer is composed of a plurality of residual blocks, so the output of each Layer is a feature map of different sizes.

[0052] The input of the Resnet network is a meta-support set and a meta-query set, which can be divided into two paths according to the input object, the first path is that the meta-support set passes through Layer2, Layer3 and Layer4 in turn; wherein the output of Layer2 and the output of Layer3 are spliced (C processing symbol in the figure) and then input into the decoder, and the output of Layer3 and Layer4 is also directly input into the decoder;

[0053] The second path is that the meta query set sequentially passes through Layer2, Layer3 and Layer4; similar to the first path, wherein the output of Layer2 and the output of Layer3 are spliced (C processing symbol in the figure) and then input into the decoder, and the outputs of Layer3 and Layer4 are directly input into the decoder.

[0054] More specifically, as shown in the Resnet network structure, each Layer (layer) is composed of multiple residual blocks, specifically as follows: Figure 3

[0055] Layer1 includes 3 residual blocks, each of which first maintains the channel number to 64 through a 1x1 convolutional layer, then extracts features through a 3x3 convolutional layer, and finally restores the channel number to 256 through another 1x1 convolutional layer.

[0056] Layer2 includes 4 residual blocks, and the convolutional layer in each residual block increases the input channel number from 256 to 512. Through the downsampling operation in the first residual block, the size of the feature map is halved;

[0057] Layer3 includes 6 residual blocks, and the convolutional layer of the residual block will increase the input channel number from 512 to 1024, and will add the input and the output through a jump connection;

[0058] Layer4 includes 3 residual blocks, and the convolutional layer of each residual block will increase the channel number from 1024 to 2048.

[0059] 2. Decoder

[0060] The decoder is composed of multiple multi-scale feature fusion modules, and three kinds of multi-scale feature fusion modules are designed in this example. The prediction feature map capable of predicting the target position is obtained by using the multi-scale feature fusion module, and the multi-scale feature fusion module includes a DGM module, a CEM module and a DAnA module.

[0061] The connection between each multi-scale feature fusion module can be seen from the accompanying Figure 2 .

[0062] The feature maps after the meta support set picture (Support) passes through Layer2 and Layer3 in the encoder are spliced and enhanced through the first CRD module; the feature map output by the first CRD module is input into the CEM module together with the meta support set picture corresponding label heat map (Support-label), to obtain a feature map containing contrast-enhanced features Figure 1 .

[0063] ​The feature maps of the meta-query set picture after Layer2 and Layer3 are spliced, and the feature enhancement is performed through the second CRD module to obtain the feature Figure 2 ;

[0064] The feature map obtained by the meta-support set picture after Layer3, the feature map obtained by the meta-query set picture after Layer3, and the support-label corresponding to the meta-support set picture are input into the DGM module, multi-scale difference fusion is performed, a feature map containing rich feature information is obtained, and then the feature map is input into the EDAM module to output the feature Figure 3 ;

[0065] Two feature maps obtained by the meta-support set picture after Layer3 and Layer4 in the encoder, two feature maps obtained by the meta-query set picture after Layer3 and Layer4 in the encoder Figure 1 are input into the DAnA module to obtain a feature Figure 4 containing multi-scale feature information.

[0066] Finally, the above four features Figure 1 , the feature Figure 2 , the feature Figure 3 and the feature Figure 4 are input into the Aggregation module for splicing to obtain a prediction feature map for predicting the target position.

[0067] More specifically, as shown in Figure 4 , the DGM module is composed of multiple functional modules. First, two inputs enter the network through different paths, one of which is processed by feature differentiation, irrelevant features are eliminated through subtraction operation, and then combined with the features generated by the multi-scale feature fusion module (MSFF), and passed through the Sigmoid activation function to limit the feature value between 0 and 1, which is used to adjust the feature weight. The support set label is also introduced, which is combined with the input feature to further strengthen the feature representation in the few-shot learning. The generated weight is applied to the original input feature through the feature weight correlation module (CWCB) for weighting operation, and finally the high-level features are extracted through the convolution layer.

[0068] More specifically, as shown in Figure 5As shown, the DAnA module uses a double-branch attention mechanism (BA) and a channel interaction spatial attention (CISA) to process and interact features from multiple input sources. First, the input features are adaptively enhanced by the DAnA, and the optimized features are output. The boundary information in the BA is used to refine and enhance the features, and the output contains boundary-optimized features. In the CISA, global context fusion is performed, and the feature expression is enhanced by combining global and local context information. The rich information obtained through adaptive enhancement, boundary optimization, and global context fusion can be used for subsequent detection or classification tasks, improving the efficiency of feature information expression and transmission, and ultimately providing the model with stronger feature discrimination ability.

[0069] More specifically, as shown in FIG. 4, the CEM module combines the heat map and the original input. The features are first input into the CAM (channel attention module) and are weighted by the channel attention mechanism to enhance the expression of important features. The processed features are divided into two branches: one branch directly outputs the CAM-processed features, and the other branch further extracts local and compressed features through 3x3 and 1x1 convolution operations. Finally, the features of the two branches are merged in the fusion module and output after MLP (multi-layer perceptron) processing, which can be used for classification or other tasks. At the same time, on the convolution branch, feedback adjustment for training is calculated through the loss function. The fusion operation enhances feature expression. It can effectively improve the contrast of input features and thus enhance the recognition ability of the model. Figure 6

[0070] 3. Detection head

[0071] The detection head includes a CRD module, a MAD module, a center point regression prediction module, and a point-box distance regression prediction module. The CRD module enhances the features of the prediction feature map output by the decoder and transmits the enhanced feature map to the center point regression prediction module for regression prediction of the center point position information and classification information.

[0072] The enhanced feature map is also input into the MAD module for multi-scale feature transformation and deep convolution operation to enhance the spatial distance information in the feature map. The result is input into the point-box distance regression prediction module for regression prediction of the distance between the center point and the target prediction box.

[0073] More specifically, the MAD module is an improved multi-scale aggregation decoding module, and its structure is as shown in FIG. 5. Figure 7 ​As shown, the feature map undergoes two parallel processing paths: one path includes weighted convolution (WTConv), ReLU activation function, and Dropout operation. First, WTConv extracts local features, followed by ReLU to increase non-linear expressiveness, and finally Dropout to prevent overfitting. The other path directly passes the input data, providing additional skip connections to help retain more original information. The results of these two paths are then fused at a merging node. Afterwards, it passes through a CRD for further processing, undergoes feature reorganization via an FRM module, and is fused with the output from CBAM, utilizing an attention mechanism to enhance focus on important features. After fusion, the feature map is processed again by FRM and CSH before the output is given.

[0074] In this embodiment, the DSConv module is replaced with the WTConv module, which retains high feature extraction capability, enhances the expressive power of convolution operations, and improves the model's performance when handling various heterogeneous and non-uniform features. WTConv allows different weights to be assigned to the convolution kernel, enabling the model to handle the weight distribution between features more flexibly, especially when extracting more detailed and complex local features.

[0075] In this embodiment, after obtaining the center point regression prediction result and the bounding box spacing prediction result, the keypoint with the highest confidence score is extracted from the input prediction result, obtaining four pieces of information: prediction score, target index, target category, and center point coordinates. Then, the center point coordinates are corrected, and the width and height of the object at the corresponding position are obtained from the width and height feature map. Next, the bounding box of each target is calculated using this information, which consists of the coordinates of the upper left and lower right corners. Finally, the bounding boxes, scores, and category information are integrated into the final detection result, which is returned as output for target detection. The entire process completes the target box prediction by transforming the heatmap and other feature maps.

[0076] Step 2: Based on the chip surface defect target detection network structure built in Step 1, use it for model training.

[0077] Step 2.1: Obtain RGB images of chip surface defects and construct meta-support sets and meta-query sets;

[0078] Step 2.2: Train the chip surface defect target detection network structure built in Step 1 using the meta-support set and meta-query set. During training, loss is calculated based on the results output by the detection head, and backpropagation is used to update the weights of the encoder and decoder. The specific process is as follows:

[0079] In the center point regression prediction module, all center points of each category are directly predicted, all center points are cropped onto the heat map using the Gaussian kernel, and the method for calculating the center point calculation loss is:

[0080]

[0081] where N is the number of center points, Y c is a Gaussian kernel function, represents the detected center point, is the detected background, and a and β are hyperparameters for controlling the contribution of each center point.

[0082] Then, in order to more accurately obtain the center point position and eliminate the influence of the discretization error, center point offset prediction is added after center point prediction, and the L1 loss is used to calculate the center point offset calculation loss, and the specific calculation method is:

[0083]

[0084] where N is the number of center points, p k is the offset true value of the kth center point, is the offset predicted value of the kth center point.

[0085] The method for obtaining the point frame distance regression loss using L1 loss is:

[0086]

[0087] where N is the number of center points, s k is the point frame distance true value of the kth center point, is the point frame distance predicted value of the kth center point.

[0088] The loss function includes center point regression loss, center point offset regression loss, and point frame distance regression loss, and is represented as:

[0089] L all =λ k L k +λ reg L reg +λ wh L wh

[0090] where λ k , λ reg and λ wh represent the weights of the center point prediction loss, the center point offset prediction loss and the point frame distance prediction loss, respectively, and L all represents the total loss of the prediction. After calculating the total loss, the network can be back propagated and the weights can be updated to complete the network training.

[0091] Embodiment 2

[0092] A meta-learning-based chip surface defect target detection network trained by the method of Embodiment 1 is adopted.

[0093] Embodiment 3

[0094] Based on the above meta-learning-based chip surface defect target detection network, the application further proposes a chip surface defect target detection method, and the specific process is as follows:

[0095] S1, an RGB picture of a chip surface defect is obtained, and a meta-support set and a meta-query set are constructed;

[0096] S2, in the preprocessing stage of the network, the RGB pictures of the chip surface defects in the meta-support set and the meta-query set are uniformly scaled to 256x256 size;

[0097] S3, the RGB pictures of the chip surface defects processed in S2 are input into the encoder for feature extraction to generate a feature map;

[0098] S4, the feature map is input into the decoder, and the feature fusion is performed through each multi-scale feature fusion module in the decoder to obtain a prediction feature map capable of predicting the target position;

[0099] S5, the prediction feature map generated in the decoder is input into the detection head, and the center point regression prediction module and the point frame interval regression prediction module are used to obtain the center point position, the point frame interval and the category prediction result of the prediction target;

[0100] S6, according to the prediction result, the non-maximum suppression is performed and the prediction box information is drawn, and the result is restored to the original 256x256 picture to obtain the category of the chip surface defect and the prediction box in the original image as shown in Figure 8 , and the prediction process of the network is completed.

[0101] The above embodiments are only used to illustrate the design idea and characteristics of the present application, and the purpose is to enable those skilled in the art to understand the content of the present application and implement it, and the protection scope of the present application is not limited to the above embodiments. Therefore, any equivalent changes or modifications made according to the principles and design ideas disclosed by the present application are within the protection scope of the present application.

[0102] The above embodiments are only used to illustrate the design idea and characteristics of the present application, and the purpose is to enable those skilled in the art to understand the content of the present application and implement it, and the protection scope of the present application is not limited to the above embodiments. Therefore, any equivalent changes or modifications made according to the principles and design ideas disclosed by the present application are within the protection scope of the present application.

Claims

1. A method for training a meta-learning-based chip surface defect target detection network, characterized in that, The method comprises the following steps: Step 1, constructing a chip surface defect target detection network supporting meta-learning, the chip surface defect target detection network comprising an encoder, a decoder and a detection head connected in sequence; wherein the encoder adopts a Resnet network, and an input of the Resnet network is a meta-support set and a meta-query set composed of pictures; the Resnet network comprises a plurality of layers, and each layer outputs a feature map of different size; the decoder is composed of a plurality of multi-scale feature fusion modules, the multi-scale feature fusion modules receiving a feature map output by the encoder and performing multi-scale feature fusion to obtain a prediction feature map used for predicting a target position, and the multi-scale feature fusion modules comprising a DGM module, a CEM module and a DAnA module; the detection head comprising a CRD module, a MAD module, a center point regression prediction module and a point frame distance regression prediction module, wherein the CRD module performs feature enhancement on the prediction feature map output by the decoder, and then the center point regression prediction module performs regression prediction on center point position information and classification information; the MAD module performs multi-scale feature transformation and depth convolution operation on the feature-enhanced feature map to enhance spatial distance information in the feature map; and then the point frame distance regression prediction module performs regression prediction on the distance between the center point and the target prediction frame; Step 2, based on the chip surface defect target detection network structure constructed in step 1, obtaining chip surface defect picture to construct a meta-support set and a meta-query set, and training the chip surface defect target detection network structure by using the meta-support set and the meta-query set.

2. The method of claim 1, wherein the method further comprises: The Resnet network comprises four layers connected in sequence, which are respectively denoted as Layer1, Layer2, Layer3 and Layer4; the number of residual blocks contained in each layer is different, and a feature map of corresponding size is output after convolution processing of each layer.

3. The method of claim 2, wherein the method further comprises: The residual block is composed of a 1x1 convolution layer, a 3x3 convolution layer and a 1x1 convolution layer connected in sequence.

4. The method of claim 1, wherein, The connection between each multi-scale feature fusion module in the decoder is as follows: the feature maps after Layer2 and Layer3 in the encoder are spliced, and feature enhancement is performed through the first CRD module; the feature map output by the first CRD module is input into the CEM module together with the corresponding label heat map Support-label of the meta-support set picture, to obtain a feature map 1 containing contrast-enhanced features; the feature maps after Layer2 and Layer3 in the encoder are spliced, and feature enhancement is performed through the second CRD module, to obtain a feature map 2; the feature map obtained by Layer3 of the meta-support set picture and the feature map obtained by Layer3 of the meta-query set picture are input into the DGM module together with the corresponding label heat map Support-label of the meta-support set picture, to perform multi-scale difference fusion and obtain a feature map containing rich feature information, and then the feature map is input into the EDAM module to output a feature map 3; The two feature maps obtained by the meta-support set picture respectively passing through Layer 3 and Layer 4 in the encoder, and the two feature maps obtained by the meta-query set picture respectively passing through Layer 3 and Layer 4 in the encoder are input into the DAnA module to obtain a feature map 4 containing multi-scale feature information; The feature map 1, the feature map 2, the feature map 3 and the feature map 4 are input into the Aggregation module for splicing to obtain a prediction feature map used for predicting a target position.

5. The method of claim 4, wherein, The DGM module is composed of multiple functional modules. First, two inputs enter the network through different paths, one of which is subjected to feature differentiation processing, irrelevant features are eliminated through subtraction operation, and then the features generated by the multi-scale feature fusion module MSFF are combined, and the features pass through the Sigmoid activation function to limit the feature values between 0 and 1, which is used to adjust the feature weight; The support set label is also introduced, which is combined with the input features to strengthen the feature representation in few-shot learning; the generated weight is applied to the original input features through the feature weight correlation module CWCB for weighting operation, and finally the high-level features are extracted through the convolution layer.

6. The method of claim 4, wherein the method further comprises: The DAnA module uses the double-branch attention mechanism BA and the channel interaction spatial attention CISA to process and interact the features from multiple input sources; the input features are first subjected to adaptive enhancement processing by the DAnA module to output optimized features, which are refined and enhanced by the boundary information in the BA to output boundary-optimized features, and then subjected to global context fusion in the CISA to enhance the feature expression by combining the global and local context information, which contains rich information after adaptive enhancement, boundary optimization and global context fusion.

7. The method of claim 4, wherein the method further comprises: The CEM module combines the heat map and the original input, and the features are first input into the CAM channel attention module to perform weighting processing on the input features through the channel attention mechanism to enhance the expression of important features; the processed features are divided into two branches: one branch directly outputs the CAM-processed features, and the other branch further extracts local and compressed features through 3x3 and 1x1 convolution operations; finally, the features of the two branches are merged in the fusion module and output after being processed by the MLP multi-layer perceptron.

8. The method of claim 1, wherein the method further comprises: In the training process of the chip surface defect target detection network, the loss is calculated based on the output results of the detection head, and the weights of the encoder and the decoder are updated by using back propagation.

9. A meta-learning based chip surface defect target detection network, characterized in that, A chip surface defect target detection network is trained by using the chip surface defect target detection network training method based on meta-learning in any one of claims 1-8, and the chip surface defect target detection network comprises an encoder, a decoder and a detection head connected in sequence; wherein the encoder adopts a Resnet network, and the input of the Resnet network is a meta-support set and a meta-query set composed of pictures; the Resnet network comprises multiple layers, and each layer outputs a feature map of different size; The decoder is composed of a plurality of multi-scale feature fusion modules, which receive feature maps output by the encoder and perform multi-scale feature fusion to obtain a prediction feature map for predicting a target position, and the multi-scale feature fusion module includes a DGM module, a CEM module and a DAnA module; The detection head includes a CRD module, a MAD module, a center point regression prediction module and a point frame distance regression prediction module, wherein the CRD module performs feature enhancement on the prediction feature map output by the decoder, and then the center point regression prediction module performs regression prediction on the center point position information and the classification information; The MAD module performs multi-scale feature transformation and deep convolution operation on the feature-enhanced feature map to enhance the spatial distance information in the feature map; and then the point frame distance regression prediction module performs regression prediction on the distance between the center point and the target prediction frame.

10. A chip surface defect target detection method based on the chip surface defect target detection network based on meta-learning according to claim 9, characterized in that, The detection method includes the following steps: S1, obtaining the RGB picture of the chip surface defect, and constructing the meta support set and the meta query set; S2, in the preprocessing stage of the network, the picture size in the meta support set and the meta query set is unified; S3, inputting the RGB picture of the chip surface defect processed in S2 into the encoder for feature extraction to generate a feature map; S4, inputting the feature map into the decoder, and performing feature fusion through each multi-scale feature fusion module in the decoder to obtain a prediction feature map capable of predicting a target position; S5, inputting the prediction feature map generated in the decoder into the detection head, and obtaining the center point position, point frame distance and category prediction result of the predicted target through the center point regression prediction module and the point frame distance regression prediction module; S6, performing non-maximum suppression according to the prediction result and drawing the prediction frame information, and restoring the result to the original picture to obtain the category and prediction frame of the chip surface defect in the original image, and complete the prediction process of the network.

Images