Wafer dislocation defect detection method, device, equipment and storage medium

Abstract

Embodiments of the present application provide a wafer dislocation defect detection method, device, equipment and storage medium, and relate to the technical field of image recognition. The detection method comprises steps S1 to S4. S1, a wafer picture is obtained. S2, according to the wafer picture, feature extraction is performed through a backbone network to obtain a first feature layer set. S3, according to the first feature layer set, a second feature layer set is obtained through a multi-layer center point network model. S4, according to the second feature layer set, a defect target center point and a distance from the center point to a bounding box are obtained through an output module. Through the detection method of the present application, the detection result and quantity of dislocation defects can be obtained by only inputting the wafer surface image into the model, a high-efficiency automatic detection method is realized, and the detection precision is higher than that of other detection methods, thereby greatly reducing the production cost of enterprises.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image recognition, in particular to a wafer dislocation defect detection method, device, equipment and storage medium. BACKGROUND

[0002] The wafer refers to a silicon wafer used for manufacturing silicon semiconductor integrated circuits, and is the most important raw material in the chip manufacturing process. The wafer manufacturing process is as follows: first, high-purity polycrystalline silicon is obtained by drawing and refining from ordinary silicon sand, and then the single crystal silicon rod is formed by slowly drawing after dissolving and adding a silicon crystal seed. After a series of processing technologies such as slicing, grinding and polishing, the single crystal silicon wafer, that is, the wafer, is formed.

[0003] The dislocation defect on the wafer is an internal microscopic defect formed in the crystal due to the existence of internal stress in the crystal when the silicon crystal rod is drawn out, which will appear on the wafer after slicing the single crystal silicon rod. This defect is the most important defect in semiconductors, and its size is usually in the micron level. When the dislocation defects in a certain area of the wafer reach a certain number, this area may cause impurity precipitation during subsequent processing, which will have a great impact on the internal circuit of the chip.

[0004] In the prior art, many commonly used dislocation defect detection methods need to perform chemical corrosion or reaction on the crystal, which will have an irreversible quality impact on the wafer and cannot be accepted by the industry. At present, most enterprises still use manual visual inspection to detect the wafer dislocation defect. This method is extremely low in efficiency and difficult to guarantee the detection accuracy.

[0005] The automatic optical detection and other defect detection methods based on traditional image algorithms will be affected by the existence of many non-defect targets similar to the dislocation defects and incomplete defect targets at the image edge in the wafer surface image, which will affect the detection performance of the model and cause misjudgment. The dense distribution of wafer dislocation defects in the image also greatly affects the detection accuracy of the model.

[0006] Therefore, the present application is proposed based on the research on the prior art. SUMMARY

[0007] The present application provides a wafer dislocation defect detection method, device, equipment and storage medium to improve at least one of the above technical problems.

[0008] In a first aspect,

[0009] The present application provides a wafer dislocation defect detection method, device, equipment and storage medium to improve at least one of the above technical problems.

[0010] S1, obtaining a wafer picture.

[0011] S2, according to the wafer picture, feature extraction is performed through the backbone network to obtain a first feature layer set.

[0012] S3, according to the first feature layer set, a second feature layer set is obtained through a multi-layer center point network model, wherein the multi-layer center point network model is constructed based on a feature pyramid and a global-local attention mechanism.

[0013] S4, according to the second feature layer set, a defect target center point and a distance from the center point to the bounding box are obtained through an output module.

[0014] The second aspect,

[0015] The embodiment of the application provides a wafer dislocation defect detection device, which comprises:

[0016] The picture acquisition module is configured to acquire a wafer picture.

[0017] The first feature acquisition module is configured to acquire a first feature layer set by performing feature extraction through a backbone network according to the wafer picture.

[0018] The second feature acquisition module is configured to acquire a second feature layer set by performing feature extraction through a multi-layer center point network model according to the first feature layer set, wherein the multi-layer center point network model is constructed based on a feature pyramid and a global-local attention mechanism.

[0019] The defect recognition module is configured to acquire a defect target center point and a distance from the center point to the bounding box through an output module according to the second feature layer set.

[0020] The third aspect,

[0021] The embodiment of the application provides a wafer dislocation defect detection device, which comprises a processor, a memory, and a computer program stored in the memory. The computer program can be executed by the processor to implement the wafer dislocation defect detection method as described in any one of the first aspect.

[0022] The fourth aspect,

[0023] The embodiment of the application provides a computer readable storage medium, which comprises a stored computer program, wherein when the computer program is running, the computer readable storage medium controls the device where the computer readable storage medium is located to execute the wafer dislocation defect detection method as described in any one of the first aspect.

[0024] By adopting the above technical solution, the application can achieve the following technical effects:

[0025] The detection method provided by the embodiment of the present application only needs to input the wafer surface image into the model to obtain the detection result and quantity of the dislocation defect, realizes the efficient and automatic detection method, and has higher detection precision compared with other detection methods, and greatly reduces the production cost of enterprises. BRIEF DESCRIPTION OF DRAWINGS

[0026] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0027] Figure 1 is a flowchart of the detection method provided by the first embodiment of the present application.

[0028] Figure 2 is a wafer picture with dislocation defects.

[0029] Figure 3 is a network structure diagram of the detection method provided by the first embodiment of the present application.

[0030] Figure 4 is a network structure diagram of a multi-layer center point network model.

[0031] Figure 5 is a network structure diagram of the GLA feature fusion mode.

[0032] Figure 6 is a network structure diagram of the output module.

[0033] Figure 7 is a position relationship diagram of the output probability heat map and the original picture corresponding sampling points under the condition of four times down sampling.

[0034] Figure 8 is a boundary box diagram of the wafer dislocation defect.

[0035] Figure 9 is a structure diagram of the detection device provided by the second embodiment of the present application. DETAILED DESCRIPTION

[0036] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0037] Embodiment one,

[0038] Referring to Figures 1 to 8 , the first embodiment of the present application provides a wafer dislocation defect detection method, which can be executed by a wafer dislocation defect detection device (hereinafter referred to as: detection device). In particular, it is executed by one or more processors in the detection device to realize steps S1 to S4.

[0039] S1, obtain a wafer picture.

[0040] It should be noted that the size of the dislocation defect on the wafer is usually in the micron level, which needs to be observed using a professional electron microscope. And as Figure 2 shown, the black dots in the figure represent dislocation defects, and wafer dislocation defects often present a dense distribution, which is a difficult challenge for traditional machine vision detection methods. When the defects are closely connected, the traditional method cannot accurately locate and count them;

[0041] It can be understood that the detection device can be a portable notebook computer, a desktop computer, a server, a smart phone or a tablet computer, etc. electronic equipment with computing performance. The detection device and the electron microscope are in communication connection, and can receive the wafer picture taken by the electron microscope.

[0042] S2, according to the wafer picture, feature extraction is performed through a backbone network to obtain a first feature layer set.

[0043] Specifically, for the input picture, the present embodiment adopts ResNet50 as the backbone network to perform feature extraction. According to the different down-sampling multiples and the receptive fields of the generated features, the feature layers (i.e. the first feature layer set) generated by ResNet50 can be divided into feature layer, feature layer, feature layer, feature layer and feature layer five parts. The down-sampling multiple of the feature layer is 2, the down-sampling multiple of the feature layer is 4, the down-sampling multiple of the feature layer is 8, the down-sampling multiple of the feature layer is 16, and the down-sampling multiple of the feature layer is 32. As Figure 3 shown by the dashed box in the upper left corner.

[0044] It can be understood that the features in the picture can be extracted through the backbone network, so as to be converted into data that can be recognized by the computer. In other embodiments, other existing backbone networks can be used, and the present application does not make specific limitations.

[0045] S3, acquiring a second feature layer set through a multi-layer center point network model according to the first feature layer set. The multi-layer center point network model is constructed based on a feature pyramid and a global-local attention mechanism. Preferably, the second feature layer set includes feature layers, feature layers feature layers, feature layers and feature layers.

[0046] Specifically, the embodiment of the application combines a feature pyramid network (FPN) into the backbone network of the model, introduces a global-local attention mechanism (GLA) to optimize the fusion mode of different feature layers in the FPN, and constructs a GLA-FPN with stronger feature extraction capability. On this basis, the application allocates targets of different scales to feature layers of different receptive fields for training, and forms a model structure of a multi-layer center point network.

[0047] As shown in Figure 3 and Figure 4 , on the basis of the above embodiment, in an optional embodiment of the application, step S3 includes S31 to S35.

[0048] S31, uniformly changing the channel number of the feature layers, feature layers and feature layers in the first feature layer set to 256 through inter-layer linking.

[0049] S32, obtaining the feature layers through channel attention and convolution according to the feature layers after the channel number is changed.

[0050] S33, obtaining the feature layers and feature layers through 2 times and 4 times down-sampling convolution according to the feature layers.

[0051] S34, obtaining the feature layers through common fusion of the feature layers after the channel number is changed and the feature layers after the channel number is changed from top to bottom.

[0052] In an optional embodiment, step S34 includes S341 to S343.

[0053] S341, obtaining the The feature layers are linked from top to bottom, 2 times up-sampling is performed by the nearest neighbor interpolation method, and the feature layers are restored to the same scale as the feature layers.

[0054] S342, the up-sampled feature layers and the feature layers after uniform channel number are fused by a GLA feature fusion method.

[0055] S343, 3x3 convolution is performed on the fused features to obtain feature layers.

[0056] S35, the up-sampled feature layers and the feature layers after uniform channel number are fused to obtain feature layers.

[0057] In an optional embodiment, step S35 includes S351 to S353.

[0058] S351, the up-sampled feature layers are linked from top to bottom, 2 times up-sampling is performed by the nearest neighbor interpolation method, and the feature layers are restored to the same scale as the feature layers.

[0059] S352, the up-sampled feature layers and the feature layers after uniform channel number are fused by a GLA feature fusion method.

[0060] S353, 3x3 convolution is performed on the fused features to obtain feature layers.

[0061] In this embodiment, by combining the GLA-FPN structure optimized by the GLA feature fusion method to the backbone network of the model, a multi-layer center point network structure is constructed, and the low-level features of the model are effectively reused and further fused with high-level features by GLA.

[0062] Assigning targets of different scales to different feature layers for training successfully improves the feature extraction capability of the model and the detection capability for targets of different scales, and the detection effect for small targets is particularly obvious, and the problem of serious detection difficulty of wafer dislocation defect small targets is successfully solved.

[0063] Specifically, in order to construct the multi-layer center point network structure based on GLA-FPN, the inventors select , and feature layers as the input features of GLA-FPN, and generate , , , and Five-layer features, constructed as follows Figure 4 As shown.

[0064] After the feature layer is input into GLA-FPN, inter-layer connections are first used to... , and The number of channels in the feature layers was standardized to 256 to facilitate subsequent fusion operations on different feature layers. Then, after standardizing the number of channels... Channel attention and Convolution generation The feature layer is then subjected to convolutional operations that downsample it by 2x and 4x to generate... and Feature layer.

[0065] and The feature layer is constructed through top-down connections, with a uniform number of channels. , , It is formed by the fusion of various characteristics. Taking the feature layer as an example, the links from top to bottom are first processed using the nearest neighbor interpolation method to unify the number of channels. The feature layer is upsampled by a factor of 2 to restore it to its original state. The feature layers are at the same scale, and then GLA feature fusion is used to fuse high-level and low-level feature content. Finally, a... Convolution to generate the final Feature layer. Similarly, we can obtain... The method of generating feature layers.

[0066] like Figure 5 As shown, based on the above embodiments, an optional embodiment of the present invention includes the following GLA feature fusion method:

[0067] low-level feature maps and high-level feature maps Add them together.

[0068] Based on the low-level feature map after addition and high-level feature maps Global feature information is obtained through channel attention.

[0069] Based on the low-level feature map after addition and high-level feature maps , obtain local feature information through nonlinear transformation.

[0070] The global feature information and the local feature information are added to the positions of the corresponding channels, and then converted through Sigmoid to obtain attention weights .

[0071] The low-level feature map is multiplied by , the high-level feature map is multiplied by , and the two products are added to obtain the fused feature layer.

[0072] Specifically, in order to improve the feature extraction ability of the model and reduce the false detection of easily confused targets, the inventors proposed a feature fusion method based on GLA to more effectively fuse feature maps, as shown in Figure 5 .

[0073] Suppose and are feature maps from lower and higher layers in GLA-FPN, respectively, where has a larger receptive field and has maintained the spatial dimension consistent with through upsampling, GLA first adds and to generate attention weights. For global features, channel attention is used to aggregate spatial information in the feature map into a scalar , which can effectively represent global feature information on different channels; while local features need to be obtained from the spatial dimension of the feature map, GLA performs nonlinear transformation on the feature map itself to obtain local feature information, and keeps the feature map dimension unchanged before and after transformation, so as to facilitate subsequent fusion. In order to reduce the amount of calculation, the dimension transformation made in GLA is realized by convolution. After obtaining global and local feature information, GLA adds global attention of each channel to all positions of the corresponding local information, which emphasizes the importance of different channels of the feature map and does not lose local feature information of the feature map. Then, the obtained result is converted to a value between by Sigmoid, as the attention weight guiding feature map fusion. After obtaining the attention weight , GLA uses it to process and respectively, multiplies with the weight directly to obtain , and multiplies with Element multiplication is performed to obtain , and feature screening on different feature maps is realized. Finally, is added to to generate the final fusion feature .

[0074] In this embodiment, after passing through the GLA-FPN structure, feature layers, feature layers feature layers, feature layers and feature layers have a downsampling factor of 8, 16, 32, 64 and 128 respectively.

[0075] In order to adapt to the receptive field of different feature layers and enhance the detection ability of the model for different scale targets, different scale sizes of defect targets are allocated to specific GLA-FPN feature layers for training and detection in this embodiment.

[0076] According to the characteristics that the feature layer with a low downsampling factor and a small receptive field in the GLA-FPN structure is more suitable for detecting small targets, and the feature layer with a high downsampling factor and a large receptive field is more suitable for detecting large targets, the present application sets , , , and feature layers to be responsible for detecting defect targets with an area of , , , and respectively.

[0077] S4, according to the second feature layer set, through the output module, the center point of the defect target and the distance from the center point to the bounding box are obtained.

[0078] Specifically, the output module is used to identify the dislocation defects of each feature layer in the second feature layer set. In the embodiment of the present application, the center point position of the target is generated through the corresponding relationship between the output probability heat map of the model and the position of the sampling point of the original image, so as to reduce the instability of the model prediction. By improving the regression target of the model to the distance from the center point to the four edges of the bounding box, the accuracy of the bounding box prediction in the dense scene is improved.

[0079] As shown in Figure 3 and Figure 6 , on the basis of the above embodiment, in an optional embodiment of the present application, step S4 specifically comprises:

[0080] Each feature layer in the second feature layer set performs center point prediction and boundary regression through an output module, obtaining the defect target center point and the distance from the defect target center point to the four sides of the bounding box for each feature layer. The weights are shared among the output modules.

[0081] Specifically, the output modules are placed after each feature layer of the GLA-FPN structure, and the weights of the output modules of different layers are shared to reduce the computational cost of the model.

[0082] Preferably, the output module includes a center point prediction branch and a regression branch.

[0083] The center point prediction branch consists of four 3×3 convolutional layers, which output a probabilistic heatmap of the center point of the defect target. : In the formula, This represents the downsampling factor of the feature layer where the probability heatmap is located. and This represents the width and height of the input image.

[0084] Probability heatmap The value at each position represents the probability that a defective target center point exists at that position, such as... This means that in The center point of the defect target was detected.

[0085] It should be noted that the original CenterNet actually predicts the center point location through two branches: a center point prediction branch and a center point offset prediction branch. The former is consistent with the center point prediction branch of this invention, but due to the model downsampling, this branch can only obtain the position of the center point on the output probability heatmap, and cannot restore the accurate position on the input image.

[0086] To compensate for the coordinate offset error caused by the center point position during model sampling, the original method authors introduced a center point offset prediction branch to predict the decimal part of the center point's coordinates in the original image. This prediction was then combined with the center point offset prediction branch's result to obtain the coordinates on the original image. This approach improves the accuracy of coordinate prediction to some extent. However, the coordinates predicted by the model not only contain non-integer cases but are also affected by the dense distribution of dislocation defects on the wafer, sometimes leading to deviations in the prediction results and affecting detection accuracy. Furthermore, each feature layer of FPN requires an output module, and adding an output branch introduces additional computational overhead.

[0087] Therefore, this invention improves the method of constructing the center point position in the original image, and for a certain feature layer, it outputs a point on the probability heatmap. Map it back to the input image The location is calculated as follows: In the formula, This indicates the downsampling factor of the current feature layer.

[0088] Since the final output size of the model is still downsampled relative to the input image, a point on the output probability heatmap represents all parts of the corresponding receptive field in the input image, and the position is calculated using Equation 1. That is, the center point of the corresponding receptive field, can Considered Sampling points at the location.

[0089] like Figure 7 As shown, each point on the output probability heatmap is associated with the corresponding sampling point in the original image. In this way, the predicted position of the center point obtained by the model can be guaranteed to correspond to the specific coordinates of the original image, rather than relying on the model to make a rough prediction.

[0090] The present invention improves the output module by removing the center point offset prediction branch in the original model and generating the center point position of the target by the correspondence between the output probability heatmap of the model and the sampling points of the original image, thereby reducing the instability of the model prediction.

[0091] The regression branch consists of four 3×3 convolutional layers, which output a regression component feature map with four channels. The four channels of the regression component feature map represent the distances from the point to the left, top, right, and bottom edges of the target bounding box, respectively.

[0092] It should be noted that the original CenterNet directly regresses the width and height of the bounding box. However, this embodiment of the invention regresses the distances from the center point to the four sides of the bounding box.

[0093] Specifically, due to the significant overlap between different targets in dense scenes, and influenced by the label assignment method, the predicted probability value of the center point in the region near the center point of two targets will also be higher in dense scenes, making it easy for the predicted center point position output by the model to be offset. In this case, allowing the model to directly regress the width and height of the bounding box will produce even more inaccurate bounding boxes, thus leading to a decrease in detection accuracy.

[0094] Therefore, this invention uses a regression branch to output a regression component feature map with four channels. The four channels of the feature map represent the distances from the point to the left, top, right, and bottom edges of the target bounding box, respectively. , , , .

[0095] In this way, the model can more accurately determine the boundaries of targets in overlapping regions. Even if the center point prediction is biased due to dense scenes, it can still be corrected by outputting different lengths. 、 、 、 distance to adjust the position of the final bounding box.

[0096] Meanwhile, this regression method is more consistent with the method of constructing the target bounding box by predicting the position of the corresponding original image sampling point through the center point prediction branch, which successfully improves the detection accuracy of the model in a simple and effective manner without increasing excessive computational overhead, and makes the multi-layer center point network more robust for dense scene detection.

[0097] The detection method of the embodiment of the present application only needs to input the wafer surface image into the model to obtain the detection result and quantity of dislocation defects, realizes an efficient and automatic detection method, and has higher detection accuracy compared with other detection methods, greatly reducing the production cost of enterprises.

[0098] On the basis of the above embodiment, in an optional embodiment of the present application, the detection method further comprises steps S5 to S7.

[0099] S5, merging the defect target center points of each feature layer, and calculating the original image sampling point coordinates of each center point.

[0100] S6, calculating the diagonal point coordinates of the bounding box according to the original image sampling point coordinates and the distances of the defect target center points of each feature layer to the four edges of the bounding box.

[0101] S7, obtaining the probability of each original image sampling point coordinate existing a defect target center point, and obtaining the first 150 results with a probability value greater than 0.1 as the detection result according to the probability.

[0102] Specifically, in the inference stage, first, the center point prediction results output by each feature layer of the model are merged, and the original image sampling point coordinates corresponding to each prediction center point are obtained according to formula 1 , and the left upper corner and right lower corner coordinates of the target bounding box are calculated after multiplying the following sampling multiple by the result of the regression branch , and the calculation method is as follows: Then, the top 150 results with a model prediction center point probability value greater than 0.1 are reserved as the final detection result.

[0103] In the training stage of the detection method of the embodiment of the present application, the positive sample position probability of the center point prediction branch is set to 1, and the negative sample is 0. As for the selection of the specific positive sample position, the present application uses an adaptive label assignment method to assign the center point positive sample, and the model needs to find the optimal positive sample position to reassign the label when calculating the loss in each round. The specific process is as follows:

[0104] First, the forward calculation is normally performed to obtain the predicted output, then the label true value and the probability heat map output generated by the model are regarded as two independent sets, when the label assignment between the two is performed, the edge connection is established, thus the label assignment process can be regarded as a bipartite graph. Then according to the quality evaluation mode, the label true value and all predicted outputs are matched, and a quality matrix of can be obtained, wherein and are the number of label true values and predicted outputs respectively. The value not belonging to the spatial prior position is set to 0, and the Hungarian algorithm is used to solve the matching position with the highest overall quality, and the corresponding predicted output position is taken as a positive sample, and the loss is calculated with the label true value, and the remaining positions are regarded as negative samples to calculate the loss.

[0105] The present application matches the label and the predicted output under the quality evaluation condition can be expressed as the following formula: wherein, indicates whether the current position belongs to the spatial prior region to which the label belongs , if true, the item takes 1, otherwise 0; and represent the predicted output and the label true value of the classification probability of the corresponding position respectively; and represent the predicted output and the label true value of the bounding box of the corresponding position respectively; is a weight hyperparameter used to balance the classification and regression parts, which is taken as 0.8 in the present application.

[0106] Through the above proposed quality evaluation mode, the model can comprehensively consider the spatial prior condition and the matching degree of the classification and regression results and the label, so as to value the edges of the bipartite graph under different matching conditions. Instead of using a fixed positive sample assignment mode, the label assignment process of the model is optimized through bipartite matching, and an adaptive “one-to-one” label assignment strategy is proposed. In the model training process, the quality of different label assignment strategies is evaluated according to the output of the model to select the optimal assignment strategy, so as to avoid the influence of unreasonable label on the model training. Through the adaptive label assignment strategy, the removal of Non-Maximum Suppression (NMS) can be realized, and the error result suppression of NMS process in the post-processing stage under the dense scene can be prevented.

[0107] For the label construction of the regression branch, since the detection result of the target is based on the sampling point on the original image, the center point positive sample position corresponding to the sampling point position The distances between the four edges of the bounding box as regression labels: In the formula, The left upper corner and right lower corner coordinate labels of the target bounding box are represented.

[0108] The overall loss function of the detection method is as follows: In the formula, And The losses of the center point prediction branch and the regression branch are represented respectively, To balance the weights of different losses, the application sets .

[0109] The training of the center point prediction branch is optimized by using the improved Focal Loss: In the formula, And The weight hyperparameters for controlling the contributions of positive and negative samples in the Focal Loss are represented, The number of positive samples is represented. The application sets , .

[0110] For the regression branch, the application uses the GIoU loss for training: In the formula, And The model prediction bounding box and the label true value bounding box of the first defect target are represented respectively. The minimum closure region containing the prediction bounding box and the label bounding box is represented, The union region between the boxes is represented.

[0111] The embodiment of the application can achieve the following beneficial effects:

[0112] First, the application can automatically and efficiently complete the dislocation defect detection task in the wafer surface image, greatly liberating the artificial production capacity and reducing the cost of enterprises. The application only needs to input the wafer surface image to give the position and number of dislocation defects, and can detect the wafer surface more comprehensively and perfectly, rather than only sampling and detecting some areas.

[0113] Second, one of the difficulties in detecting wafer dislocation defects in images is that the target scale changes greatly, and small targets are mainly used. Small targets have always been a difficulty in target detection tasks. In view of this, the application combines the FPN structure to the model backbone network to form a multi-layer center point network structure, further reuses and fuses the features extracted by the backbone network, and distributes targets of different scales to different feature layers of the receptive field for training, effectively enhancing the detection ability of the model for small targets.

[0114] Thirdly, there are many non-defect targets similar to dislocation defects in the wafer surface image, and incomplete defect targets at the edge of the image, which will affect the detection performance of the model and cause misjudgment. Therefore, the feature fusion operation in the FPN is improved, the global feature information and the local feature information after the fusion of the feature maps at different levels are combined to obtain attention information more conducive to the target detection task, a global-local attention fusion mode GLA is proposed to enhance the feature extraction capability of the model, and a GLA-FPN is constructed to further improve the detection capability of the model for the easily confused targets.

[0115] Fourthly, the dense distribution of wafer dislocation defects in the image also has a great influence on the detection performance of the model. In the scene where the targets are densely distributed, the model sometimes misses multiple targets that overlap or are very close to each other, or makes inaccurate judgments on the target boundaries, resulting in a decrease in detection accuracy. In view of this, the present application effectively alleviates the problem of inaccurate detection in the dense scene by improving the output module of the model and the label assignment mode.

[0116] As for the output module, the present application removes the center point offset prediction branch in the original method, generates the target center point position according to the position correspondence between the output probability heat map and the sampling points of the original image, reduces the instability of the model prediction, and further improves the regression target of the model on the regression branch to the distance from the center point to the four edges of the bounding box, so as to improve the accuracy of the bounding box prediction in the dense scene.

[0117] As for the label assignment, instead of using a fixed center point position calculation method, the present application dynamically assigns the optimal positive sample label to each round of training by matching the prediction output and the label true value at different positions of the model, realizes optimal “one-to-one” label assignment, removes the NMS post-processing process, and prevents it from producing incorrect suppression on the output results in the dense scene.

[0118] Embodiment two,

[0119] The embodiment of the present application provides a wafer dislocation defect detection device, which comprises:

[0120] A picture acquisition module 1 is used to acquire a wafer picture.

[0121] A first feature acquisition module 2 is used to acquire a first feature layer set by performing feature extraction on the wafer picture through a backbone network.

[0122] A second feature acquisition module 3 is used to acquire a second feature layer set by a multi-layer center point network model according to the first feature layer set. The multi-layer center point network model is constructed based on a feature pyramid and a global-local attention mechanism.

[0123] The defect identification module 4 is configured to acquire, according to the second feature layer set, a defect target center point and a distance from the center point to a bounding box through the output module.

[0124] In an optional embodiment, the second feature acquisition module 3 specifically comprises:

[0125] The inter-layer linking unit is configured to uniformly set the channel number of the feature layer, feature layer and feature layer in the first feature layer set to 256 through inter-layer linking.

[0126] The feature layer acquisition unit is configured to acquire, according to the feature layer after the channel number is uniformly set, a feature layer through channel attention and convolution.

[0127] , The feature layer acquisition unit is configured to acquire, according to the feature layer, a feature layer and a feature layer through 2-fold and 4-fold down-sampling convolution.

[0128] The feature layer acquisition unit is configured to acquire, according to the feature layer after the channel number is uniformly set and the feature layer after the channel number is uniformly set after being linked from top to bottom, a feature layer through common fusion.

[0129] The feature layer acquisition unit is configured to acquire, according to the feature layer after the channel number is uniformly set and the feature layer after being linked from top to bottom, a feature layer through common fusion.

[0130] In an optional embodiment, the feature layer acquisition unit specifically comprises:

[0131] The first up-sampling subunit is configured to acquire, according to the feature layer after the channel number is uniformly set and the feature layer after being linked from top to bottom, a feature layer with the same scale as the

[0132] feature layer through 2-fold up-sampling by the nearest neighbor interpolation method. The first GLA fusion subunit is configured to acquire, according to the The feature layers are fused.

[0133] The first convolution subunit is configured to perform 3x3 convolution on the fused feature layers to obtain feature layers.

[0134] In an optional embodiment, The feature layer acquisition unit specifically includes:

[0135] The second up-sampling subunit is configured to perform 2x up-sampling on the feature layers with unified channel numbers by using the nearest neighbor interpolation method, so that the feature layers are restored to the same size as the feature layers.

[0136] The second GLA fusion subunit is configured to fuse the up-sampled feature layers and the feature layers with unified channel numbers by using a GLA feature fusion manner.

[0137] The second convolution subunit is configured to perform 3x3 convolution on the fused feature layers to obtain feature layers.

[0138] In an optional embodiment, the defect identification module 4 is specifically configured to:

[0139] Each feature layer in the second feature layer set is configured to perform center point prediction and boundary regression by using an output module to obtain a defect target center point and a distance from the defect target center point to four edges of a boundary box of each feature layer. The weight is shared between each output module.

[0140] In an optional embodiment, the wafer dislocation defect detection device further includes:

[0141] The merging module is configured to merge the defect target center points of each feature layer and calculate original image sampling point coordinates of each center point.

[0142] The boundary point calculation module is configured to calculate diagonal point coordinates of the boundary box according to the original image sampling point coordinates and the distances from the defect target center points of each feature layer to four edges of the boundary box.

[0143] The screening module is configured to obtain a probability that each original image sampling point coordinate exists a defect target center point, and obtain the first 150 results with a probability value greater than 0.1 as the detection results according to the probability.

[0144] Embodiment three,

[0145] ​​The embodiment of the present application provides a wafer dislocation defect detection device, which comprises a processor, a memory and a computer program stored in the memory. The computer program can be executed by the processor to realize the wafer dislocation defect detection method as described in any one of the embodiment one.

[0146] Embodiment four,

[0147] The embodiment of the present application provides a computer readable storage medium, which comprises a stored computer program. When the computer program is running, the computer readable storage medium controls the device where the computer readable storage medium is located to execute the wafer dislocation defect detection method as described in any one of the embodiment one.

[0148] In several embodiments provided by the present application, it should be understood that the disclosed apparatus and method can also be implemented by other manners. The apparatus and method embodiments described above are only illustrative, for example, the flow chart and block diagram in the drawings show the possible implementation architecture, function and operation of the apparatus, method and computer program product according to the embodiments of the present application. In this regard, each block in the flow chart or block diagram can represent a module, program segment or part of code, which contains one or more executable instructions for realizing the specified logic function. It should also be noted that, in some alternative implementation manners, the functions annotated in the blocks can also occur in the order different from that annotated in the drawings. For example, two continuous blocks can actually be executed substantially in parallel, and sometimes they can also be executed in reverse order, which depends on the functions involved. It should also be noted that each block in the block diagram and / or flow chart, and the combination of blocks in the block diagram and / or flow chart, can be realized by a special hardware-based system which executes the specified function or action, or can be realized by a combination of special hardware and computer instructions.

[0149] In addition, each functional module in the embodiments of the present application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0150] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application or parts of the present application that essentially contribute to the prior art or parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, an electronic device, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes. It should be noted that in this document, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the processes, methods, articles or devices that include a series of elements not only include those elements, but also include other elements not explicitly listed or inherent to such processes, methods, articles or devices. Without more limitations, the element defined by the statement "includes a" does not exclude the presence of other identical elements in the process, method, article or device that includes the element.

[0151] The terms used in the embodiments of the present application are merely for the purpose of describing particular embodiments and are not intended to limit the present application. The singular forms "a", "an" and "the" used in the embodiments of the present application and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise.

[0152] It should be understood that the term "and / or" used herein is merely a description of the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B, which can represent the three cases of A alone, A and B together, and B alone. In addition, the character " / " in this document generally represents an "or" relationship between the front and rear associated objects.

[0153] Depending on the context, the word "if" as used herein can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting". Similarly, depending on the context, the phrase "if it is determined" or "if (a stated condition or event) is detected" can be interpreted as "when it is determined" or "in response to determining" or "when (a stated condition or event) is detected" or "in response to detecting (a stated condition or event)".

[0154] The "first / second" mentioned in the embodiments are only to distinguish similar objects, and do not represent a specific order for the objects. Understandably, the "first / second" can be interchanged in a specific order or sequence as appropriate. It should be understood that the objects distinguished by "first / second" can be interchanged as appropriate, so that the embodiments described herein can be implemented in an order other than those illustrated or described herein.

[0155] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. Various modifications and changes can be made to the present application by those skilled in the art. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for detecting dislocation defects in a wafer, characterized by, The method comprises: obtaining a wafer picture; performing feature extraction on the wafer picture through a backbone network to obtain a first feature layer set; obtaining a second feature layer set through a multi-layer center point network model based on the first feature layer set; the multi-layer center point network model is constructed based on a feature pyramid and a global-local attention mechanism; obtaining a defect target center point and a distance from the center point to a bounding box through an output module based on the second feature layer set; The backbone network is ResNet50; the first feature layer set includes feature layers, feature layers, feature layers, feature layers and feature layers; the down sampling multiple of the feature layer is 2, the down sampling multiple of the feature layer is 4, the down sampling multiple of the feature layer is 8, the down sampling multiple of the feature layer is 16 and the down sampling multiple of the feature layer is 32; The second set of feature layers includes feature layers, feature layers feature layers, feature layers and feature layers; obtaining a second feature layer set through a multi-layer center point network model based on the first feature layer set, specifically including: The number of channels of the feature layers in the first feature layer set is unified to 256 through inter-layer linking. feature layers, feature layers and feature layers are unified to 256. According to the unified number of channels characteristic layer, through channel attention and convolution, obtain characteristic layer; According to the described characteristic layer, the convolution of 2 times and 4 times down-sampling is carried out, and the characteristic layer and characteristic layer; According to the unified number of channels after The characteristic layer is linked from top to bottom, and the number of channels is unified The characteristic layer is fused together, and the Characteristic layer; According to the unified number of channels after The characteristic layer is linked from top to bottom, and the number of channels is unified The characteristic layer is fused together, and the Characteristic layer; obtaining a defect target center point and a distance from the center point to a bounding box through an output module based on the second feature layer set, specifically including: Each feature layer in the second feature layer set performs center point prediction and bounding box regression through an output module to obtain a defect target center point and a distance from the center point to four edges of a bounding box of each feature layer; the output modules share weights; The output module comprises a center point prediction branch and a regression branch; the center point prediction branch comprises four 3*3 convolution layers, and is configured to output a probability heat map of a defect target center point ; wherein ; wherein represents a down-sampling multiple of a feature layer where the probability heat map is located, and represent a width and a height of the input image; the regression branch comprises four 3*3 convolution layers, and is configured to output a regression component feature map with a channel number of 4; wherein the four channels of the regression component feature map respectively represent distances of the center point to left, upper, right and lower edges of a target bounding box.

2. The wafer dislocation defect detection method of claim 1, characterized in that According to the unified number of channels after The characteristic layer is linked from top to bottom, and the number of channels is unified The characteristic layer is fused together, and the The characteristic layer, specifically includes: According to the unified number of channels The feature layers are linked from top to bottom, and 2 times up-sampling is performed through the nearest neighbor interpolation method to make them recover the same scale as the feature layers. The up-sampled feature layers and the feature layers after uniforming the number of channels are fused by a GLA feature fusion manner. The up-sampled feature layers and the feature layers after uniforming the number of channels are fused by a GLA feature fusion manner.​ For the fused features, a 3x3 convolution is performed to obtain the feature layer; According to the unified number of channels after The characteristic layer is linked from top to bottom, and the number of channels is unified The characteristic layer is fused together, and the The characteristic layer, specifically includes: According to the unified number of channels The feature layers are linked from top to bottom, and 2 times up-sampling is performed through the nearest neighbor interpolation method to make them recover the same scale as the feature layers. The up-sampled feature layers and the feature layers after uniforming the number of channels are fused by a GLA feature fusion manner. The up-sampled feature layers and the feature layers after uniforming the number of channels are fused by a GLA feature fusion manner.​ For the fused features, a 3x3 convolution is performed to obtain the feature layer.

3. The method for detecting wafer dislocation defects according to claim 2, characterized in that, the GLA feature fusion mode comprises: adding low-level feature maps and high-level feature maps together; According to the added low-level feature map and high-level feature map , global feature information is obtained through channel attention; According to the added low-level feature map and high-level feature map , local feature information is obtained through a nonlinear transformation; The global feature information and the local feature information are added at positions corresponding to channels, and then converted by Sigmoid to obtain attention weights ; low-level feature maps and Multiply, and combine the high-level feature maps and Multiply the two products and add them together to obtain the fused feature layer.

4. The method for detecting wafer dislocation defects according to claim 1, characterized in that, obtaining a defect target center point and a distance from the center point to a bounding box through an output module based on the second feature layer set, and then further comprising: merging the defect target center points of each feature layer and calculating the original picture sampling point coordinates of each center point; calculating the diagonal point coordinates of the bounding box based on the original picture sampling point coordinates and the distances from the defect target center points of each feature layer to four edges of the bounding box; obtaining the probability that each original picture sampling point coordinate exists a defect target center point, and obtaining the first 150 results with a probability value greater than 0.1 as the detection results based on the probability.

5. A device for detecting dislocation defects in a wafer, characterized by The wafer dislocation defect detection method of any one of claims 1 to 4 is suitable for being executed; The detection device comprises: a picture acquisition module configured to obtain a wafer picture; a first feature acquisition module configured to perform feature extraction on the wafer picture through a backbone network to obtain a first feature layer set; a second feature acquisition module configured to obtain a second feature layer set through a multi-layer center point network model based on the first feature layer set; the multi-layer center point network model is constructed based on a feature pyramid and a global-local attention mechanism; a defect identification module configured to obtain a defect target center point and a distance from the center point to a bounding box through an output module based on the second feature layer set.

6. A wafer dislocation defect detection device, characterized in that, The computer program can be executed by the processor to implement the wafer dislocation defect detection method of any one of claims 1 to 4.

7. A computer readable storage medium characterized in that, The computer readable storage medium comprises a stored computer program, wherein the computer readable storage medium controls the device where the computer readable storage medium is located to execute the wafer dislocation defect detection method of any one of claims 1 to 4 when the computer program runs.

Images