Wafer surface defect detection method

Abstract

The application discloses a wafer surface defect detection method, and relates to the technical field of detection, which comprises the following steps: inputting a collected wafer defect image into a semantic segmentation network model for pixel-level labeling, and obtaining a mask image and a first defect prediction result of the wafer defect image; wherein the wafer defect image only contains one type of defect, and the semantic segmentation network model contains a multi-scale module for improving the prediction accuracy; inputting the wafer defect image and the mask image into a classification network prediction model for prediction, and obtaining a second defect prediction result; and determining a final prediction result of the image based on the first defect prediction result and the second defect prediction result. The semantic segmentation network model and the classification learning network model are adopted, the influence of background noise on the classification model is filtered out, and the mask image containing the entire background is used for judgment, so that the accuracy of the prediction result is improved.

Description

Technical Field

[0001] This application relates to the field of detection technology, and in particular to a method for detecting defects on the surface of a wafer. Background Technology

[0002] Currently, wafer defect detection methods mainly consist of two steps. The first step involves locating the approximate location of the defect based on the detection algorithm built into the industrial camera and taking an image of it for subsequent processing.

[0003] The second step, to differentiate between defect categories, involves using AI algorithms to classify the defects. Common classification methods include:

[0004] (1) Classify the small icons directly. However, this method may not be effective for images with a small percentage of defects and may be prone to overfitting because background noise can have a negative impact on the classification model.

[0005] (2) Perform pixel-level annotation on the small images and then perform semantic segmentation. Although this method can segment defects from the images, the classification effect may be poor because some defects are not accurately classified based on local information alone. Summary of the Invention

[0006] This application provides a method for detecting defects on the wafer surface, solving the problem of inaccurate detection and classification of wafer defect types. The solution includes:

[0007] The collected wafer defect image is input into a semantic segmentation network model for pixel-level annotation to obtain a mask image of the wafer defect image and a first defect prediction result; wherein, the wafer defect image contains only one type of defect, and the semantic segmentation network model contains a multi-scale module to improve prediction accuracy;

[0008] The wafer defect map and the mask map are input together into the classification network prediction model for prediction to obtain the second defect prediction result;

[0009] The final image prediction result is determined based on the first defect prediction result and the second defect prediction result.

[0010] Specifically, obtaining the mask image through the semantic segmentation network model includes:

[0011] The prediction feature maps at different resolutions of the wafer defect map are extracted sequentially to obtain the first prediction feature map, the second prediction feature map, and so on, until the i-th prediction feature map at the target resolution.

[0012] The number of channels in the i-th predicted feature map is reduced to 1 / 4 of the original number, and multi-scale max pooling is performed in three separate paths. The pooling results of the three paths are then combined with the feature map before max pooling to obtain the i-th intermediate predicted feature map.

[0013] The i-th intermediate prediction feature map is upsampled to obtain the (i-1)-th intermediate prediction feature map, and then spliced ​​and fused with the (i-1)-th prediction feature map and upsampled to obtain the (i-2)-th intermediate prediction feature map.

[0014] Based on the (i-2)th intermediate predicted feature map, continue to stitch, fuse and upsample with the corresponding predicted feature map according to the resolution until the mask map is obtained; i is a positive integer greater than 2.

[0015] Specifically, the k-th predicted feature map and the k-th intermediate feature map have the same resolution and size, with k located between 1 and i; the size includes the feature map length, width, and number of channels, and the resolution is positively correlated with the image size;

[0016] The step of sequentially extracting predicted feature maps at different resolutions of the wafer defect map includes:

[0017] The first predicted feature map is convolved and normalized to obtain the second predicted feature map;

[0018] The second predicted feature map is convolved and normalized, and then downsampled to obtain the third predicted feature map, until the (i-1)th predicted feature map is downsampled to obtain the i-th predicted feature map.

[0019] Specifically, the step of concatenating and fusing the three pooling results with the feature map before max pooling to obtain the i-th intermediate predicted feature map includes:

[0020] The feature map after the three-way max pooling operation is concatenated with the feature map before the pooling operation. The concatenated feature map is then subjected to convolution and normalization operations. The resulting i-th intermediate predicted feature map has the same resolution as the i-th predicted feature map.

[0021] Specifically, based on the (i-2)th intermediate predicted feature map, the corresponding predicted feature map is further concatenated, fused, and upsampled according to the resolution until the mask image is obtained, including:

[0022] The intermediate prediction feature map is sequentially concatenated and fused with the prediction feature map corresponding to the resolution. The fusion is performed by convolution and upsampling until the first intermediate prediction feature map is obtained. The length and width of the first intermediate prediction feature map are the same as those of the first prediction feature map.

[0023] Continue performing convolution operations on the first intermediate predicted feature map to obtain the mask map with the same resolution as the first predicted feature map.

[0024] Specifically, the step of inputting the wafer defect map and the mask map into a classification network prediction model for prediction to obtain a second defect prediction result includes:

[0025] The classification network prediction model performs overlapping matching of the mask image and the wafer defect image to obtain a defect image;

[0026] The defect image is identified to obtain the second defect prediction result.

[0027] Specifically, the first defect prediction result is seg_result; the second defect prediction result is cls_result;

[0028] The final prediction result for the image is predicted_result, which is represented as follows:

[0029] predict_result=α·seg_result+β·cls_result

[0030] α and β are hyperparameters, determined based on model accuracy.

[0031] Specifically, the numerical range of predict_result indicates the defect type on the wafer surface, and the defect type of the wafer defect map is determined and output based on the numerical value.

[0032] The beneficial effects of the technical solution provided in this application include at least the following: combining a semantic segmentation network model and a classification learning network model, the original complete wafer defect image is input into the semantic segmentation network model for layer-by-layer extraction, and max pooling and fusion are performed through a multi-scale module to transform the low-level spatial information of the image into high-level semantic information; subsequently, upsampling and feature fusion techniques are used to generate a predicted mask image and a first defect prediction result using high-level semantic and low-level spatial information. After obtaining the mask image, the mask image and the original wafer defect image are further processed... Figure 1 The same data is input into the classification network prediction model. A pixel-level annotated wafer defect map with a complete background is obtained by superimposing the mask image and the original wafer defect map. Then, a classifier predicts the second defect prediction result. Finally, the final image prediction result is determined based on the two defect prediction values. Compared to the original direct annotation and small-image pixel-level annotation, this method filters out the influence of background noise on the classification model and improves the accuracy of the prediction results by using a mask image that includes the entire background. Attached Figure Description

[0033] Figure 1 This is a background image of wafer surface defects taken with an industrial camera;

[0034] Figure 2 This is a flowchart of the wafer surface defect detection method provided in the embodiments of this application;

[0035] Figure 3 This is a schematic diagram of the wafer surface defect detection method provided in the embodiments of this application;

[0036] Figure 4 This embodiment of the application provides a schematic diagram of the process for obtaining a mask image. Detailed Implementation

[0037] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0038] In this article, "multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0039] like Figure 1 As shown, the area highlighted on the left is the actual wafer defect captured by the camera. This method of directly labeling small images is not effective for classifying defects that make up a small portion of the image. The area on the right is a defect image selected using pixel-level extraction and labeling, with the background already processed by "cutting out" the background. These small images are then filtered by a classifier to obtain the detection results. However, for some defect types, pixel-level segmentation labeling alone is insufficient for accurate classification based on local information, so the accuracy of the recognition needs to be improved.

[0040] Figure 2 This is a flowchart of a wafer surface defect detection method provided in an embodiment of this application; it includes the following steps:

[0041] Step 201: Input the collected wafer defect map into the semantic segmentation network model for pixel-level annotation to obtain the mask map of the wafer defect map and the first defect prediction result.

[0042] This step requires obtaining complete wafer defect maps, which is crucial for ensuring no loss of background information. During training, this approach aims to ensure that each defect map contains only one defect type label to facilitate early model training.

[0043] refer to Figure 3The diagram shown is a schematic of the structure of the model executing the wafer surface defect detection method. After the wafer defect map is input into the semantic segmentation network model (backbone1), a predicted feature map is obtained, which is then transformed multiple times to obtain a mask map.

[0044] The semantic segmentation network model automatically annotates the complete wafer defect map at the pixel level, processes the image to obtain a mask map, and outputs the first defect prediction result. The first defect prediction result can initially calculate the predicted value based on the defects selected and annotated in the mask map.

[0045] The processing steps of a semantic segmentation network model include the following:

[0046] A. Sequentially extract the predicted feature maps of the wafer defect map at different resolutions to obtain the first predicted feature map, the second predicted feature map, and so on, until the i-th predicted feature map at the target resolution.

[0047] First, the model extracts the normal feature map of the wafer defect image, which is the feature map at the original resolution state. It defines this as the first predicted feature map and performs convolution and normalization operations on it step by step to reduce its feature map size and increase the number of channels in order to obtain spatial information in the high-dimensional space of the image.

[0048] like Figure 4 The diagram shows the process of obtaining the mask. The first predicted feature map is convolved and normalized to obtain the second predicted feature map. Then, the second predicted feature map is convolved and normalized, and downsampled to obtain the third predicted feature map. This process is repeated until the (i-1)th predicted feature map is downsampled to obtain the ith predicted feature map.

[0049] Assuming the first predicted feature map has a resolution of H×W×C, after one conv-bn-relu operation (convolution, normalization layer, and ReLU activation function), a second predicted feature map with a resolution of H / 2×W / 2×128 is obtained. Subsequent conv-bn-relu operations are then performed, along with downsampling (e.g., maxpooling) to halve the size and double the number of channels, resulting in a third predicted feature map with a resolution of H / 4×W / 4×256, thus acquiring more spatial information. This process is repeated to obtain a fourth predicted feature map with a resolution of H / 8×W / 8×512, and so on, until the i-th predicted feature map at the target resolution is obtained (e.g., a fifth predicted feature map with a resolution of H / 16×W / 16×1024).

[0050] B. Reduce the number of channels in the i-th predicted feature map to 1 / 4 of the original number, and perform multi-scale max pooling operations in three separate paths. Then, combine the pooling results from the three paths with the feature map before max pooling to obtain the i-th intermediate predicted feature map.

[0051] This step primarily involves high-dimensional spatial information fusion for the multi-scale module. Multi-scale maintains the size of the i-th predicted feature map unchanged, reducing the number of channels to 1 / 4 of the original. For example, a feature map of H / 16×W / 16×1024 is transformed into a feature map of H / 16×W / 16×256. Then, it is divided into three paths, each undergoing multi-scale max-pooling. The main purpose of max-pooling here is to extract features using different pooling kernel sizes, increasing the network's receptive field, and fusing the three pooled feature maps with the original feature map before pooling. After fusing the four parts, a conv-bn-relu process is applied to obtain the i-th intermediate predicted feature map with the same resolution as the i-th predicted feature map. Figure 4 The resolution of the i-th intermediate predicted feature map generated in the process is H / 16×W / 16×1024, where i is a positive integer greater than 2.

[0052] C, upsample the i-th intermediate predicted feature map to obtain the (i-1)-th intermediate predicted feature map, and concatenate and fuse it with the (i-1)-th predicted feature map and perform upsampling operations to obtain the (i-2)-th intermediate predicted feature map.

[0053] The next step is to upsample the intermediate predicted feature maps sequentially and then stitch them together with predicted feature maps of the same resolution to integrate more image information.

[0054] from Figure 4 As can be seen, the H / 16×W / 16×1024 feature map first undergoes a convolution operation and upsampling, halving the number of channels to obtain the (i-1)th intermediate predicted feature map with a resolution of H / 8×W / 8×512. Then, it is concatenated and fused with the previous (i-1)th predicted feature map. After fusion, it undergoes a conv-bn-relu and upsampling operation to obtain the (i-2)th intermediate predicted feature map with a resolution of H / 4×W / 4×256.

[0055] D, based on the (i-2)th intermediate predicted feature map, continues to stitch, fuse and upsample with the corresponding predicted feature map according to the resolution until the mask map is obtained.

[0056] The number of concatenation and fusion operations depends on the number of downsampling (dimensionality reduction) operations. Therefore, upsampling aims to increase dimensionality, and fusion requires selecting concatenation operations with the same resolution. After the final concatenation and fusion, the size of the first intermediate predicted feature map is the same as the size of the first predicted feature map, for example, H×W, but the number of channels may differ from the original number of channels. For example... Figure 4 The resolution of the first intermediate predicted feature map is H×W×128. Compared to the original number of channels, after a final conv-bn-relu operation, a single-channel mask map with a resolution of H×W×1 is obtained. The mask map, by its very nature, can extract the region of interest (ROI) from the original image. Multiplying the image to be processed with a pre-made ROI mask yields the ROI image. Image values ​​within the ROI remain unchanged, while image values ​​outside the ROI are all 0. This improves the visibility of defective areas in the image. Furthermore, the mask map can also mask areas without defects, resulting in a more intuitive image display. The filtered background noise in the source image also facilitates the accuracy of subsequent classification networks.

[0057] Since the semantic segmentation network model has undergone extensive training in the early stages, it also has a certain ability to recognize mask images. At the same time as generating the mask image, the first defect prediction result seg_result of the mask image will also be output.

[0058] Step 202: Input the wafer defect map and mask map into the classification network prediction model for prediction to obtain the second defect prediction result.

[0059] The second prediction result is mainly determined by the classification network prediction model. Based on the obtained mask image, the mask image is further compared with the original wafer defect. Figure 1 The same input is fed into the classification network prediction model.

[0060] During processing, the classification network model performs a bitwise AND operation between the original image and the mask image. The resulting image is a mask applied to the background, completely or partially obscuring the graphic object, while the image within the mask remains unchanged. This produces a wafer defect image with pixel-level annotations based on the entire background. A classifier then performs image recognition to obtain the second defect prediction result, cls_result. In terms of accuracy, cls_result is more accurate than seg_result.

[0061] Step 203: Determine the final image prediction result based on the first defect prediction result and the second defect prediction result.

[0062] This method determines the final image prediction result based on the first defect prediction result and the second defect prediction result, using the following formula:

[0063] predict_result=α·seg_result+β·cls_result

[0064] predict_result represents the final prediction result of the image, where α and β are hyperparameters or weights, which are determined by the model accuracy.

[0065] In the initial training phase, each defect image is assigned a defect type label. After extensive model training, α and β are determined, and each determined defect type has a specific predict_result numerical range. For the input detection image, the model uses the predict_result numerical range to indicate the defect type on the wafer surface, and determines and outputs the defect type of the wafer defect image based on the numerical value.

[0066] In summary, this application combines a semantic segmentation network model and a classification learning network model. The original, complete wafer defect image is input into the semantic segmentation network model for layer-by-layer extraction. Max pooling and fusion operations are performed through a multi-scale module to transform low-level spatial information into high-level semantic information. Subsequently, upsampling and feature fusion techniques are used to generate a predicted mask image and the first defect prediction result using both high-level semantic and low-level spatial information. After obtaining the mask image, the mask image and the original wafer defect image are then further processed... Figure 1 The same data is input into the classification network prediction model. A pixel-level annotated wafer defect map with a complete background is obtained by superimposing the mask image and the original wafer defect map. Then, a classifier predicts the second defect prediction result. Finally, the final image prediction result is determined based on the two defect prediction values. Compared to the original direct annotation and small-image pixel-level annotation, this method filters out the influence of background noise on the classification model and improves the accuracy of the prediction results by using a mask image that includes the entire background.

[0067] The preferred embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above. The devices and structures not described in detail should be understood as being implemented in a conventional manner in the art. Any person skilled in the art can make many possible changes and modifications, or equivalent changes to equivalent embodiments without departing from the technical solution of the present invention. This does not affect the substantive content of the present invention. Therefore, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention shall still fall within the protection scope of the technical solution of the present invention.

Claims

1. A wafer surface defect detection method characterized by, The method comprises: inputting the collected wafer defect map into a semantic segmentation network model for pixel-level labeling to obtain a mask map and a first defect prediction result of the wafer defect map; wherein the wafer defect map contains only one defect type, and the semantic segmentation network model contains a multi-scale module for improving prediction accuracy; obtaining the mask map specifically comprises extracting prediction feature maps of the wafer defect map at different resolutions in sequence to obtain a first prediction feature map, a second prediction feature map, and an i-th prediction feature map at a target resolution; reducing the number of channels of the i-th prediction feature map to 1 / 4 of the original number, and performing multi-scale maximum pooling operations on three paths respectively, splicing and fusing the three-path pooling results together with the feature map before maximum pooling to obtain an i-th intermediate prediction feature map; performing up-sampling on the i-th intermediate prediction feature map to obtain an i-1-th intermediate prediction feature map, and performing splicing and fusing and up-sampling operations on the i-1-th intermediate prediction feature map and the i-1-th prediction feature map to obtain an i-2-th intermediate prediction feature map; continuing to perform splicing and fusing and up-sampling on the i-2-th intermediate prediction feature map and the corresponding prediction feature map based on the resolution until the mask map is obtained; i is a positive integer greater than 2; inputting the wafer defect map and the mask map into a classification network prediction model for prediction to obtain a second defect prediction result; determining a final prediction result of the picture based on the first defect prediction result and the second defect prediction result; the first defect prediction result is seg_result; and the second defect prediction result is cls_result; the final prediction result of the picture is predict_result, and is represented as follows: predict_result = a seg_result + b cls_result wherein a and b are hyperparameters determined based on model accuracy; the numerical interval of predict_result indicates the defect type of the wafer surface, and the defect type of the wafer defect map is determined and output according to the numerical size.

2. The method of claim 1, wherein, The resolution and size of the k-th prediction feature map and the k-th intermediate feature map are the same, and k is between 1 and i; the size includes the length, width and number of channels of the feature map, and the resolution is positively correlated with the size of the image; the extracting of the prediction feature maps of the wafer defect map at different resolutions in sequence comprises: performing convolution and normalization operations on the first prediction feature map to obtain the second prediction feature map; performing convolution and normalization operations on the second prediction feature map and down-sampling to obtain a third prediction feature map, and performing down-sampling on the i-1-th prediction feature map to obtain the i-th prediction feature map.

3. The method of claim 2, wherein, the splicing and fusing of the three-path maximum pooling operation results together with the feature map before maximum pooling to obtain the i-th intermediate prediction feature map comprises: splicing the feature map after the three-path maximum pooling operation and the feature map before the pooling operation, performing convolution and normalization operations on the spliced feature map to obtain the i-th intermediate prediction feature map, and the resolution of the i-th intermediate prediction feature map is the same as that of the i-th prediction feature map.

4. The method of claim 3, wherein, On the basis of the i-2 intermediate prediction feature map, continue to perform splicing fusion and upsampling according to the resolution and the corresponding prediction feature map until the mask map is obtained, comprising: The intermediate prediction feature map is sequentially spliced and fused with the prediction feature map corresponding to the resolution, and convolution and upsampling operations are performed on the fusion until the first intermediate prediction feature map is obtained; the length and width of the first intermediate prediction feature map are the same as those of the first prediction feature map; Continue to perform convolution operation on the first intermediate prediction feature map to obtain the mask map with the same resolution as the first prediction feature map.

5. The method of claim 4, wherein, The wafer defect map and the mask map are input into a classification network prediction model for prediction to obtain a second defect prediction result, comprising: The classification network prediction model overlaps and matches the mask map and the wafer defect map to obtain a defect image; The defect image is identified to obtain the second defect prediction result.

6. The method of claim 1, wherein, The numerical interval of predict_result indicates the defect type on the wafer surface, and the defect type of the wafer defect map is judged and output according to the numerical size.

Images