A wafer surface defect extraction method based on Mask R-CNN and graph convolution

Abstract

For wafer surface defect segmentation and extraction, this invention proposes an MG-Net method based on Mask R-CNN and graph convolutional inference. After wafer cleaning and mounting, high-resolution images are acquired using a bright / dark field microscope. These images undergo adaptive Gamma correction and nonlocal mean denoising, followed by instance-level boundary labeling and block normalization. Multi-scale features are extracted using ResNet-101+FPN, and candidate features are generated via RPN and RoIAlign. A defect association graph convolutional module (spatial distance + feature similarity) then identifies morphologically similar defects, and FCN generates pixel-level masks. A small-object weighted loss is used, fused with Focal loss and mask boundary loss for training. After defect segmentation, the model performs mask fusion and connected component analysis, outputting defect counts, size distribution, and wafer image distribution, achieving segmentation, extraction, and quantitative characterization of various defects on wafers of different specifications.

Description

Technical Field

[0001] This invention relates to the field of semiconductor wafer defect detection technology, specifically to a wafer surface defect segmentation and extraction method based on Mask R-CNN and graph convolutional inference network. Background Technology

[0002] Wafer surface defect detection is a core component of yield control in semiconductor manufacturing. The morphology, size, quantity, and spatial distribution of surface defects such as particles, scratches, pits, contamination, and pattern missing parts directly determine the yield and reliability of chip manufacturing. As integrated circuit process nodes continue to shrink, the size of wafer surface defects has entered the sub-micron and even nanometer scale. Furthermore, the distribution of defects on wafers exhibits strong randomness, uneven density, and correlation with process fluctuations, posing significant challenges to accurate defect detection and quantitative analysis. Accurately segmenting, extracting, and statistically analyzing wafer surface defects is of significant scientific and engineering value for yield analysis, process improvement, and failure tracing. Traditional wafer defect detection methods typically employ manual visual inspection combined with microscopic verification for defect identification and statistics, which suffers from low efficiency, strong subjectivity, and a high risk of missed detections. In recent years, researchers have introduced traditional image processing techniques such as threshold segmentation, template matching, and differential comparison for defect extraction. However, these methods lack robustness against nanoscale defects, low-contrast defects, and interference from complex circuit pattern backgrounds, making it difficult to achieve automated, high-precision defect segmentation. With the development of deep learning technology, target detection and instance segmentation networks such as Faster R-CNN, YOLO, and Mask R-CNN have achieved significant results in general visual scenarios and have been gradually introduced into industrial surface defect detection. However, wafer surface defects are characterized by small scale, similar inter-class morphology, and spatial clustering and process association of defects within the same field of view. Single-structure detection or segmentation networks still have the following shortcomings: (1) The features of small defects are easily lost during deep downsampling, resulting in a high rate of missed detection of small target defects; (2) The detection process isolates each defect instance and does not utilize the spatial and process association information between defect instances within the same field of view, making it easy for defects of similar morphology to be confused with each other; (3) The proportion of defect pixels in the whole image is extremely low, and the positive and negative samples are seriously unbalanced, resulting in inaccurate mask boundaries. Therefore, there is an urgent need for an automatic segmentation and extraction method that is tailored to the characteristics of wafer surface defects and has the ability to express small targets at multiple scales and infer instances. Summary of the Invention

[0003] To address the limitations of existing technologies in automatically and accurately segmenting and extracting wafer surface defects, such as interference from circuit pattern backgrounds in wafer images, loss of minute defect features, and easy confusion between defects with similar shapes, the present invention aims to provide a wafer surface defect segmentation and extraction method based on Mask R-CNN and graph convolutional inference networks. The specific technical solution adopted is as follows: A Mask R-CNN-based method... A wafer surface defect segmentation and extraction method based on R-CNN and graph convolutional inference networks includes: surface cleaning and vacuum mounting of the wafer; acquiring high-resolution optical images of the wafer surface using a wafer defect detection microscope with combined bright-field and dark-field illumination; performing adaptive Gamma correction and nonlocal mean denoising data enhancement on the acquired images based on the wafer surface reflection characteristics; performing instance-level manual annotation on the enhanced images along the defect boundaries to obtain a reference image dataset; performing multi-scale slicing and normalization processing on the reference image dataset to obtain a dataset; designing a multi-scale feature extraction backbone based on ResNet-101 and feature pyramid networks to convert input image information into tensors to extract multi-scale defect features; designing a candidate defect region generation module based on a region proposal network to generate candidate defect boxes; and designing defect instances based on RoIAlign. The system includes: a feature alignment module; a defect association graph convolutional inference module (comprising a graph node construction submodule and a graph convolutional inference submodule) for modeling spatial and process relationships between defect instances; a pixel-level mask generation module based on a fully convolutional network for outputting defect instance masks; fusion to obtain an MG-Net fusion model; a small-target weighted loss function designed to address the low proportion and similar morphology of small-target defects, resulting in loss function weights for small-target defects; multi-task supervision calculation based on the loss function weights to obtain model feedback; training based on the MG-Net fusion model to obtain a trained model; using the trained model to segment and extract defect instances from optical images of the wafer surface; mask fusion and connected component analysis of the segmentation and extraction results to obtain defect image results; and defect counting, size distribution statistics, and wafer image distribution data analysis of the defect image results.

[0004] Furthermore, the wafer fabrication and image acquisition method includes: selecting an 8-inch or 12-inch wafer sample, ultrasonically cleaning it for no less than 10 minutes with a standard cleaning solution of ammonia-hydrogen peroxide-deionized water, drying it with nitrogen, and then placing it on a vacuum adsorption stage to complete the mounting; using an automated wafer defect detection microscope with bright-field and dark-field composite illumination, scanning and acquiring optical images of the wafer surface with a pixel resolution of no less than 4096×3000 pixels per field of view at a magnification of 20x to 100x objective lens, with an overlap rate of no less than 10% between adjacent fields of view for image stitching.

[0005] Further, the data augmentation and annotation method includes: performing data augmentation on the acquired image according to an adaptive Gamma correction formula, the Gamma correction formula being as follows: I_out(x,y) = 255×(I_in(x,y) / 255)^γ, γ =log(0.5) / log(I_mean / 255), where I_out(x,y) represents the grayscale value of the data-augmented image; I_in(x,y) represents the pixel grayscale value of the input image at (x,y); I_mean represents the average grayscale value of the input image; γ represents the adaptive correction Gamma coefficient; performing denoising processing on the augmented image using non-local mean filtering; manually annotating each defect instance in the denoised image along the defect boundary to generate instance-level reference image data; and performing multi-scale sliding slicing of the reference image into 1024×1024 pixel blocks and linearly mapping the pixel values ​​to the [0,1] interval to obtain a standard dataset.

[0006] Furthermore, the method for obtaining the MG-Net fusion model includes: using a ResNet-101 pre-trained on ImageNet as the backbone network, connecting it to a feature pyramid network, reshaping the input image into a 1024×1024×3 tensor, and extracting defect feature maps at five scales from P2 to P6 from bottom to top; the region proposal network slides across the feature maps at each scale to generate anchor boxes with different aspect ratios, and candidate defect regions are obtained through foreground classification and bounding box regression; RoIAlign is used to align the features of the candidate defect regions to fixed-size features of 14×14; and a defect association map rollup is designed. The inference module uses candidate defect instances within the same field of view as graph nodes. It constructs an adjacency matrix based on the spatial distance and feature similarity between instances, and propagates the association information between nodes through two layers of graph convolution to achieve the association inference of defect categories. A pixel-level mask generation module is built based on a fully convolutional network. The aligned features are upsampled by four 3×3 convolutions and one deconvolution, and a 28×28 defect instance mask is output. The multi-scale feature extraction backbone, candidate defect region generation module, defect instance feature alignment module, defect association graph convolutional inference module and pixel-level mask generation module are fused to obtain the MG-Net fusion model.

[0007] Furthermore, the method for obtaining the small target weighted loss function includes: addressing the problems of extremely low pixel proportion of small target defects, similar defect categories and morphologies, and severe imbalance between positive and negative samples, introducing a small target scale weighting factor based on Focal loss, and combining it with mask boundary loss to obtain the small target weighted loss function, calculated as follows: L_cls = -Σw_s·(1-p_t)^γ_f·log(p_t), w_s = 1 + log(S_max / S_i), L_box = SmoothL1(b_pred,b_gt), L_mask = BCE(M_pred, M_gt), L_edge = BCE(|Sobel(M_pred)|, |Sobel(M_gt)|), L = λ1·L_cls + λ2·L_box + λ3·L_mask + λ4·L_edge, where p_t is the predicted class probability of the defect instance; (1-p_t)^γ_f is the Focal modulation factor, γ_f takes a value of 2; w_s is the small target scale weighting factor; S_i is the area of ​​the i-th defect instance; S_max is the upper limit of the area of ​​defect instances in the dataset; b_pred and b_gt are the predicted and ground truth bounding boxes; L_box is the Smooth L1 loss for bounding box regression; M_pred is the predicted defect mask; M_gt is the ground truth defect mask; Sobel(·) is the differentiable Sobel edge extraction operation; L_edge is the mask boundary loss; λ1, λ2, λ3, and λ4 are multi-task balancing coefficients; the loss function is used to perform multi-task supervision calculation on the MG-Net fusion model to obtain model feedback.

[0008] Furthermore, the model training method includes: using defect region pixels as foreground targets and normal wafer surface region pixels as backgrounds; initializing the multi-scale feature extraction backbone with ResNet-101 weights pre-trained on ImageNet and performing transfer learning; freezing the shallow stages of the backbone network for warm-up training during training, and then unfreezing all parameters for fine-tuning; setting the initial learning rate to 1×10^-3 and dynamically adjusting it according to a step decay strategy; setting the batch size to 4 and iterating for at least 150 epochs; stopping training when the average precision of the validation set no longer increases for 15 consecutive epochs and saving the optimal model weights.

[0009] Furthermore, the mask fusion and connected component analysis method includes: filtering and fusing the defect instance masks output by the model according to a confidence threshold and mapping them to the entire field of view image; using morphological closing operations to fill the small holes inside the mask and smooth the defect boundaries; marking each defect instance according to connected component analysis and removing pseudo-defect regions with an area smaller than a preset threshold; calculating the equivalent circle diameter, aspect ratio, and centroid coordinates of each defect instance; mapping the defect centroid to the wafer coordinate system to generate a wafer defect distribution map, statistically analyzing the total number of defects, size distribution, and spatial clustering characteristics, and obtaining the final defect image results and statistical parameters.

[0010] This invention offers the following superior advantages: By performing standard cleaning and vacuum adsorption mounting on wafers, optical images of the wafer surface are acquired using a detection microscope with combined bright-field and dark-field illumination. The sample field of view exhibits rich defect morphology and scale variations, providing ample samples for deep learning models. Due to the uneven brightness of the acquired wafer images influenced by surface reflection characteristics, adaptive Gamma correction and nonlocal mean denoising are employed for image data augmentation, improving model generalization ability while enriching data diversity. Instance-level pixel annotations are performed along defect boundaries, converting the images into instance-level reference images. Multi-scale segmentation and normalization of the image data reduce computational load and improve training efficiency, resulting in a standard dataset. Given the small scale and diverse morphology of wafer surface defects, a multi-scale feature extraction backbone is constructed using ResNet-101 and a feature pyramid network to enhance the shallow detail representation of small defects. Region proposal networks and RoIAlign are used to obtain… Alignment features of candidate defect regions are obtained. Due to spatial clustering and process correlation among defect instances within the same field of view, a defect association graph convolutional inference module is designed. This module uses defect instances as graph nodes and constructs an adjacency matrix based on spatial distance and feature similarity. Association information is propagated through graph convolution, improving the ability to distinguish morphologically similar defects. A fully convolutional mask generation module outputs pixel-level defect masks. Because small target defects account for a very low proportion and positive and negative samples are severely imbalanced, a small target weighted loss function is designed. Focal modulation factors and scale weighting factors are introduced, combined with mask boundary loss for multi-task supervision, improving the detection rate of small targets and the accuracy of mask boundaries without sacrificing recall. The trained model is used to segment instances in the wafer surface optical image. Since the model segmentation results may contain minor noise and holes, morphological closing operations and connected component analysis are used for post-processing to obtain complete defect image results. Finally, statistical and data analysis is performed on defect counting, size distribution, and wafer image distribution in the defect images. This invention fully considers the complexity of wafer surface defects in terms of scale, morphology, and distribution. It designs a unique multi-scale feature extraction backbone, candidate region generation, instance feature alignment, and defect association graph convolutional inference module, which enhances the small target expression ability and instance association inference ability of deep learning segmentation model, improves the completeness, accuracy, and category discrimination reliability of defect segmentation, and provides a reliable technical means for semiconductor manufacturing yield analysis and process improvement. Attached Figure Description

[0011] To more clearly illustrate the solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Figure 1This is a flowchart of a wafer surface defect segmentation and extraction method based on Mask R-CNN and graph convolutional inference network, provided by an embodiment of the present invention. Figure 2 This is a network structure diagram of the MG-Net fusion model provided in one embodiment of the present invention. Detailed Implementation

[0012] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of a wafer surface defect segmentation and extraction method based on Mask R-CNN and graph convolutional inference networks proposed according to the present invention. In the following description, different embodiments or one embodiment do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The specific scheme of a wafer surface defect segmentation and extraction method based on Mask R-CNN and graph convolutional inference networks provided by the present invention is described in detail below with reference to the accompanying drawings.

[0013] Please see Figure 1 This paper illustrates a wafer surface defect segmentation and extraction method based on Mask R-CNN and graph convolutional inference network according to an embodiment of the present invention. The method includes the following steps.

[0014] Step S1: The wafer undergoes surface cleaning and vacuum mounting. High-resolution optical images of the wafer surface are acquired using a wafer defect inspection microscope with combined bright-field and dark-field illumination. This embodiment of the invention is mainly applied to the segmentation of surface defects on 8-inch and 12-inch wafers. To obtain wafer surface defect image data, the wafer sample is first cleaned and mounted according to standard procedures, and then surface images are acquired by scanning each field of view using an automated inspection microscope. Since this embodiment of the invention uses a deep learning model for defect segmentation, the model network requires a large number of image samples containing defect morphologies to ensure that the model has sufficient generalization ability. Preferably, in one embodiment of the present invention, the wafer fabrication and image acquisition method includes: selecting a 12-inch wafer sample, ultrasonically cleaning it for 15 minutes with SC-1 standard cleaning solution (ammonia, hydrogen peroxide and deionized water mixed in a volume ratio of 1:1:5), drying it with nitrogen gas, and placing it on a vacuum adsorption stage; using an automated wafer defect detection microscope with bright-field and dark-field composite illumination, scanning and acquiring optical images of the wafer surface one field of view at a 50x objective magnification, with a single field of view pixel resolution of 4096×3000 pixels, and an adjacent field of view overlap rate of 15% for image stitching, acquiring no less than 200 fields of view for each wafer to ensure the diversity and representativeness of the dataset.

[0015] Step S2: Based on the wafer surface reflection characteristics, adaptive Gamma correction and nonlocal mean denoising data augmentation are applied to the acquired images; instance-level manual annotation is performed on the augmented images along the defect boundaries to obtain a reference image dataset; multi-scale slicing and normalization processing are performed on the reference image dataset to obtain a new dataset. Since the acquired wafer images exhibit brightness and reflection differences across fields of view, adaptive Gamma correction is used for image data augmentation, improving model generalization ability while enriching data diversity; due to significant differences in grayscale distribution between defect areas and normal circuit pattern areas, pixel-by-pixel instance-level annotation is performed manually along the defect boundaries; since defect segmentation and extraction only require focusing on relative grayscale differences, image normalization is performed to reduce computational load and improve training efficiency, resulting in a standard dataset. Preferably, in one embodiment of the present invention, the data augmentation method includes: enhancing the image according to the adaptive Gamma correction formula, as shown below: I_out(x,y) = 255×(I_in(x,y) / 255)^γ, γ = log(0.5) / log(I_mean / 255), where I_mean is the average gray level of the image; using non-local mean filtering with a search window of 21×21 and similar blocks of 7×7 to suppress image noise; expanding the dataset to 8-10 times its original size by performing horizontal flipping, vertical flipping, 90° / 180° / 270° rotation, and random translation on each original image; using the Labelme annotation tool to perform instance-level pixel annotation along the defect boundary, labeling the defect categories as particles, scratches, pits, and contamination, and generating instance mask reference images; sliding the image into 1024×1024 pixel blocks and adjusting all pixel values ​​according to I_norm = (I - I_min) / (I_max) -I_min) is linearly mapped to the interval [0,1] and randomly divided into training set, validation set and test set in a ratio of 8:1:1.

[0016] Step S3: Design a multi-scale feature extraction backbone based on ResNet-101 and a feature pyramid network; design a candidate defect region generation module based on a region proposal network; design a defect instance feature alignment module based on RoIAlign; design a defect association graph convolutional inference module; design a pixel-level mask generation module based on a fully convolutional network, and fuse them to obtain the MG-Net fusion model. In one embodiment of the present invention, the method for obtaining the MG-Net fusion model includes: extracting multi-scale detailed features of defects using a multi-scale feature extraction backbone; obtaining defect candidate boxes using a candidate defect region generation module; obtaining fixed-size instance features through a defect instance feature alignment module; modeling the association between instances and completing category inference through a defect association graph convolutional inference module; and outputting defect instance masks using a pixel-level mask generation module. Preferably, in one embodiment of the present invention, the multi-scale feature extraction backbone design method includes: the input layer reshaping the input image into a 1024×1024×3 tensor; using ResNet-101 pre-trained on ImageNet as the backbone network, extracting features sequentially through five stages: conv1, layer1, layer2, layer3, and layer4; connecting the outputs of each stage to the feature pyramid network, smoothing them through lateral connections and 3×3 convolutions, and outputting defect feature maps at five scales from P2 to P6, with each scale having 256 feature channels. Preferably, in one embodiment of the present invention, the design method of the defect association graph convolutional inference module includes: the region proposal network generates anchor boxes with three aspect ratios of 32×32 to 512×512 on the feature maps P2 to P6, and obtains the top 512 candidate defect regions through foreground classification and bounding box regression; the candidate region features are aligned to a fixed size of 14×14×256 by bilinear interpolation using RoIAlign; the candidate defect instances in the same field of view are used as graph nodes, and the node features are 256-dimensional vectors after global average pooling of the aligned features. The adjacency matrix is ​​constructed by weighting the centroid spatial distance between instances and the cosine similarity of features. The association information is propagated between nodes through two layers of graph convolution. The graph convolution output is added to the original feature residual of the node and then sent to the classification and bounding box regression head to complete the defect category association inference; the pixel-level mask generation module performs four 3×3 convolutions and one 2x deconvolution upsampling on the aligned features to output a 28×28 defect instance mask, which is then fused to obtain the MG-Net fusion model.

[0017] Step S4: Design a weighted loss function for small targets, considering their low defect ratio and similar morphological categories, to obtain the loss function weights for small target defects. Perform multi-task supervised calculations based on these weights to obtain model feedback. The grayscale and structural information of the defect region and the normal surface region are obtained through the above process. Because the pixel ratio of small target defects is extremely low, the grayscale difference between them and the background of the circuit pattern is not significant, and they are easily affected by texture interference, a weighted loss function for small targets is designed and calculated based on the above feature information to obtain model feedback. Preferably, in one embodiment of the present invention, the model feedback acquisition method includes: obtaining the small target defect loss weights according to the small target weighted loss function calculation formula, and calculating the model feedback, the calculation formula being as follows: L_cls = -Σ w_s·(1-p_t)^γ_f·log(p_t), w_s = 1 + log(S_max / S_i), L_box = SmoothL1(b_pred, b_gt), L_mask = BCE(M_pred, M_gt), L_edge =BCE(|Sobel(M_pred)|, |Sobel(M_gt)|), L = λ1·L_cls + λ2·L_box + λ3·L_mask +λ4·L_edge, where γ_f is 2; w_s is the small target scale weighting factor; S_i is the area of ​​the i-th defect instance; S_max is 1024 pixels; λ1, λ2, λ3, and λ4 are 1.0, 1.0, 1.0, and 0.5 respectively; joint multi-task supervision calculation is performed on the four tasks of classification, regression, masking, and boundary to obtain model feedback.

[0018] Step S5: Train the model according to the MG-Net fusion model to obtain a trained model; use the trained model to segment and extract defect instances from the optical image of the wafer surface; perform mask fusion and connected component analysis on the segmentation and extraction results to obtain defect image results; perform defect counting, size distribution statistics, and wafer image distribution data analysis on the defect image results. Preferably, in one embodiment of the present invention, training the model according to the MG-Net fusion model includes: using a transfer learning strategy, using ResNet-101 weights pre-trained on ImageNet to initialize the multi-scale feature extraction backbone; during training, first freeze the backbone network layer1 and layer2 for 30 epochs of warm-up training, then unfreeze all parameters for fine-tuning; the initial learning rate is set to 1×10^-3, and decays stepwise at a rate of 0.1 at the 80th and 120th epochs; the batch size is set to 4, using the SGD optimizer with a momentum of 0.9 and a weight decay coefficient of 1×10^-4; iteratively train for 150 epochs, and stop training when the mean accuracy of the validation set no longer increases for 15 consecutive epochs, saving the optimal model weights. The trained model is used to segment optical images of the wafer surface, minimizing incomplete segmentation results and category confusion caused by the similarity between defects and circuit patterns, thereby improving segmentation accuracy and completing the segmentation of wafer surface defects. Preferably, in one embodiment of the present invention, the mask fusion and connected component analysis method includes: filtering the defect instance mask output by the model with a confidence threshold of 0.5, removing duplicate detections through non-maximum suppression, and then fusing and mapping it to the entire field-of-view image; performing morphological closing operations using 3×3 circular structuring elements to fill small holes inside the mask and smooth defect boundaries; labeling each defect instance according to connected component analysis and removing pseudo-defect regions with an area less than 9 pixels; calculating the equivalent circle diameter, aspect ratio, and centroid coordinates of each defect instance; stitching the images of each field of view according to the acquisition coordinates, and mapping the defect centroid to the wafer coordinate system to generate a wafer map, statistically analyzing the total number of defects, the size distribution and spatial aggregation characteristics of the four types of defects, and calculating quantitative indicators such as defect density per unit area and radial distribution entropy. Thus, the segmentation and extraction of wafer surface defects is completed.

[0019] In summary, the wafer surface is cleaned and vacuum-mounted. High-resolution optical images of the wafer surface are acquired using a wafer defect inspection microscope with combined bright-field and dark-field illumination. Adaptive Gamma correction and nonlocal mean denoising data enhancement are applied to the acquired images based on the wafer surface reflection characteristics. Instance-level manual annotation is performed on the enhanced images along the defect boundaries to obtain a reference image dataset. Multi-scale slicing and normalization are performed on the reference image dataset to obtain a new dataset. A multi-scale feature extraction backbone is designed based on ResNet-101 and a feature pyramid network. A candidate defect region generation module is designed based on a region proposal network. Defects are designed... This invention comprises a convolutional inference module for correlation graphs; a pixel-level mask generation module based on a fully convolutional network, which is then fused to obtain an MG-Net fusion model; a small-object weighted loss function designed to address the low proportion of small-object defects and their similar morphological categories; multi-task supervision calculation based on the loss function weights to obtain model feedback; training based on the MG-Net fusion model to obtain a trained model; using the trained model to segment and extract defect instances from optical images of wafer surfaces; performing mask fusion and connected component analysis on the segmentation and extraction results to obtain defect image results; and performing defect counting, size distribution statistics, and wafer image distribution data analysis on the defect image results. This invention can segment, extract, and quantitatively characterize surface defects such as particles, scratches, pits, and contaminants on wafers of different sizes, such as 8-inch and 12-inch wafers, providing technical support for semiconductor manufacturing yield analysis and process improvement.

[0020] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous. The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments.

Claims

1. A method for wafer surface defect segmentation and extraction based on Mask R-CNN and graph convolutional inference network, characterized in that, The method includes: surface cleaning and vacuum adsorption mounting of the wafer; acquiring high-resolution optical images of the wafer surface using a wafer defect detection microscope with combined bright-field and dark-field illumination; performing adaptive Gamma correction and nonlocal mean denoising data enhancement on the acquired images based on the wafer surface reflection characteristics; performing instance-level manual annotation on the enhanced images along the defect boundaries to obtain a reference image dataset; performing multi-scale slicing and normalization processing on the reference image dataset to obtain a dataset; designing a multi-scale feature extraction backbone based on ResNet-101 and feature pyramid network to convert input image information into tensors to extract multi-scale defect features; designing a candidate defect region generation module based on region proposal network to generate candidate defect boxes; designing a defect instance feature alignment module based on RoIAlign; and designing defect association. The graph convolutional inference module includes a graph node construction submodule and a graph convolutional inference submodule, used to model the spatial and process relationships between defect instances; a pixel-level mask generation module is designed based on a fully convolutional network to output defect instance masks; an MG-Net fusion model is obtained through fusion; a small target weighted loss function is designed to address the low proportion of small target defects and their similar class morphology, obtaining the loss function weights for small target defects; multi-task supervision calculation is performed based on the loss function weights to obtain model feedback; the MG-Net fusion model is trained to obtain a trained model; the trained model is used to segment and extract defect instances from the wafer surface optical image; mask fusion and connected component analysis are performed on the segmentation and extraction results to obtain defect image results; and defect counting, size distribution statistics, and wafer image distribution data analysis are performed on the defect image results.

2. The wafer surface defect segmentation and extraction method based on Mask R-CNN and graph convolutional inference network according to claim 1, characterized in that, The wafer fabrication and image acquisition method includes: selecting an 8-inch or 12-inch wafer sample, ultrasonically cleaning it for no less than 10 minutes with a standard cleaning solution of ammonia-hydrogen peroxide-deionized water, drying it with nitrogen, and then placing it on a vacuum adsorption stage to complete the mounting; using an automated wafer defect detection microscope with bright-field and dark-field combined illumination, scanning and acquiring optical images of the wafer surface with a pixel resolution of no less than 4096×3000 pixels per field of view at a magnification of 20x to 100x objective lens, with an overlap rate of no less than 10% between adjacent fields of view for image stitching.

3. The wafer surface defect segmentation and extraction method based on Mask R-CNN and graph convolutional inference network according to claim 1, characterized in that, The data augmentation and annotation method includes: performing data augmentation on the acquired image according to the adaptive Gamma correction formula, which is as follows: I_out(x,y) = 255×(I_in(x,y) / 255)^γ, γ= log(0.5) / log(I_mean / 255), where I_out(x,y) represents the gray value of the image after data augmentation; I_in(x,y) represents the pixel gray value of the input image at (x,y); I_mean represents the average gray value of the input image; and γ represents the adaptive correction Gamma coefficient; performing denoising processing on the augmented image using non-local mean filtering; manually annotating each defect instance in the denoised image along the defect boundary to generate instance-level reference image data; and performing multi-scale sliding slicing of the reference image into 1024×1024 pixel blocks and linearly mapping the pixel values ​​to the [0,1] interval to obtain a standard dataset.

4. The wafer surface defect segmentation and extraction method based on Mask R-CNN and graph convolutional inference network according to claim 1, characterized in that, The method for obtaining the MG-Net fusion model includes: using ResNet-101 pre-trained on ImageNet as the backbone network, connecting it to a feature pyramid network, reshaping the input image into a 1024×1024×3 tensor, and extracting defect feature maps at five scales from P2 to P6 from bottom to top; the region proposal network slides across the feature maps at each scale to generate anchor boxes with different aspect ratios, and obtains candidate defect regions through foreground classification and bounding box regression; using RoIAlign to align the features of the candidate defect regions to fixed-size features of 14×14; and designing convolutional inference for the defect association graph. The module takes candidate defect instances within the same field of view as graph nodes, constructs an adjacency matrix based on spatial distance and feature similarity between instances, and propagates association information between nodes through two layers of graph convolution to achieve association inference of defect categories; a pixel-level mask generation module is constructed based on a fully convolutional network, which performs four 3×3 convolutions and one deconvolution upsampling on the aligned features to output a 28×28 defect instance mask; the multi-scale feature extraction backbone, candidate defect region generation module, defect instance feature alignment module, defect association graph convolutional inference module and pixel-level mask generation module are fused to obtain the MG-Net fusion model.

5. The wafer surface defect segmentation and extraction method based on Mask R-CNN and graph convolutional inference network according to claim 1, characterized in that, The method for obtaining the small target weighted loss function includes: addressing the issues of extremely low pixel proportions of small target defects, similar defect categories and morphologies, and severe imbalance between positive and negative samples, introducing a small target scale weighting factor based on Focal loss, and combining it with mask boundary loss to obtain the small target weighted loss function. The calculation formula is as follows: L_cls =-Σ w_s·(1-p_t)^γ_f·log(p_t), w_s = 1 + log(S_max / S_i), L_box = SmoothL1(b_pred, b_gt), L_mask = BCE(M_pred, M_gt), L_edge = BCE(|Sobel(M_pred)|, |Sobel(M_gt)|), L = λ1·L_cls + λ2·L_box + λ3·L_mask + λ4·L_edge, where p_t is the predicted class probability of the defect instance; (1-p_t)^γ_f is the Focal modulation factor, γ_f takes a value of 2; w_s is the small target scale weighting factor; S_i is the area of ​​the i-th defect instance; S_max is the upper limit of the area of ​​defect instances in the dataset; b_pred and b_gt are the predicted and ground truth bounding boxes; L_box is the Smooth L1 loss for bounding box regression; M_pred is the predicted defect mask; M_gt is the ground truth defect mask; Sobel(·) is the differentiable Sobel edge extraction operation; L_edge is the mask boundary loss; λ1, λ2, λ3, and λ4 are multi-task balancing coefficients; the loss function is used to perform multi-task supervision calculation on the MG-Net fusion model to obtain model feedback.

6. The wafer surface defect segmentation and extraction method based on Mask R-CNN and graph convolutional inference network according to claim 1, characterized in that, The model training method includes: using defective region pixels as foreground targets and normal wafer surface region pixels as backgrounds; initializing the multi-scale feature extraction backbone with ResNet-101 weights pre-trained on ImageNet and performing transfer learning; freezing the shallow stages of the backbone network for pre-training during training, and then unfreezing all parameters for fine-tuning; setting the initial learning rate to 1×10^-3 and dynamically adjusting it according to a step decay strategy; setting the batch size to 4 and iterating for at least 150 epochs; stopping training when the average precision of the validation set no longer increases for 15 consecutive epochs and saving the optimal model weights.

7. The wafer surface defect segmentation and extraction method based on Mask R-CNN and graph convolutional inference network according to claim 1, characterized in that, The mask fusion and connected component analysis method includes: filtering and fusing the defect instance masks output by the model according to a confidence threshold and mapping them to the entire field of view image; using morphological closing operations to fill the small holes inside the mask and smooth the defect boundaries; marking each defect instance according to connected component analysis and removing pseudo-defect regions with an area smaller than a preset threshold; calculating the equivalent circle diameter, aspect ratio, and centroid coordinates of each defect instance; mapping the defect centroid to the wafer coordinate system to generate a wafer defect distribution map, statistically analyzing the total number of defects, size distribution, and spatial clustering characteristics, and obtaining the final defect image results and statistical parameters.

Images