A few-sample wafer defect image background removal and data enhancement method based on space-frequency fusion

Abstract

The application discloses a few-sample wafer defect image background removing and data enhancement method based on frequency-space fusion, relates to the technical field of wafer defect detection, and can be applied to wafer defect identification, small-sample machine learning model training and industrial visual detection system construction in a semiconductor manufacturing process. In order to solve the problems of few-sample wafer defect sample scarcity, incomplete process texture background suppression, low-quality synthetic defect sample and the like in the prior art, the application realizes three core stages of background removing and defect enhancement by constructing a few-sample wafer defect library and adopting an FSF-TBS module to complete few-sample wafer image defect transplantation, accurately extracts wafer defect features, efficiently suppresses and removes process background, and simultaneously generates high-quality defect enhancement samples. The application can significantly improve the signal-to-noise ratio of a wafer defect image, effectively alleviate the plight of defect sample shortage, improve defect identification precision under small-sample conditions, and is suitable for semiconductor industrial visual detection and small-sample model training scenes.

Description

Technical Field

[0001] This invention relates to the field of wafer defect detection technology, and in particular to a method for background removal and data enhancement of few-sample wafer defect images based on frequency-space fusion, which can be applied to wafer defect identification in semiconductor manufacturing processes, training of few-sample machine learning models, and construction of industrial vision inspection systems. Background Technology

[0002] As the core carrier of semiconductor chips, the detection of surface defects in wafers is a key step in ensuring chip yield.

[0003] With the widespread application of deep learning in industrial inspection, data-driven wafer defect segmentation or recognition models require a large number of labeled samples for training. However, in real-world industrial scenarios, defect-free wafer samples are readily available, while defect samples (such as scratches, particles, dents, etc.) are scarce, creating a typical few-sample learning dilemma.

[0004] Existing wafer defect data enhancement methods are mostly limited to geometric transformations or grayscale perturbations in the spatial domain, and have not designed targeted background removal mechanisms for the highly structured and repetitive process texture backgrounds of wafer images.

[0005] Some methods incorporate frequency domain information, but only use frequency features as auxiliary features, failing to achieve deep interactive fusion of frequency and spatial features. This results in incomplete suppression of process textures, easily misjudging similar textures as defects, and producing low-quality synthetic samples that cannot meet the training requirements of few-sample defect recognition models.

[0006] Therefore, this invention provides a method for processing few-sample wafer defects that combines frequency domain features and spatial domain features, thereby achieving accurate removal of process background and generation of high-quality defect samples. Summary of the Invention

[0007] To address the shortcomings of existing technologies, this invention provides a method for background removal and data enhancement of few-sample wafer defect images based on frequency-space fusion, solving the technical problems of scarce few-sample wafer defect samples, incomplete suppression of process texture backgrounds, and low quality of synthesized samples in existing technologies. This method achieves accurate extraction of wafer defect features, efficient removal of process backgrounds, and generation of high-quality defect samples through three main stages: construction of a few-sample wafer defect library, background removal and defect enhancement using the FSF-TBS module, and defect transplantation from few-sample wafer images.

[0008] The technical solution of this invention is: a method for background removal and data enhancement of few-sample wafer defect images based on frequency-space fusion, comprising the following steps:

[0009] S1) Construction of a small-sample wafer defect library

[0010] To address the issue of extremely limited sample numbers for each type of wafer defect, instead of directly using original defect samples for model training, we first decouple and purify the features of scarce defect samples, removing process texture noise mixed in with the samples, and constructing a standardized, high-purity small-sample wafer defect feature library to provide high-quality defect feature sources for subsequent background removal and defect transplantation.

[0011] S101) Data Acquisition

[0012] To address seven typical defects encountered in wafer inspection scenarios, 189 defect images were collected from real-world industrial environments. Data acquisition was performed using a field emission scanning electron microscope (FET), which boasts nanometer-level resolution, enabling clear observation of the defect's microscopic morphology, cross-sectional structure, and edge contours. Defect categories included: contamination, adhesive layer abnormalities, scratches, peeling abnormalities, poor etching, poor development, and substrate defects.

[0013] S102) Data Preprocessing

[0014] All samples were uniformly sized and scaled to 512×512, and grayscale was normalized to eliminate brightness and contrast interference caused by differences in imaging devices.

[0015] Manually annotate a small number of defective samples to generate defect mask images, marking the pixel correspondence between the defective region and the background region to ensure accurate segmentation of the defective region and the background region.

[0016] S103) Decoupling and purification of defect features

[0017] A frequency-space joint feature extraction network is employed to decouple features from preprocessed few-sample defect samples, separating defect-specific features from process texture background features. Spatial and frequency domain features are extracted from the input defect image, yielding spatial and frequency domain feature maps, respectively.

[0018] S104) Construction of a small-sample wafer defect feature library

[0019] For each type of defect sample with few samples, a structured defect feature library is constructed, and the defect feature library is classified and indexed to achieve standardized management and rapid retrieval of defect features.

[0020] S2) FSF-TBS module performs background removal and defect enhancement

[0021] To address the strong interference of wafer image process texture background, a frequency-space fusion task-destructive background suppression module (FSF-TBS module) was designed. It breaks through the limitation of traditional TBS modules that rely solely on spatial domain features. Through deep interaction between frequency and spatial features, it achieves precise destruction of process texture background and enhancement of defect features.

[0022] The FSF-TBS module, as the core of the entire architecture, performs background purification on samples in the few-sample defect feature library and provides enhanced defect features for subsequent defect transplantation.

[0023] S201) FSF-TBS Module Core Principles

[0024] The FSF-TBS module, short for Frequency-Spatial Fusion Task-Breaking Background Suppression Module, is innovative in that it embeds a frequency-spatial interaction attention mechanism into the two core scoring processes of the TBS module, enabling frequency domain guidance to spatial domain. Specifically, it includes four sub-modules: frequency-spatial dual-domain feature encoding, query-relevant frequency-spatial interaction scoring, target-relevant frequency-spatial interaction scoring, and task-destructive background suppression.

[0025] S202) Specific working method of FSF-TBS module

[0026] Step 1: The input support set features and query set features are processed through the spatial attention submodule and the frequency attention submodule, respectively, to obtain three vectors: Q, K, and V. The support set features are the labeled defect sample features from a few-sample defect library, and the query set features are the image features to be processed.

[0027] Step 2: Construct a frequency-space interaction scoring mechanism for the support set (a small number of labeled defect samples) and query set (wafer images to be processed) in few-shot learning.

[0028] Step 3: Perform frequency-space interaction scoring on the query set images themselves.

[0029] Step 4: Defect Feature Enhancement and Output. The query set feature map after background suppression is fused with the clean defect features. Through feature upsampling and residual connection, the details and contrast of the defect features are enhanced to obtain the enhanced defect feature map. At the same time, the wafer image after background removal is output.

[0030] S3) Few-sample wafer image defect transplantation

[0031] By utilizing a massive number of defect-free wafer samples as a background substrate, accurate transfer of defect features is achieved. The transfer process does not pursue visual "realism," but focuses on the effectiveness of the defect recognition task, ensuring that the transferred defect features are clearly identifiable and suitable for training of defect recognition models with few samples.

[0032] S301) Defect-free wafer background preprocessing

[0033] Based on a standardized, defect-free background image, candidate regions suitable for placing defects are selected, taking care to avoid fixed structural areas such as lithographic marks and cutting paths to ensure the natural distribution of defects, and generating candidate region masks.

[0034] S302) Defect Feature Adaptive Transplantation

[0035] Geometric transformations are applied to the segmented defect regions to generate a large number of new defect samples. The transformed defects are then fused with the original defect-free background to obtain synthetic samples that are highly consistent with the real image.

[0036] S4) Quantitative evaluation indicators

[0037] To verify the effectiveness of the plan, it is necessary to evaluate it from both quantitative indicators and qualitative effects.

[0038] S401) Quantitative Indicators

[0039] The accuracy of defect segmentation is evaluated using the average intersection-to-union ratio (MIoU).

[0040] Structural similarity (SSIM) is used to evaluate the structural similarity between synthetic images and real wafer images.

[0041] The maximum mean difference (MMD) is used to assess whether the synthetic sample distribution closely approximates the real sample distribution.

[0042] S402), Qualitative effect

[0043] By visually comparing the original image with the image results after processing through steps S1, S2, and S3, observe whether the background is clean and whether the defect edges are clear. Attached Figure Description

[0044] Figure 1 This is a flowchart of a method for background suppression and defect feature enhancement in wafer defect detection.

[0045] Figure 2 This is the FSF-TBS module algorithm architecture diagram. Detailed Implementation

[0046] The following is in conjunction with the appendix Figure 1 The present invention provides a method for background removal and data enhancement of few-sample wafer defect images based on frequency-space fusion, comprising the following steps:

[0047] S1) Construction of a small-sample wafer defect library

[0048] To address the issue of extremely limited sample numbers for each type of wafer defect, instead of directly using original defect samples for model training, we first decouple and purify the features of scarce defect samples, removing process texture noise mixed in with the samples, and constructing a standardized, high-purity small-sample wafer defect feature library to provide high-quality defect feature sources for subsequent background removal and defect transplantation.

[0049] S101) Data Acquisition

[0050] To address seven typical defects in wafer inspection scenarios, 189 defect images were collected from real-world industrial environments. The data acquisition equipment was a field emission scanning electron microscope (FET), which boasts nanometer-level resolution, enabling clear observation of the defect's microscopic morphology, cross-sectional structure, and edge contours.

[0051] Defect categories include: dirt, adhesive layer abnormalities, scratches, peeling abnormalities, poor etching, poor development, and poor substrate.

[0052] The experimental dataset contains 45 images of dirt defects, accounting for approximately 23.81% of the total sample; 20 images of adhesive layer abnormalities, accounting for approximately 10.58%; 32 images of scratch defects, accounting for approximately 16.93%; 18 images of peeling abnormalities, accounting for approximately 9.52%; 28 images of etching defects, accounting for approximately 14.81%; 25 images of development defects, accounting for approximately 13.23%; and 21 images of substrate defects, accounting for approximately 11.11%.

[0053] S102) Data Preprocessing

[0054] All samples were uniformly sized and scaled to 512×512, and grayscale was normalized to eliminate brightness and contrast interference caused by differences in imaging devices.

[0055] Manually annotate a small number of defective samples to generate defect mask images, marking the pixel correspondence between the defective region and the background region to ensure accurate segmentation of the defective region and the background region.

[0056] All wafer defect SEM images were manually annotated using the Labelme open-source annotation tool. The annotation process was completed by professionals with semiconductor process knowledge.

[0057] Use a rectangle to precisely select the defect area, and strictly determine the position and size of the annotation box according to the actual outline and boundary range of the defect to ensure that the annotation area completely covers the defect body and does not contain too much invalid background area, and there are no omissions or errors in annotation.

[0058] S103) Decoupling and purification of defect features

[0059] A frequency-space joint feature extraction network is used to decouple the features of the preprocessed few defect samples, separating the defect-specific features from the process texture background features.

[0060] Spatial feature extraction is performed on the input defect image. Spatial structural features of the image, including the geometric shape of the defect and gray-level distribution, are extracted through a convolutional neural network to obtain a spatial feature map.

[0061] The input defect image is subjected to frequency domain feature decomposition. Two-dimensional discrete cosine transform (2D-DCT) is used to transform the image from the spatial domain to the frequency domain, dividing the frequency features into low-frequency components (corresponding to the overall wafer layout and global structure), mid-frequency components (corresponding to the periodic patterns of process textures), and high-frequency components (corresponding to defect edges, minor anomalies, and detailed features). Adaptive normalization is then applied to the frequency features to obtain the frequency domain feature map.

[0062] S104) Construction of a small-sample wafer defect feature library

[0063] For each type of few-sample defect sample, a structured defect feature library is constructed by combining the corresponding defect mask image, defect spatial morphological parameters, and frequency domain feature statistics.

[0064] The defect feature library is classified and indexed, and sub-libraries are established according to seven defect types. Each sub-library stores the original image, clean feature map, mask image and feature parameters of the defect sample, so as to realize standardized management and fast access to defect features.

[0065] S2) FSF-TBS module performs background removal and defect enhancement

[0066] To address the strong interference from process texture backgrounds in wafer images, a frequency-space fusion task-oriented destructive background suppression module (FSF-TBS module) was designed. This module overcomes the limitations of traditional TBS modules that rely solely on spatial domain features. Through deep interaction between frequency and spatial features, it achieves precise destruction of process texture backgrounds and enhancement of defect features. As the core of the entire architecture, this module purifies the background of samples in a small-sample defect feature library and provides enhanced defect features for subsequent defect transplantation.

[0067] S201) FSF-TBS Module Core Principles

[0068] The FSF-TBS module, short for Frequency-Spatial Fusion Task-Breaking Background Suppression Module, is innovative in that it embeds a frequency-spatial interaction attention mechanism into the two core scoring processes of the TBS module, enabling frequency domain guidance to spatial domain. Specifically, it includes four sub-modules: frequency-spatial dual-domain feature encoding, query-relevant frequency-spatial interaction scoring, target-relevant frequency-spatial interaction scoring, and task-destructive background suppression.

[0069] S202) Specific working method of FSF-TBS module

[0070] Step 1: The input support set features and query set features are processed through the spatial attention submodule and the frequency attention submodule, respectively, to obtain three vectors: Q, K, and V. The support set features are the labeled defect sample features from a few-sample defect library, and the query set features are the image features to be processed.

[0071] The support set feature input spatial attention submodule extracts the spatial structure attention weights A of the support set. Spatial_s This weight focuses on key areas such as the spatial morphology and edge contours of defects, suppressing the spatial response of background textures. The query set features are input into the spatial attention submodule, which extracts the spatial attention weight A from the query set. Spatial_Q It focuses on potential defect areas in the query image and suppresses the spatial response of redundant background.

[0072] The support set features and query set features are input together into the 2D-DCT frequency attention submodule, which extracts the support set frequency features and query set frequency features respectively.

[0073] Spatial attention weight A Spatial_s The query vector Q is obtained by weighted fusion with the frequency features of the support set. The spatial attention weights A are then applied. Spatial_Q The key vector K and value vector V are obtained by weighted fusion with the frequency features of the query set.

[0074] Step 2: Construct a frequency-space interaction scoring mechanism for the support set (a small number of labeled defect samples) and query set (wafer images to be processed) in few-shot learning.

[0075] Query-relevant frequency-space interaction scoring is a mechanism for constructing frequency-space interaction scoring between the support set and the query set in few-shot learning. It achieves accurate matching between defect features of the support set and image features of the query set, avoiding false matching of process textures.

[0076] Step 3: Perform frequency-space interaction scoring on the query set images themselves.

[0077] Target-relevant frequency-space interaction scoring performs frequency-space interaction scoring on the query set images themselves, enabling self-recognition and destruction of the process texture background in the query set.

[0078] Step 4: Defect Feature Enhancement and Output. The query set feature map after background suppression is fused with the clean defect features. Through feature upsampling and residual connection, the details and contrast of the defect features are enhanced to obtain the enhanced defect feature map. At the same time, the wafer image after background removal is output.

[0079] S3) Few-sample wafer image defect transplantation

[0080] By utilizing a massive number of defect-free wafer samples as a background substrate, accurate transfer of defect features is achieved. The transfer process does not pursue visual "realism," but focuses on the effectiveness of the defect recognition task, ensuring that the transferred defect features are clearly identifiable and suitable for training of defect recognition models with few samples.

[0081] S301) Defect-free wafer background preprocessing

[0082] Based on a standardized, defect-free background image, candidate regions suitable for placing defects are selected, taking care to avoid fixed structural areas such as lithographic marks and cutting paths to ensure the natural distribution of defects, and generating candidate region masks.

[0083] S302) Defect Feature Adaptive Transplantation

[0084] Geometric transformations such as rotation, translation, scaling, and flipping are applied to the segmented defect regions to generate a large number of new defect samples. This expands the diversity of defects, resulting in the transformed defects and their masks.

[0085] The transformed defects are then fused with the original defect-free background to obtain a synthetic sample that is highly consistent with the real image.

[0086] A pixel-by-pixel fusion method is adopted, using Poisson fusion to fuse the amplified defect features with the original model texture.

[0087] S4) Quantitative evaluation indicators

[0088] To verify the effectiveness of the plan, it is necessary to evaluate it from both quantitative indicators and qualitative effects.

[0089] S401) Quantitative Indicators

[0090] The accuracy of defect segmentation is evaluated using the average intersection-to-union ratio (MIoU).

[0091]

[0092] Wherein, TP represents the number of pixels correctly predicted by the model as the corresponding defect, FP represents the number of pixels that the model mistakenly predicted as the corresponding defect, and FN represents the number of pixels that the model missed, which should have been defects but were predicted as background.

[0093]

[0094] Where K represents the total number of defect categories.

[0095] Structural similarity (SSIM) is used to evaluate the structural similarity between the synthesized image and the real wafer image. Similarity is measured from three aspects: brightness, contrast, and structure, to ensure that the synthesized wafer image is structurally close to the real one and free of artifacts.

[0096]

[0097] in, and The mean of the image. For covariance.

[0098] The maximum mean difference (MMD) is used to evaluate whether the distribution of synthetic samples closely resembles the distribution of real samples. It measures the distance between the real defect distribution and the synthetic defect distribution in the feature space. The smaller the MMD, the closer the synthetic sample distribution is to the real one, which can be effectively used for training with few samples and avoid pattern collapse.

[0099]

[0100] in, The distribution of real defect samples. The distribution of synthetic defect samples, This is the kernel mapping function.

[0101] S402), Qualitative effect

[0102] By visually comparing the original image with the image results after processing through steps S1, S2, and S3, observe whether the background is clean and whether the defect edges are clear.

[0103] The above embodiments and description are intended only to illustrate the core concept and preferred implementation of the present invention. It is clearly stated that, without departing from the basic concept and scope of protection of the present invention, there are many possible variations and optimizations. All such variations and improvements are considered to be included within the scope of protection defined by the claims of the present invention.

Claims

1. A method for background removal and data augmentation of few-sample wafer defect images based on frequency-space fusion, characterized in that, Includes the following steps: S1) Construction of a small-sample wafer defect library To address the issue of extremely limited sample sizes for each type of wafer defect (≤20 samples per type), instead of directly using the original defect samples for model training, we first decouple and purify the features of the scarce defect samples, remove process texture noise mixed in the samples, and construct a standardized, high-purity small-sample wafer defect feature library to provide high-quality defect feature sources for subsequent background removal and defect transplantation. S101) Data Acquisition To address seven typical defects in wafer inspection scenarios, a total of 189 defect images were collected from real industrial sites. S102) Data Preprocessing All samples were uniformly sized and scaled to 512×512, and grayscale normalization was performed to eliminate brightness and contrast interference caused by differences in imaging devices. For the few defective samples, manual annotation was performed to generate defect mask images, marking the pixel correspondence between the defective and background areas to ensure accurate segmentation of the defective and background regions. S103) Decoupling and purification of defect features A frequency-space joint feature extraction network is used to decouple the features of the preprocessed few defect samples, separating the defect-specific features from the process texture background features. S104) Construction of a small-sample wafer defect feature library For each type of defect sample with few samples, a structured defect feature library is constructed, and the defect feature library is classified and indexed to achieve standardized management and rapid retrieval of defect features. S2) FSF-TBS module performs background removal and defect enhancement To address the strong interference from process texture backgrounds in wafer images, a frequency-space fusion task-oriented destructive background suppression module (FSF-TBS module) was designed. This module overcomes the limitations of traditional TBS modules that rely solely on spatial domain features. Through deep interaction between frequency and spatial features, it achieves precise destruction of process texture backgrounds and enhancement of defect features. As the core of the entire architecture, this module purifies the background of samples in a small-sample defect feature library and provides enhanced defect features for subsequent defect transplantation. S201) FSF-TBS Module Core Principles The FSF-TBS module, short for Frequency-Spatial Fusion Task-Breaking Background Suppression Module, is innovative in that it embeds a frequency-spatial interaction attention mechanism into the two core scoring processes of the TBS module, enabling frequency domain guidance to spatial domain. Specifically, it includes four sub-modules: frequency-spatial dual-domain feature encoding, query-relevant frequency-spatial interaction scoring, target-relevant frequency-spatial interaction scoring, and task-destructive background suppression. S202) Specific working method of FSF-TBS module Step 1: The input support set features and query set features are processed through the spatial attention submodule and the frequency attention submodule, respectively, to obtain three vectors: Q, K, and V. The support set features are the labeled defect sample features from a few-sample defect library, and the query set features are the image features to be processed. Step 2: Construct a frequency-space interaction scoring mechanism for the support set (a small number of labeled defect samples) and query set (wafer images to be processed) in few-shot learning. Step 3: Perform frequency-space interaction scoring on the query set images themselves. Step 4: Defect feature enhancement and output. The query set feature map after background suppression is fused with the defect-clean features. By upsampling the features and connecting the residuals, the details and contrast of the defect features are enhanced to obtain an enhanced defect feature map. At the same time, the wafer image after background removal is output. S3) Few-sample wafer image defect transplantation By utilizing a massive number of defect-free wafer samples as a background substrate, accurate transfer of defect features is achieved. The transfer process does not pursue visual "realism," but focuses on the effectiveness of the defect recognition task, ensuring that the transferred defect features are clearly identifiable and suitable for training of a few-sample defect recognition model. S301) Defect-free wafer background preprocessing Based on a standardized, defect-free background image, candidate regions suitable for placing defects are selected, taking care to avoid fixed structural areas such as lithographic marks and cutting paths to ensure the natural distribution of defects, and generating candidate region masks. S302) Defect Feature Adaptive Transplantation Geometric transformations (rotation, translation, scaling, flipping, etc.) are applied to the segmented defect regions to generate a large number of new defect samples. The transformed defects are then fused with the original defect-free background to obtain a synthetic sample that is highly consistent with the real image. S4) Quantitative evaluation indicators To verify the effectiveness of the plan, it is necessary to evaluate it from both quantitative indicators and qualitative effects. S401) Quantitative Indicators The mean intersection-to-union ratio (MIoU) is used to evaluate defect segmentation accuracy. Structural similarity (SSIM) is used to evaluate the structural similarity between the synthesized image and the real wafer image. Maximum mean difference (MMD) is used to evaluate whether the synthetic sample distribution closely approximates the real sample distribution. S402), Qualitative effect By visually comparing the original image with the image results after processing through steps S1, S2, and S3, observe whether the background is clean and whether the defect edges are clear.

Images