Multi-scale feature fusion wafer defect detection method

Abstract

The invention relates to the field of computer vision and semiconductor manufacturing detection, in particular to a multi-scale feature fusion wafer defect detection method, which comprises the following steps of: acquiring a wafer image, and labeling particle defects existing in the image by using a manual labeling tool; the marked image is preprocessed; and constructing and training a defect intelligent identification model based on the convolutional neural network, and performing defect detection on the processed wafer image by using the model. The wafer particle defect intelligent detection method provided by the invention has remarkable advantages in the aspects of detection precision, recognition speed, defect classification, system stability, deployability and the like, can be widely applied to multiple fields of integrated circuit manufacturing, semiconductor packaging, silicon wafer detection and the like, and has good popularization and application prospects and industrial value.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer vision and semiconductor manufacturing detection, in particular to a multi-scale feature fusion wafer defect detection method, which is suitable for defect screening and quality control links in wafer production lines. BACKGROUND

[0002] In the integrated circuit manufacturing process, particle defects on the wafer surface are one of the main factors affecting chip yield. Such particles often come from dust, metal debris, chemical residues or material peeling in the production environment, with characteristics of small size, low contrast, random distribution, and blurred edges. As the process node continues to shrink, the size of the particles tends to be nanoscale, and their impact on chip performance is increasingly significant. Any small particle can cause open circuit, short circuit or functional failure of the device. Therefore, how to achieve high-sensitivity detection, accurate positioning, rapid identification and type classification of particle defects on the wafer surface has become a key technical problem that the industry needs to solve.

[0003] Traditional optical detection equipment (such as automatic optical detection instruments) relies on regular image processing methods and faces the following technical bottlenecks:

[0004] 1. Limited defect recognition capability

[0005] The wafer surface is mostly a mirror surface reflection structure, and the particles exhibit low contrast, high reflectivity, and non-uniform morphology, making it difficult for traditional algorithms to distinguish between real defects and background noise, especially when the particle diameter is less than 1 micrometer, the detection accuracy decreases significantly.

[0006] 2. Weak defect classification capability

[0007] Existing detection systems mostly rely on "whether there is a defect" as the basis for judgment, which cannot further distinguish the types, properties and sources of defects, and is not conducive to process tracing and classification analysis.

[0008] 3. Difficulty in identifying small size targets

[0009] Most wafer particles are small targets, and in traditional image recognition networks, feature down-sampling often leads to information loss, resulting in missed detection or positioning deviation.

[0010] 4. Limited real-time performance and deployability

[0011] Current deep models are large in size and high in computational complexity, making them difficult to deploy on actual production lines and difficult to meet the speed and stability requirements of online detection.

[0012] 5. Insufficient model generalization capability

[0013] Wafer surface images are influenced by process differences, light changes, camera settings, and other factors, and have strong scene diversity. Models obtained from a single training set often lack generalization ability in cross-batch and cross-process applications, and are prone to miss detection and false detection.

[0014] In the wafer production process, particle defects are one of the main factors causing product failure. Traditional manual inspection has problems such as low efficiency, high false detection rate, and is difficult to meet the needs of high-precision and high-throughput industrial detection; traditional image processing-based detection methods rely on image grayscale changes, edge detection, threshold segmentation and other manual rule methods; traditional classifier-based detection methods classify and detect particle types by manually extracting texture, color, edge and other features; surface defect recognition methods based on deep learning focus on larger size defects such as scratches and damage, and the number of targets is small, and particle defect detection is still a bottleneck.

[0015] In summary, the existing technology in wafer particle defect detection application mainly has the following deficiencies:

[0016] 1. Poor recognition ability for small particles

[0017] Particle defects are often extremely small in size (such as 0.2-3 μm), and traditional image processing methods have difficulty distinguishing subtle differences from the background, which can lead to missed detection; shallow machine learning relies on limited manual feature extraction capabilities, making it difficult to cover all defect types.

[0018] 2. Poor adaptability to complex working conditions

[0019] Wafer surfaces have diverse background interference (such as scratches, patterns, stains, etc.), and existing methods have poor generalization ability, are prone to false positives or missed detection, and require frequent parameter adjustments, which is not practical.

[0020] 3. Low detection efficiency, difficult to meet online detection needs

[0021] Traditional methods have slow detection speeds and cannot be used in real-time applications on high-speed wafer production lines, making it difficult to adapt to wafer manufacturing scenarios.

[0022] 4. Lack of defect positioning and classification ability

[0023] Traditional methods can only determine whether there is a defect, but cannot accurately mark the defect location and category, which is not conducive to subsequent tracing and automatic rejection operations.

[0024] 5. Models are difficult to deploy on edge devices

[0025] Some deep neural networks based on large models have large computational loads and high parameter quantities, which are not suitable for embedded edge detection devices and have high deployment costs. SUMMARY

[0026] The application provides a multi-scale feature fusion target detection method for wafer particle defect detection, which can automatically, quickly and accurately identify the tiny particle defects on the wafer surface without affecting the production efficiency, thereby improving the yield and product quality.

[0027] The technical scheme adopted by the application to achieve the above object is:

[0028] A multi-scale feature fusion wafer defect detection method, comprising the following steps:

[0029] 1) Collecting wafer images and using artificial labeling tools to label the particle defects existing in the images;

[0030] 2) Preprocessing the labeled images;

[0031] 3) Constructing and training a defect intelligent recognition model based on a convolutional neural network, and using the model to process the wafer images for defect detection.

[0032] The labeled content includes defect type, defect position and defect size.

[0033] The step 2) is specifically: according to the requirements, the labeled images are processed using any one of the following processing methods:

[0034] (1) Rotating, scaling and flipping the images to simulate different shooting angles and scale differences;

[0035] (2) Brightness disturbance and contrast enhancement are performed on the images to adapt to different lighting conditions;

[0036] (3) Selecting images with tiny particles, combining all the selected images into one image, and completing defect simulation superposition of the image;

[0037] (4) Introducing a multi-image synthesis strategy, randomly combining several images in the data set into a large image to obtain image samples with complex backgrounds, and realizing image fusion enhancement.

[0038] The defect intelligent recognition model performs the following steps:

[0039] The backbone feature extraction network uses a lightweight residual structure and multiple channels with different channel numbers to extract semantic features of different levels in the image;

[0040] The feature fusion module integrates low-level detail features and high-level semantic features through a pyramid feature fusion mechanism;

[0041] The defect discrimination and positioning module outputs the defect position, type and confidence through multiple detection heads in parallel.

[0042] The pyramid feature fusion mechanism combines feature maps at different resolutions between layers, and the combined results are enhanced by channel attention and spatial attention.

[0043] The detection head is trained by introducing a class balance factor and a size weight parameter in the loss function.

[0044] The loss function includes a positioning loss and a confidence loss, and the positioning loss and the confidence loss are weighted and added as a total loss, wherein:

[0045] The positioning loss L loc is:

[0046]

[0047] Wherein, λ coord represents an indicator, and the predicted bounding box contains four parameters, namely: x i and y i represent the horizontal and vertical coordinates of the center point, w i represents the width of the bounding box, h i represents the height of the bounding box, 1 obj is an indicator function, which takes the value of 1 if there is an object at this position, i.e. the target is detected, otherwise 0, and represent the horizontal and vertical coordinates of the center of the real bounding box, represents the width of the real bounding box, represents the height of the real bounding box;

[0048] The confidence loss L conf is:

[0049]

[0050] Wherein, c i represents the confidence score of the predicted bounding box, represents the confidence score of the real bounding box, λ noobj is a weight coefficient of a non-target area, used to adjust the contribution of the loss of the confidence of the non-target area to the total loss, 1 noobj is an indicator function, used to indicate whether there is no target at a certain position, i.e. the position is empty or has no object.

[0051] The present application has the following advantages and benefits:

[0052] 1. Improve the accuracy and sensitivity of wafer defect detection

[0053] The scheme constructs a special image dataset for wafer particle defects, and introduces a multi-scale feature extraction structure and an attention mechanism, so that the model can effectively identify particles with a diameter of less than 0.5 microns, which is significantly better than the detection ability of traditional image processing methods for small particle defects.

[0054] In addition, the model introduces a feature enhancement module to strengthen the response to weak edges and fuzzy boundary particles, effectively reducing the missed detection rate and improving the overall detection sensitivity. In complex background and uneven lighting interference conditions, it can still stably identify high-density and low-contrast defect areas.

[0055] 2. Realize automatic classification and positioning of multi-class particle defects

[0056] Unlike traditional detection methods that can only make a preliminary judgment of "defect or not", the present scheme has the ability to identify multiple types of particle defects. By training a multi-class target recognition model, the system can automatically distinguish different types of defects (such as metal particles, dust particles, residual glue particles, and material shedding), and combine positioning frame labeling to mark the accurate position and size, realizing the integration of defect type identification, spatial positioning, and size regression.

[0057] This feature helps subsequent quality traceability, defect distribution thermal analysis, and process optimization decisions.

[0058] 3. Strengthen the detection ability of small size targets and reduce the risk of missed detection

[0059] The present scheme specially designs a small target detection enhancement mechanism to address the difficulty of small target recognition. By using methods such as shallow high-resolution feature map access detection head, down-sampling ratio optimization, and introducing size-sensitive weights in the loss function, the system can retain image detail information while improving the detection ability of small particles.

[0060] Experiments show that in a typical wafer particle defect test set, compared with general detection models, the system's average detection accuracy for small targets (area ratio <1%) is improved by more than 12 percentage points, effectively reducing the detection omission caused by small target size.

[0061] 4. Support the online deployment needs of high-throughput wafer production lines

[0062] Considering the high requirements of wafer manufacturing lines for detection speed, the present scheme introduces a lightweight feature extraction backbone network and a reasoning acceleration engine in the model structure design, and through model pruning, quantization, and deployment format optimization, finally realizes the efficient operation of the detection model on industrial GPUs or edge devices.

[0063] In actual deployment scenarios, the system can achieve a single wafer image processing time of less than 10 milliseconds, meeting the detection beat requirements of more than 30 wafers per hour, and adapting to the real-time operation requirements of modern automated production lines.

[0064] 5. Improve the scalability and adaptability of the defect detection system

[0065] The present scheme adopts an end-to-end supervised learning method, and the model has continuous training and transfer learning capabilities. When the process is changed, new models are introduced, or new defect types appear, only a small amount of samples need to be supplemented and fine-tuned to complete the model adaptive upgrade, without the need to rebuild the entire system, greatly reducing the later maintenance cost.

[0066] In addition, the system also supports defect image archiving, automatic sample screening and online incremental learning, and has strong self-optimization and dynamic adaptation capabilities, suitable for complex detection requirements of multiple scenarios, multiple batches, and multiple wafer materials.

[0067] 6. Reduce the burden and error rate of manual detection

[0068] Traditional wafer particle defect detection still relies partly on manual inspection, especially in small batch sample verification and initial stage of new product introduction. Manual methods have low detection efficiency, large subjective errors, and poor repeatability. The introduction of the present system can greatly reduce manual intervention, reduce the influence of human factors on detection results, and improve production consistency and quality stability.

[0069] In multiple experimental comparisons, the average misjudgment rate of the intelligent detection system is less than 3%, which is much lower than the error level of manual detection. BRIEF DESCRIPTION OF DRAWINGS

[0070] Figure 1 Multi-scale feature fusion target detection model structure diagram;

[0071] Figure 2 Wafer transmission and detection flowchart. DETAILED DESCRIPTION

[0072] The present application will be further described in detail below in combination with the drawings and examples.

[0073] The present application is based on an end-to-end intelligent visual recognition algorithm framework, combined with multi-scale image feature extraction, attention mechanism, small target enhancement and efficient deployment optimization, to build an intelligent method suitable for wafer surface particle defect automatic detection and classification, as shown in Figure 2 The method comprises the following steps:

[0074] 1. Wafer image acquisition and sample data set construction

[0075] First, in a high-cleanliness experimental environment, select silicon wafer samples from multiple process batches, use a high-resolution industrial camera to perform full-surface scanning of the wafer surface, and obtain clear color or grayscale images. The image resolution is not less than 5000x5000 pixels to ensure that the details of the particle defects can be completely retained.

[0076] Subsequently, the particle defects present in the image are labeled using a manual labeling tool, and the labeling content includes:

[0077] 1) Defect type (such as dust, metal particles, and delamination residue)

[0078] 2) Defect location (coordinate information recorded in the form of a bounding box)

[0079] 3) Defect size (converted from pixel size to actual micron value)

[0080] On this basis, a standardized defect image sample set is constructed, with a total of not less than 10,000 samples and covering different background interference, lighting conditions, and image noise situations.

[0081] 2. Image enhancement and data preprocessing

[0082] To improve the generalization ability and robustness of the model, various data enhancement processing is performed on the sample images, including but not limited to:

[0083] 1) Image rotation, mirroring, and scale scaling;

[0084] 2) Local brightness perturbation and contrast adjustment;

[0085] 3) Noise injection and blur simulation;

[0086] 4) Multi-image stitching (mosaic synthesis), region cropping, etc.

[0087] The enhanced images are uniformly converted to a uniform size (such as 1024x1024 pixels), and are divided into training set, validation set, and test set according to the ratio of 8:1:1. All labeling information is stored as a structured label file (such as JSON or XML format).

[0088] 3. Construction and training of intelligent identification model

[0089] As shown in Figure 1 , the present application selects a convolutional neural network with multi-scale feature extraction capability as the basic recognition framework. The model structure consists of three parts:

[0090] 1) Main feature extraction network: used to extract different levels of semantic information in the image, using a lightweight residual structure that balances computational efficiency and feature expression ability;

[0091] 2) Feature fusion module: Introduce pyramid feature fusion structure to integrate low-level detail features and high-level semantic features to adapt to the detection of multi-size targets;

[0092] 3) Defect discrimination and positioning module: Use multiple detection heads to output defect position, type and confidence in parallel to realize joint learning of target recognition and positioning regression tasks.

[0093] The model training process is as follows:

[0094] 1) Loss function: Comprehensive use of bounding box regression loss, classification loss and target existence loss;

[0095] 2) Optimizer: Select adaptive moment estimation optimizer (Adam);

[0096] 3) Learning rate adjustment strategy: Adopt cosine annealing scheduling mechanism;

[0097] 4) The number of training rounds is not less than 300 rounds, and early stopping mechanism is introduced in the training process to prevent overfitting;

[0098] 5) Use GPU to accelerate calculation to ensure training efficiency;

[0099] 6) After the final training is completed, select the model with the best performance on the validation set as the deployment version, and export it in a lightweight format (such as ONNX or TensorRT).

[0100] 4. Defect detection process and system deployment

[0101] In the deployment stage, the trained model is loaded into an industrial computer or edge computing device, and the following modules are combined to build a complete detection system:

[0102] 1) Image acquisition module: Automatically acquire the wafer image to be tested and pass it into the processing queue;

[0103] 2) Image preprocessing module: Perform resolution resampling, normalization and other processing on the input image;

[0104] 3) Inference module: Call the defect recognition model for real-time inference and output the defect type, position and confidence;

[0105] 4) Result analysis module: Visualize the recognition results and generate a structured detection report;

[0106] 5) Alarm and interception mechanism: Automatically trigger quality control signals for out-of-limit defects and link with the mechanical arm for defective product sorting.

[0107] The entire system can achieve an average detection time of ≤10ms for single wafer images in actual production line operation, meeting the real-time requirements.

[0108] 5. Detection result storage and system integration

[0109] 1) After detection is completed, the system writes defect information into a local database and supports the following integration functions:

[0110] 2) Interface with a manufacturing execution system (MES) to upload defect data and implement quality traceability;

[0111] 3) Support for defect image archiving, automatic numbering and label recording to facilitate subsequent query and re-inspection;

[0112] 4) Generate a defect distribution heat map for defect trend analysis;

[0113] 5) Support for remote data synchronization, online model updates and version management.

[0114] In addition, the system supports parameter setting and state monitoring through a graphical interface, facilitating equipment management and maintenance by engineers.

[0115] The present application has the following improvements

[0116] (1) High-quality image sample construction and enhancement

[0117] To improve the learning ability of the algorithm for particle defects, a large-scale image sample set containing multiple wafer types, different process batches and complex background conditions is first constructed. The data set contains multiple types of particle defects, including but not limited to:

[0118] 1) Metal particle class (such as mechanical spalling, material fragments)

[0119] 2) Chemical residue class (such as glue particles, corrosion products)

[0120] 3) Environmental particle class (such as air dust, fiber residues)

[0121] The following enhancement methods are used to improve the robustness of the model:

[0122] 1) Image rotation, scaling, and flipping: simulate different shooting angles and scale differences;

[0123] 2) Brightness perturbation, contrast enhancement: adapt to different lighting conditions;

[0124] 3) Defect simulation superposition: synthesize small particle patterns to enrich small target samples;

[0125] 4) Image fusion enhancement: introduce multi-image synthesis strategies to increase background complexity.

[0126] All samples are precisely framed using a manual annotation tool and classified for subsequent supervised training.

[0127] (2) Feature extraction network structure optimization

[0128] The system adopts a multi-layer convolutional structure as the feature extraction backbone, and obtains discriminative multi-scale semantic features by extracting the edges, textures, shapes and context information of the image layer by layer. The main optimization points include:

[0129] 1) Lightweight feature extraction backbone network: Select a deep convolutional network with high parameter efficiency, such as a cross-stage residual structure, to improve the balance between speed and accuracy.

[0130] 2) Multi-scale feature fusion structure: Introduce a pyramid fusion strategy to combine feature maps at different resolutions between layers, and strengthen the network's response to targets of different sizes.

[0131] 3) Attention mechanism introduction: Combine channel attention and spatial attention modules to guide the network to focus on areas with high defect information density and improve the recognition ability of weak targets.

[0132] (3) Detection strategy for small particle targets

[0133] Since particle defects are generally small in size, features are easily erased during network downsampling, so the following improvements are proposed:

[0134] 1) Shallow high-resolution branch access detection head: Preserve low-level feature maps to improve the contribution of fine-grained information to detection;

[0135] 2) Enhanced small target weight loss function design: Introduce a class balance factor and size weight parameter in the loss function to increase the training gradient for small size targets;

[0136] Positioning loss function:

[0137]

[0138] where λ coord represents an indicator, and the four parameters of the bounding box: x i and y i represent the center coordinates, w i represents the width, and h i represents the height.

[0139] Confidence loss function:

[0140]

[0141] where c i represents the confidence score of the predicted bounding box, represents the confidence score of the true bounding box.

[0142] 3) Reduce receptive field step: By fine-tuning network structure, reduce feature map down-sampling factor, improve response to detailed target.

[0143] (4) Model compression and real-time deployment optimization

[0144] To achieve the online detection of industrial application requirements, the following deployment optimization scheme is proposed:

[0145] Model pruning: analyze the importance of channels in the network, eliminate redundant structures, and reduce the number of parameters;

[0146] Quantization and accelerated inference: quantize the network from floating-point precision to low-bit width format (such as 16-bit or 8-bit), and cooperate with high-performance inference engine (such as tensor acceleration library);

[0147] 1) Export deployment model format: support cross-platform conversion (such as ONNX format) and adapt to multiple running platforms such as industrial PC, embedded GPU, edge computing chip, etc.

[0148] 2) Detection process pipelining: combine image preprocessing, parallel computing and asynchronous processing of detection tasks to improve overall throughput and meet online production line rhythm.

[0149] (5) Defect information structured output and system integration

[0150] The detection module finally outputs structured defect information, including:

[0151] 1) Defect position coordinates (center point, bounding box size)

[0152] 2) Defect type label (particle type)

[0153] 3) Confidence score (used to filter low-confidence detection results)

[0154] 4) Defect distribution map (used for defect heat distribution analysis)

[0155] The system can be seamlessly integrated with manufacturing execution system (MES) or quality traceability platform to realize defect statistics, process analysis and process closed-loop optimization. At the same time, it supports historical defect image archiving and backtracking verification, forming a quality monitoring database.

Claims

1. A multi-scale feature fusion wafer defect detection method, characterized in that, The method comprises the following steps: 1) Collecting wafer images and labeling particle defects existing in the images using a manual labeling tool; 2) Preprocessing the labeled images; 3) Constructing and training a defect intelligent recognition model based on a convolutional neural network, and using the model to process the preprocessed wafer images for defect detection.

2. The multi-scale feature fusion wafer defect detection method of claim 1, wherein, The labeled content includes defect type, defect position, and defect size.

3. The multi-scale feature fusion wafer defect detection method of claim 1, wherein, The step 2) is specifically: according to requirements, the following any one processing method is used to process the labeled images: (1) Rotate, scale, and flip the images to simulate different shooting angles and scale differences; (2) Perform brightness disturbance and contrast enhancement on the images to adapt to different lighting conditions; (3) Select images with tiny particles, combine all the selected images into one image, and complete defect simulation superposition of the image; (4) Introduce a multi-image synthesis strategy, randomly combine several images in the data set into a large image, obtain an image sample with a complex background, and realize image fusion enhancement.

4. The multi-scale feature fusion wafer defect detection method of claim 1, wherein, The defect intelligent recognition model performs the following steps: The backbone feature extraction network extracts semantic features of different levels in the image by using a lightweight residual structure and multiple channels with different channel numbers; The feature fusion module integrates low-level detailed features and high-level semantic features through a pyramid feature fusion mechanism; The defect discrimination and positioning module outputs the defect position, type, and confidence through multiple detection heads in parallel.

5. The multi-scale feature fusion wafer defect detection method of claim 4, wherein, The pyramid feature fusion mechanism combines feature maps at different resolutions between layers, and enhances the combined results through channel attention and spatial attention.

6. The multi-scale feature fusion wafer defect detection method of claim 4, wherein, The detection heads are trained by introducing a class balance factor and a size weight parameter into the loss function.

7. The multi-scale feature fusion wafer defect detection method of claim 6, wherein, The loss function includes a positioning loss and a confidence loss, and the positioning loss and the confidence loss are weighted and added as a total loss, wherein: The positioning loss L loc is: wherein λ coord represents an indicator, four parameters of the predicted bounding box, i.e., x i and y i represent the horizontal and vertical coordinates of the center point, w i represents the width of the bounding box, h i represents the height of the bounding box, 1 obj is an indicator function, which takes the value of 1 if there is an object at the position, i.e., the target is detected, otherwise 0, and represent the horizontal and vertical coordinates of the center of the real bounding box, represents the width of the real bounding box, represents the height of the real bounding box; The confidence loss L conf is: wherein c i denotes the confidence score of the predicted bounding box, denotes the confidence score of the real bounding box, λ noobj is a weight coefficient of the non-target region, used to adjust the contribution of the loss of the non-target region confidence to the total loss, 1 noobj is an indicator function, used to indicate whether a certain position has no target, i.e. the position is empty or has no object.

Images