Wafer surface defect visual detection method, system, storage medium and device

Abstract

The invention discloses a wafer surface defect visual detection method, system, storage medium and device, and the method comprises the steps: collecting a multi-mode image, and collecting the image based on two preset modes; performing three-dimensional shape reconstruction and phase calculation based on the acquired wafer image to obtain a three-dimensional point cloud on the surface of the wafer; performing feature extraction and data enhancement based on the three-dimensional point cloud to obtain a multi-channel feature tensor; inputting the feature tensor into a trained double-branch convolutional neural network model to obtain model output, wherein the model output comprises a defect type and a quantization parameter; and generating a detection report based on the model output, and outputting the detection report. According to the method, high-speed, high-precision and nondestructive detection of wafer surface nanoscale defects is realized by fusing multi-mode structured light projection and a deep learning technology, the method has excellent defect distinguishing capability and industrial field robustness, comprehensive defect quantitative data can be output, and a reliable online detection solution is provided for semiconductor manufacturing.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically, to a method, system, storage medium, and apparatus for visual inspection of defects on wafer surfaces. Background Technology

[0002] In the semiconductor manufacturing industry, the wafer serves as the substrate for integrated circuits, and its surface quality directly determines the chip's performance, yield, and reliability. Microscopic defects on the wafer surface, such as "fisheyes" and "pin differences," are among the key factors affecting chip manufacturing quality. "Fisheyes" typically refer to tiny pits on the wafer surface caused by uneven chemical mechanical polishing or defects in the material itself. Their diameter is generally in the micrometer range, and their depth is in the nanometer range. "Pin differences" refer to the height differences between different areas on the wafer surface, usually caused by uneven thin film deposition or etching process deviations. Their height differences can reach tens of nanometers. These microscopic defects can seriously affect the depth of focus in subsequent photolithography processes, leading to pattern distortion, uneven linewidth, and even short circuits or open circuits, which can have a fatal impact on chip performance.

[0003] As semiconductor process nodes continue to shrink to 7 nanometers, 5 nanometers, and even more advanced processes, the requirements for wafer surface quality have reached the atomic level. Traditional contact measurement methods, such as profilometers, can no longer meet the requirements of modern semiconductor manufacturing for efficiency and non-destructive testing because they may scratch the wafer surface and are slow. Therefore, non-contact inspection technology based on machine vision has become the mainstream direction for wafer surface defect detection.

[0004] Existing machine vision-based wafer surface defect detection solutions are mainly divided into three categories: detection methods based on two-dimensional image processing, detection methods based on confocal microscopy, and detection methods based on interferometry.

[0005] One detection method, based on two-dimensional image processing, primarily utilizes a high-resolution camera to acquire two-dimensional images of the wafer surface and then uses image processing algorithms to identify surface defects. A typical workflow includes: first, acquiring wafer surface images using a bright-field or dark-field illumination system; then, preprocessing the images, including filtering and noise reduction, and contrast enhancement; next, extracting features using algorithms such as edge detection, texture analysis, or template matching; and finally, determining the presence of defects using a classifier. For example, a typical implementation uses a high-resolution CCD camera with a ring-shaped LED light source to acquire wafer surface images, followed by edge detection using a modified Canny operator, and then defect classification using a support vector machine. This method performs well in detecting obvious macroscopic defects, but it is essentially a two-dimensional imaging method and cannot acquire three-dimensional surface morphology information. Its detection capability is limited for microscopic defects such as fisheyes and pinhole defects, which require high-resolution information for accurate identification. Especially when the grayscale contrast between the defect and the background is low, it is prone to missed detections.

[0006] Secondly, confocal microscopy-based inspection methods utilize spatial pinholes to eliminate defocused light, achieving higher axial resolution than traditional optical microscopes. In wafer inspection, a series of images with different focal planes are acquired through longitudinal scanning, and then the three-dimensional morphology of the surface is reconstructed using algorithms. Its longitudinal resolution can reach the nanometer level, theoretically capable of detecting fisheye depth and pinhole height. However, this method requires a precise longitudinal scanning mechanism, resulting in slow inspection speeds and difficulty meeting the real-time requirements of online inspection. Furthermore, confocal microscopes are highly sensitive to vibration and exhibit poor stability in industrial environments. In addition, their relatively small field of view necessitates extensive image stitching to achieve full wafer inspection, further reducing inspection efficiency.

[0007] Interferometric detection methods, such as white-light interferometers and phase-shifting interferometers, are commonly used tools for measuring surface topography. They utilize the principle of light interference to reconstruct the three-dimensional morphology of a surface by analyzing interference fringes, achieving nanometer-level or even sub-nanometer-level longitudinal resolution. For example, phase-shifting interferometry acquires multiple interferograms by changing the phase difference between the reference and measurement lights, and then calculates the surface height using a phase extraction algorithm. While this method is highly accurate, it is extremely sensitive to environmental vibrations and air disturbances, requiring strict vibration isolation and temperature control, resulting in high equipment costs. More importantly, traditional interferometric methods suffer from phase entanglement problems when encountering surface discontinuities or steep edges, leading to height measurement errors. This is particularly detrimental to the detection of pin segment differences with significant height variations. Summary of the Invention

[0008] The purpose of this invention is to provide a method, system, storage medium, and device for visual inspection of wafer surface defects, which solves the problems of existing wafer defect inspection technologies being unable to balance high precision and high speed, being sensitive to environmental interference, and being difficult to accurately identify and quantify three-dimensional micro-defects.

[0009] The first aspect of this invention provides a method for visual inspection of defects on a wafer surface, comprising the following steps:

[0010] Acquire multi-mode images, including images acquired based on two preset modes;

[0011] Three-dimensional topography reconstruction and phase calculation are performed based on the acquired wafer images to obtain the three-dimensional point cloud of the wafer surface;

[0012] Based on the three-dimensional point cloud, feature extraction and data augmentation are performed to obtain a multi-channel feature tensor;

[0013] The feature tensor is input into a trained dual-branch convolutional neural network model to obtain the model output, which includes the defect type and quantization parameters.

[0014] A detection report is generated based on the model output, and the detection report is output.

[0015] In this solution, the acquisition of multi-mode images specifically includes:

[0016] The preset displacement stage is controlled to move to the initial detection position, and a uniform illumination image of unstructured light is acquired for reflectivity correction.

[0017] Images of the wafer surface are acquired using a fast scanning mode to identify regions of interest;

[0018] Four deformable grating images of the region of interest or the entire wafer are acquired using a fine scanning mode.

[0019] In this scheme, the step of performing three-dimensional topography reconstruction and phase calculation based on the acquired wafer image to obtain a three-dimensional point cloud of the wafer surface specifically includes:

[0020] The wrapping phase is calculated pixel-by-pixel for the four distorted grating images, using the following formula:

[0021] ;

[0022] in, To wrap the phase, , , , There are four images in total;

[0023] The low-frequency absolute phase is obtained by expanding the low-frequency phase. Furthermore, the high-frequency phase is expanded using the relationship between high and low frequency phases, as shown in the following formula:

[0024] ;

[0025] in, For high-frequency absolute phase, For high-frequency wrapping phase, For the floor function, For low-frequency absolute phase, Low frequency, High frequency;

[0026] The absolute phase is obtained based on the high-frequency absolute phase and the low-frequency absolute phase. The three-dimensional point cloud is obtained by reconstructing the three-dimensional point cloud data of the wafer surface using a phase-height mapping model with preset calibration parameters. .

[0027] In this scheme, the step of obtaining a multi-channel feature tensor based on the 3D point cloud through feature extraction and data augmentation specifically includes:

[0028] Four feature maps are calculated from the three-dimensional point cloud, wherein,

[0029] Obtain the height map by taking the Z value of the 3D point cloud in the Z direction;

[0030] Calculate the gradient of height in the X and Y directions and calculate the magnitude to obtain a gradient magnitude map;

[0031] Calculate the average curvature or Gaussian curvature of each point in a 3D point cloud to obtain a local curvature map;

[0032] The LBP texture map is obtained by calculating the local binary pattern on the height map;

[0033] The four feature maps are concatenated along the channel dimension and then enhanced online to obtain a multi-channel feature tensor. Online enhancement methods include adding noise or simulating illumination.

[0034] In this scheme, the step of inputting the feature tensor into a trained dual-branch convolutional neural network model to obtain the model output specifically includes:

[0035] The dual-branch convolutional neural network model includes a semantic segmentation branch and a high-level regression and classification branch;

[0036] The feature tensor is input into the dual-branch convolutional neural network model. Based on the semantic segmentation branch, a fisheye defect segmentation mask is obtained, and based on the height regression and classification branches, a height deviation map and a defect category probability vector are obtained.

[0037] The final defect type is determined based on the defect category probability vector, whereby the defect categories include fisheye and needle segment difference.

[0038] The depth and diameter of the fisheye region are calculated as quantization parameters based on the fisheye defect segmentation mask and height deviation map.

[0039] The height difference between needle segments in different regions is calculated based on the height deviation map and used as the quantization parameter.

[0040] In this solution, all defect categories and quantitative parameters are summarized to generate structured inspection reports and / or visualization charts for external output. The visualization charts include defect distribution maps.

[0041] A second aspect of the present invention also provides a wafer surface defect visual inspection system, including a memory and a processor. The memory includes a wafer surface defect visual inspection method program, which, when executed by the processor, performs the following steps:

[0042] Acquire multi-mode images, including images acquired based on two preset modes;

[0043] Three-dimensional topography reconstruction and phase calculation are performed based on the acquired wafer images to obtain the three-dimensional point cloud of the wafer surface;

[0044] Based on the three-dimensional point cloud, feature extraction and data augmentation are performed to obtain a multi-channel feature tensor;

[0045] The feature tensor is input into a trained dual-branch convolutional neural network model to obtain the model output, which includes the defect type and quantization parameters.

[0046] A detection report is generated based on the model output, and the detection report is output.

[0047] In this solution, the acquisition of multi-mode images specifically includes:

[0048] The preset displacement stage is controlled to move to the initial detection position, and a uniform illumination image of unstructured light is acquired for reflectivity correction.

[0049] Images of the wafer surface are acquired using a fast scanning mode to identify regions of interest;

[0050] Four deformable grating images of the region of interest or the entire wafer are acquired using a fine scanning mode.

[0051] In this scheme, the step of performing three-dimensional topography reconstruction and phase calculation based on the acquired wafer image to obtain a three-dimensional point cloud of the wafer surface specifically includes:

[0052] The wrapping phase is calculated pixel-by-pixel for the four distorted grating images, using the following formula:

[0053] ;

[0054] in, To wrap the phase, , , , There are four images in total;

[0055] The low-frequency absolute phase is obtained by expanding the low-frequency phase. Furthermore, the high-frequency phase is expanded using the relationship between high and low frequency phases, as shown in the following formula:

[0056] ;

[0057] in, For high-frequency absolute phase, For high-frequency wrapping phase, For the floor function, For low-frequency absolute phase, Low frequency, High frequency;

[0058] The absolute phase is obtained based on the high-frequency absolute phase and the low-frequency absolute phase. The three-dimensional point cloud is obtained by reconstructing the three-dimensional point cloud data of the wafer surface using a phase-height mapping model with preset calibration parameters. .

[0059] In this scheme, the step of obtaining a multi-channel feature tensor based on the 3D point cloud through feature extraction and data augmentation specifically includes:

[0060] Four feature maps are calculated from the three-dimensional point cloud, wherein,

[0061] Obtain the height map by taking the Z value of the 3D point cloud in the Z direction;

[0062] Calculate the gradient of height in the X and Y directions and calculate the magnitude to obtain a gradient magnitude map;

[0063] Calculate the average curvature or Gaussian curvature of each point in a 3D point cloud to obtain a local curvature map;

[0064] The LBP texture map is obtained by calculating the local binary pattern on the height map;

[0065] The four feature maps are concatenated along the channel dimension and then enhanced online to obtain a multi-channel feature tensor. Online enhancement methods include adding noise or simulating illumination.

[0066] In this scheme, the step of inputting the feature tensor into a trained dual-branch convolutional neural network model to obtain the model output specifically includes:

[0067] The dual-branch convolutional neural network model includes a semantic segmentation branch and a high-level regression and classification branch;

[0068] The feature tensor is input into the dual-branch convolutional neural network model. Based on the semantic segmentation branch, a fisheye defect segmentation mask is obtained, and based on the height regression and classification branches, a height deviation map and a defect category probability vector are obtained.

[0069] The final defect type is determined based on the defect category probability vector, whereby the defect categories include fisheye and needle segment difference.

[0070] The depth and diameter of the fisheye region are calculated as quantization parameters based on the fisheye defect segmentation mask and height deviation map.

[0071] The height difference between needle segments in different regions is calculated based on the height deviation map and used as the quantization parameter.

[0072] In this solution, all defect categories and quantitative parameters are summarized to generate structured inspection reports and / or visualization charts for external output. The visualization charts include defect distribution maps.

[0073] A third aspect of the present invention provides a computer-readable storage medium comprising a machine program for a visual inspection method of wafer surface defects, wherein when executed by a processor, the program implements the steps of a visual inspection method for wafer surface defects as described in any of the preceding claims.

[0074] A fourth aspect of the present invention provides a visual inspection device for wafer surface defects, comprising:

[0075] The system comprises an acquisition unit, a structured light projection unit, an optical platform unit, and a computing unit.

[0076] The acquisition unit includes a high-resolution monochrome CMOS camera for capturing wafer images;

[0077] The structured light projection unit includes a multi-mode projector for projecting a preset grating pattern onto the wafer surface.

[0078] The optical platform unit includes a telecentric lens, an active vibration isolation platform, and a displacement stage. The telecentric lens is used to eliminate perspective errors; the active vibration isolation platform is used to eliminate vibrations; and the displacement stage is used to fix the wafer and move and rotate it.

[0079] The computing unit includes an industrial computer with a high-performance GPU, used to execute a visual inspection method for wafer surface defects as described above.

[0080] In this scheme, the resolution of the camera in the acquisition unit is no less than 500w pixels, and the frame rate is no less than 30fps; the grating pattern projected by the structured light projection unit includes a four-step phase-shifting sinusoidal grating and a specific coded composite grating, wherein the light intensity distribution of the four-step phase-shifting sinusoidal grating is as follows:

[0081] ;in, It is the background light intensity. It's about stripe contrast. It is the phase field to be solved. The number of phases is and the phase difference is . .

[0082] This invention discloses a visual inspection method, system, storage medium, and device for wafer surface defects. By integrating multi-mode structured light projection and deep learning technology, it achieves high-speed, high-precision, and non-destructive inspection of nanoscale defects on wafer surfaces. It possesses excellent defect discrimination capabilities and robustness in industrial settings, and can output comprehensive defect quantification data. This provides a reliable online inspection solution for semiconductor manufacturing, with the following specific benefits:

[0083] 1. It achieves rapid 3D detection with nanometer-level precision. By combining multi-mode structured light projection with robust phase unfolding algorithm, it ensures nanometer-level longitudinal measurement accuracy while significantly improving detection efficiency through the combination of fast mode and high-precision mode, which can meet the cycle time requirements of online detection.

[0084] 2. It has excellent defect identification and differentiation capabilities. By extracting multi-dimensional features such as height, gradient, and curvature, and using a dual-branch deep learning network, this invention can accurately distinguish between fisheye (local pit) and needle segment difference (regional height difference), and accurately locate and quantify them, which is difficult to achieve with traditional two-dimensional methods or single three-dimensional measurement methods.

[0085] 3. It possesses strong robustness and environmental adaptability. The robust phase unrolling algorithm proposed in this invention effectively overcomes the phase entanglement problem at the edge of the pin segment difference. Multi-mode illumination and HDR imaging reduce the influence of uneven surface reflectivity. It has a higher tolerance to environmental vibration and illumination fluctuations than the interferometric method and is more suitable for industrial field environments.

[0086] 4. Comprehensive detection information that can guide process improvement: This invention outputs not only the presence or absence of defects, but also detailed quantitative data including the type of defect, precise location, geometric dimensions (fisheye diameter, depth), height difference (needle segment difference), etc. This high-value information can be directly fed back to process engineers to trace the root cause of the problem and optimize manufacturing parameters.

[0087] 5. High integration and strong automation: This invention automates the entire process from image acquisition to final result output without human intervention. Furthermore, the hardware platform is compactly designed and easy to integrate into existing wafer processing equipment (such as coating and developing machines and CMP equipment) to achieve in-situ, online quality monitoring. Attached Figure Description

[0088] Figure 1 A flowchart of a visual inspection method for wafer surface defects according to the present invention is shown;

[0089] Figure 2 A block diagram of a wafer surface defect visual inspection system according to the present invention is shown;

[0090] Figure 3A schematic diagram of the structure of a visual inspection device for wafer surface defects according to the present invention is shown. Detailed Implementation

[0091] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0092] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0093] Based on an in-depth analysis of the existing technical solutions in the background section, the following key drawbacks can be summarized in their ability to detect fisheye and pinhole defects in wafers:

[0094] 1. Insufficient or excessively costly ability to acquire three-dimensional topographic information: Two-dimensional image processing methods completely lack height information; although confocal microscopy and interferometry can acquire three-dimensional information, they are difficult to apply on a large scale in industrial settings due to slow speed, demanding environmental requirements, or high costs.

[0095] 2. The contradiction between detection efficiency and accuracy is prominent: high-precision detection methods (such as confocal and interferometry) often come at the cost of detection speed, which cannot meet the needs of semiconductor production lines for full-wafer, high-speed online detection.

[0096] 3. Sensitivity to environmental disturbances: In particular, interferometric measurement methods are extremely sensitive to vibration, temperature fluctuations, airflow, etc. Even slight environmental changes can lead to measurement failure, resulting in poor stability.

[0097] 4. Limited ability to identify specific defects: Existing methods lack dedicated feature extraction and recognition algorithms for "fisheye" and "needle-segment differences". For example, two-dimensional methods have difficulty distinguishing between surface dirt and real fisheye pits; interferometry is prone to errors when measuring discontinuous areas.

[0098] 5. Low system complexity and integration: Existing high-precision inspection systems are usually complex in structure and large in size, making it difficult to integrate them into existing wafer manufacturing equipment for online, real-time quality monitoring.

[0099] To address these shortcomings, the present invention aims to provide a high-speed and high-precision visual inspection method for fisheye and pinhole defects on wafer surfaces based on the fusion of multi-mode structured light projection and deep learning. The method aims to achieve rapid, accurate, and robust non-destructive inspection of fisheye and pinhole defects on wafer surfaces through innovative optical system design, combined with efficient 3D reconstruction algorithms and a dedicated intelligent defect recognition model.

[0100] Specifically, Figure 1 A flowchart of a visual inspection method for wafer surface defects according to this application is shown.

[0101] Firstly, such as Figure 1 As shown, this application discloses a visual inspection method for wafer surface defects, including the following steps:

[0102] S102, acquire multi-mode images, wherein images are acquired based on two preset modes;

[0103] S104, Based on the acquired wafer image, perform three-dimensional topography reconstruction and phase calculation to obtain the three-dimensional point cloud of the wafer surface;

[0104] S106, Based on the three-dimensional point cloud, feature extraction and data augmentation are performed to obtain a multi-channel feature tensor;

[0105] S108, The feature tensor is input into the trained dual-branch convolutional neural network model to obtain the model output, the model output including the defect type and quantization parameters;

[0106] S110, Generate a detection report based on the model output, and output the detection report.

[0107] It should be noted that in this embodiment, images are acquired through two modes. The first mode is the fast mode, which can quickly and initially locate suspected defect areas. The second mode is the fine mode (high-precision mode), which can accurately identify, classify, and quantify defects in the defect areas.

[0108] Furthermore, in this embodiment, the acquired wafer image is subjected to three-dimensional topography reconstruction and phase calculation to obtain a three-dimensional point cloud of the wafer surface. The process includes wrapping phase calculation, low-frequency and high-frequency phase unwrapping, and finally using a phase-height mapping model to perform 3D reconstruction to obtain the corresponding 3D point cloud. Then, based on the 3D point cloud, feature extraction and data augmentation are performed to obtain a multi-channel feature tensor. Finally, the feature tensor is input into a trained dual-branch convolutional neural network model to obtain the model output, and a detection report is generated based on the model output. The detection report can be output, for example, a visual output of the corresponding defect distribution map.

[0109] According to an embodiment of the present invention, the acquisition of multi-mode images specifically includes:

[0110] The preset displacement stage is controlled to move to the initial detection position, and a uniform illumination image of unstructured light is acquired for reflectivity correction.

[0111] Images of the wafer surface are acquired using a fast scanning mode to identify regions of interest;

[0112] Four deformable grating images of the region of interest or the entire wafer are acquired using a fine scanning mode.

[0113] It should be noted that, in this embodiment, during application, the DMD projector projects two sets of grating patterns onto the wafer surface. First, the displacement stage is moved to the initial detection position, and a uniform illumination image with unstructured light is acquired for reflectivity correction. This image is used for subsequent reflectivity correction to eliminate the influence of uneven reflectivity of the wafer surface material itself on the measurement results. Further, images of the wafer surface are acquired based on a fast scanning mode to identify the region of interest, corresponding to suspected defect areas. Then, four deformed grating images of the region of interest or the entire wafer are acquired based on a fine scanning mode, corresponding to... , , , .

[0114] According to an embodiment of the present invention, the step of performing three-dimensional topography reconstruction and phase calculation based on the acquired wafer image to obtain a three-dimensional point cloud of the wafer surface specifically includes:

[0115] The wrapping phase is calculated pixel-by-pixel for the four distorted grating images, using the following formula:

[0116] ;

[0117] in, To wrap the phase, , , , There are four images in total;

[0118] The low-frequency absolute phase is obtained by expanding the low-frequency phase. Furthermore, the high-frequency phase is expanded using the relationship between high and low frequency phases, as shown in the following formula:

[0119] ;

[0120] in, For high-frequency absolute phase, For high-frequency wrapping phase, For the floor function, For low-frequency absolute phase, Low frequency, High frequency;

[0121] The absolute phase is obtained based on the high-frequency absolute phase and the low-frequency absolute phase. The three-dimensional point cloud is obtained by reconstructing the three-dimensional point cloud data of the wafer surface using a phase-height mapping model with preset calibration parameters. .

[0122] It should be noted that, in this embodiment, the wrapping phase is calculated pixel-by-pixel for the four deformed grating images. Due to the properties of the arctangent function, The phase is enclosed within the interval [-π, π], hence it is called the enclosed phase. This is to obtain a continuous absolute phase. Phase unwrapping is required, but traditional spatiotemporal phase unwrapping methods are prone to failure when faced with drastic height variations (such as pin differences) on the wafer surface. Therefore, this invention proposes a robust phase unwrapping algorithm guided by multiple frequencies. This algorithm additionally projects a set of frequencies... The (low-frequency) sinusoidal grating is first expanded to determine its low-frequency phase. Because its fringes are relatively wide, phase transitions are less likely to occur, making expansion relatively easy. The expanded low-frequency absolute phase is then obtained. Then, the high-frequency absolute phase is obtained by guiding the expansion of the high-frequency phase using the relationship between high and low frequency phases. After obtaining the absolute phases of the low and high frequencies, the three-dimensional point cloud S(x,y,z) of the wafer surface can be reconstructed using pre-calibrated calibration parameters (including the intrinsic and extrinsic parameters of the camera and projector) and the phase-height mapping model or triangulation principle. The phase-height mapping can typically be expressed as: ;in, For phase-height mapping, These are coefficients related to the system's geometry. It is the phase distribution of the reference plane, obtained through calibration.

[0123] According to an embodiment of the present invention, the step of obtaining a multi-channel feature tensor based on the three-dimensional point cloud through feature extraction and data augmentation specifically includes:

[0124] Four feature maps are calculated from the three-dimensional point cloud, wherein,

[0125] Obtain the height map by taking the Z value of the 3D point cloud in the Z direction;

[0126] Calculate the gradient of height in the X and Y directions and calculate the magnitude to obtain a gradient magnitude map;

[0127] Calculate the average curvature or Gaussian curvature of each point in a 3D point cloud to obtain a local curvature map;

[0128] The LBP texture map is obtained by calculating the local binary pattern on the height map;

[0129] The four feature maps are concatenated along the channel dimension and then enhanced online to obtain a multi-channel feature tensor. Online enhancement methods include adding noise or simulating illumination.

[0130] It should be noted that, in this embodiment, directly identifying micron-level fisheye defects and nanometer-level pinhole defects from a 3D point cloud is extremely challenging. This invention does not directly input the original point cloud into the network, but instead extracts multi-dimensional feature maps sensitive to these two types of defects. Specifically, four feature maps are calculated from the 3D point cloud, where the height map is obtained by taking the Z-value of the 3D point cloud in the Z-direction. It directly reflects the surface undulations of the wafer; and calculates the gradient of height in the X and Y directions and calculates the magnitude to obtain a gradient magnitude map. This is used to enhance defect edges; and to calculate the average curvature or Gaussian curvature of each point in the 3D point cloud to obtain a local curvature map. Calculating the average curvature or Gaussian curvature of each point in the point cloud is very effective for identifying local concave features such as fisheyes. The calculation can be approximated as:

[0131] ;

[0132] in, It is height The gradient in the x and y directions, The second-order gradient is used, and the local binary pattern is calculated on the height map to obtain the LBP texture map, which is used to capture micro-texture changes. Then, the four feature maps are spliced ​​in the channel dimension to form a multi-channel feature tensor, which is used as the input of the subsequent deep learning network. In order to improve the generalization ability of the model, this embodiment also includes online data augmentation of the feature tensor, including randomly adding Gaussian noise to simulate measurement error, simulate small illumination changes, and random elastic deformation.

[0133] According to an embodiment of the present invention, the step of inputting the feature tensor into a trained dual-branch convolutional neural network model to obtain the model output specifically includes:

[0134] The dual-branch convolutional neural network model includes a semantic segmentation branch and a high-level regression and classification branch;

[0135] The feature tensor is input into the dual-branch convolutional neural network model. Based on the semantic segmentation branch, a fisheye defect segmentation mask is obtained, and based on the height regression and classification branches, a height deviation map and a defect category probability vector are obtained.

[0136] The final defect type is determined based on the defect category probability vector, whereby the defect categories include fisheye and needle segment difference.

[0137] The depth and diameter of the fisheye region are calculated as quantization parameters based on the fisheye defect segmentation mask and height deviation map.

[0138] The height difference between needle segments in different regions is calculated based on the height deviation map and used as the quantization parameter.

[0139] It should be noted that, in this embodiment, the dual-branch convolutional neural network model includes a semantic segmentation branch and a height regression and classification branch. The semantic segmentation branch is based on the U-Net architecture, with a multi-channel feature tensor as input and a binary segmentation mask of the same resolution as the input. This mask is used for pixel-level localization of fisheye defect regions. The loss function employs a composite loss function combining Dice loss and cross-entropy to address the class imbalance problem between defects and the background. The height regression and classification branch adopts an encoder-decoder structure, where the encoder is shared with the segmentation branch to extract common features. The decoder is divided into two sub-heads: a height regression head, which outputs a single-channel image of the same resolution as the input, where each pixel value represents the relative height deviation of that point. The output is used to accurately calculate the height value of the pin segment difference. And the defect classification head: performs pooling and fully connected operations on the global features extracted by the encoder, and outputs the defect category probability of each preset detection area (e.g., no defect, only fisheye, only pin segment difference, mixed defect).

[0140] Furthermore, during network training, the two branches are trained jointly and optimized using a weighted total loss function: ,in, , , These are segmentation loss, regression loss, and classification loss. The weights are balanced, and after training, the feature tensor is input into the dual-branch convolutional neural network model. A fisheye defect segmentation mask is obtained based on the semantic segmentation branch, and a height deviation map and defect category probability vector are obtained based on the height regression and classification branches. The final defect type is determined based on the defect category probability vector, and the defect categories include fisheye and needle segment difference. The depth and diameter of the fisheye region are calculated as quantization parameters based on the fisheye defect segmentation mask and the height deviation map. The height difference of needle segment differences between different regions is calculated as the quantization parameter based on the height deviation map.

[0141] According to an embodiment of the present invention, all defect categories and quantitative parameters are summarized to generate a structured inspection report and / or a visualization chart for external output, wherein the visualization chart includes a defect distribution map.

[0142] It should be noted that, in this embodiment, the dual-branch convolutional neural network of the present invention is used to identify, locate and quantify defects to obtain defect categories and quantification parameters, and then all data are summarized to generate a structured inspection report, such as JSON or XML format, and / or a visualization chart (including a defect distribution map). In practical applications, this can trigger a sorting signal to trigger sorting or be uploaded to the Manufacturing Execution System (MES) to guide wafer unloading.

[0143] Figure 2 A block diagram of a wafer surface defect visual inspection system according to the present invention is shown.

[0144] Secondly, such as Figure 2 As shown, this invention discloses a wafer surface defect visual inspection system, including a memory and a processor. The memory includes a wafer surface defect visual inspection method program, which, when executed by the processor, performs the following steps:

[0145] Acquire multi-mode images, including images acquired based on two preset modes;

[0146] Three-dimensional topography reconstruction and phase calculation are performed based on the acquired wafer images to obtain the three-dimensional point cloud of the wafer surface;

[0147] Based on the three-dimensional point cloud, feature extraction and data augmentation are performed to obtain a multi-channel feature tensor;

[0148] The feature tensor is input into a trained dual-branch convolutional neural network model to obtain the model output, which includes the defect type, defect location, and quantization parameters.

[0149] A detection report is generated based on the model output, and the detection report is output.

[0150] It should be noted that, in this embodiment, when the wafer surface defect visual inspection system disclosed in this invention is applied, the specific process corresponds to the wafer surface defect visual inspection method described in the above embodiment. Since the specific implementation details of the system application are consistent with the content of the above wafer surface defect visual inspection, no further details will be provided in this embodiment.

[0151] A third aspect of the present invention provides a computer-readable storage medium comprising a wafer surface defect visual inspection method program, wherein when the wafer surface defect visual inspection method program is executed by a processor, it implements the steps of a wafer surface defect visual inspection method as described in any of the preceding claims.

[0152] A fourth aspect of the present invention provides a visual inspection device for wafer surface defects, comprising:

[0153] The system comprises an acquisition unit, a structured light projection unit, an optical platform unit, and a computing unit.

[0154] The acquisition unit includes a high-resolution monochrome CMOS camera for capturing wafer images;

[0155] The structured light projection unit includes a multi-mode projector for projecting a preset grating pattern onto the wafer surface.

[0156] The optical platform unit includes a telecentric lens, an active vibration isolation platform, and a displacement stage. The telecentric lens is used to eliminate perspective errors; the active vibration isolation platform is used to eliminate vibrations; and the displacement stage is used to fix the wafer and move and rotate it.

[0157] The computing unit includes an industrial computer with a high-performance GPU, used to execute a visual inspection method for wafer surface defects as described above.

[0158] It should be noted that, in this embodiment, as Figure 3 As shown, this is a schematic diagram of the device structure. The acquisition unit includes a high-resolution monochrome CMOS camera for capturing wafer images. It has a high dynamic range (HDR) function to ensure accurate capture of subtle grayscale changes under structured light projection, avoiding overexposure or underexposure. The structured light projection unit includes a multi-mode projector for projecting a preset grating pattern onto the wafer surface. The core component is a digital micromirror device (DMD) projector, which can project the preset grating pattern at high speed and with high precision. This module can project pattern sequences in two modes: one is a four-step phase-shift sinusoidal grating for high-precision 3D reconstruction, and the other is a composite grating with specific coding for rapid defect screening. The optical platform unit includes a telecentric lens, an active vibration isolation platform (not shown), and a displacement stage. The telecentric lens is used to eliminate perspective errors. The active vibration isolation platform is used to eliminate vibrations. The displacement stage is used to fix the wafer and move and rotate it. The wafer is fixed on the displacement stage by a precision vacuum chuck. The displacement stage can achieve precise movement in the XY direction and rotation in the θ direction to achieve full wafer scanning and inspection. The computing unit includes an industrial computer with a high-performance GPU (not shown) for executing a wafer surface defect visual inspection method as described above.

[0159] Furthermore, in this embodiment, the resolution of the acquisition unit camera is not less than 500w pixels, and the frame rate is not less than 30fps; the grating pattern projected by the structured light projection unit includes a four-step phase-shift sinusoidal grating and a specifically coded composite grating. Correspondingly, a specially designed specifically coded composite grating is projected, which integrates multiple spatial frequency and direction information for quickly and initially locating suspected defect areas and narrowing the scope of subsequent fine inspection. Furthermore, the light intensity distribution of the four-step phase-shift sinusoidal grating is as follows:

[0160] ;

[0161] in, It is the background light intensity. It's about stripe contrast. It is the phase field to be solved. The number of phases is and the phase difference is . .

[0162] This invention discloses a visual inspection method, system, storage medium, and device for wafer surface defects. By integrating multi-mode structured light projection and deep learning technology, it achieves high-speed, high-precision, and non-destructive inspection of nanoscale defects on wafer surfaces. It also has excellent defect discrimination capabilities and robustness in industrial settings, and can output comprehensive defect quantification data, providing a reliable online inspection solution for semiconductor manufacturing.

[0163] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0164] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0165] In addition, in the various embodiments of the present invention, each functional unit can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0166] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0167] Alternatively, if the integrated units of this invention are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

Claims

1. A method for visual inspection of defects on a wafer surface, characterized in that, Includes the following steps: Acquire multi-mode images, including images acquired based on two preset modes; Three-dimensional topography reconstruction and phase calculation are performed based on the acquired wafer images to obtain the three-dimensional point cloud of the wafer surface; Based on the three-dimensional point cloud, feature extraction and data augmentation are performed to obtain a multi-channel feature tensor; The feature tensor is input into a trained dual-branch convolutional neural network model to obtain the model output, which includes the defect type and quantization parameters. A detection report is generated based on the model output, and the detection report is output.

2. The method for visual inspection of wafer surface defects according to claim 1, characterized in that, The acquisition of multi-mode images specifically includes: The preset displacement stage is controlled to move to the initial detection position, and a uniform illumination image of unstructured light is acquired for reflectivity correction. Images of the wafer surface are acquired using a fast scanning mode to identify regions of interest; Four deformable grating images of the region of interest or the entire wafer are acquired using a fine scanning mode.

3. The method for visual inspection of wafer surface defects according to claim 2, characterized in that, The process of performing 3D topography reconstruction and phase calculation based on the acquired wafer images to obtain a 3D point cloud of the wafer surface specifically includes: The wrapping phase is calculated pixel-by-pixel for the four distorted grating images, using the following formula: ； in, To wrap the phase, , , , There are four images in total; The low-frequency absolute phase is obtained by expanding the low-frequency phase. Furthermore, the high-frequency phase is expanded using the relationship between high and low frequency phases, as shown in the following formula: ； in, For high-frequency absolute phase, For high-frequency wrapping phase, For the floor function, For low-frequency absolute phase, Low frequency, High frequency; The absolute phase is obtained based on the high-frequency absolute phase and the low-frequency absolute phase. The three-dimensional point cloud is obtained by reconstructing the three-dimensional point cloud data of the wafer surface using a phase-height mapping model with preset calibration parameters. .

4. The method for visual inspection of wafer surface defects according to claim 3, characterized in that, The process of extracting features and augmenting data based on the 3D point cloud to obtain a multi-channel feature tensor specifically includes: Four feature maps are calculated from the three-dimensional point cloud, wherein, Obtain the height map by taking the Z value of the 3D point cloud in the Z direction; Calculate the gradient of height in the X and Y directions and calculate the magnitude to obtain a gradient magnitude map; Calculate the average curvature or Gaussian curvature of each point in a 3D point cloud to obtain a local curvature map; The LBP texture map is obtained by calculating the local binary pattern on the height map; The four feature maps are concatenated along the channel dimension and then enhanced online to obtain a multi-channel feature tensor. Online enhancement methods include adding noise or simulating illumination.

5. The method for visual inspection of wafer surface defects according to claim 4, characterized in that, The step of inputting the feature tensor into the trained dual-branch convolutional neural network model to obtain the model output specifically includes: The dual-branch convolutional neural network model includes a semantic segmentation branch and a high-level regression and classification branch; The feature tensor is input into the dual-branch convolutional neural network model. Based on the semantic segmentation branch, a fisheye defect segmentation mask is obtained, and based on the height regression and classification branches, a height deviation map and a defect category probability vector are obtained. The final defect type is determined based on the defect category probability vector, whereby the defect categories include fisheye and needle segment difference. The depth and diameter of the fisheye region are calculated as quantization parameters based on the fisheye defect segmentation mask and height deviation map. The height difference between needle segments in different regions is calculated based on the height deviation map and used as the quantization parameter.

6. The method for visual inspection of wafer surface defects according to claim 5, characterized in that, Summarize all defect categories and quantitative parameters to generate structured inspection reports and / or visualization charts for external output. The visualization charts include defect distribution maps.

7. A visual inspection system for wafer surface defects, characterized in that, The system includes a memory and a processor. The memory contains a program for visually inspecting wafer surface defects. When the processor executes the program for visually inspecting wafer surface defects, it performs the following steps: Acquire multi-mode images, including images acquired based on two preset modes; Three-dimensional topography reconstruction and phase calculation are performed based on the acquired wafer images to obtain the three-dimensional point cloud of the wafer surface; Based on the three-dimensional point cloud, feature extraction and data augmentation are performed to obtain a multi-channel feature tensor; The feature tensor is input into a trained dual-branch convolutional neural network model to obtain the model output, which includes the defect type, defect location, and quantization parameters. A detection report is generated based on the model output, and the detection report is output.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a wafer surface defect visual inspection method program, which, when executed by a processor, implements the steps of a wafer surface defect visual inspection method as described in any one of claims 1 to 6.

9. A visual inspection device for wafer surface defects, characterized in that, include: The system comprises an acquisition unit, a structured light projection unit, an optical platform unit, and a computing unit. The acquisition unit includes a high-resolution monochrome CMOS camera for capturing wafer images; The structured light projection unit includes a multi-mode projector for projecting a preset grating pattern onto the wafer surface. The optical platform unit includes a telecentric lens, an active vibration isolation platform, and a displacement stage. The telecentric lens is used to eliminate perspective errors; the active vibration isolation platform is used to eliminate vibrations; and the displacement stage is used to fix the wafer and move and rotate it. The computing unit includes an industrial computer with a high-performance GPU, used to execute a wafer surface defect visual inspection method according to any one of claims 1-6.

10. A wafer surface defect visual inspection device according to claim 9, characterized in that, The camera of the acquisition unit has a resolution of no less than 500w pixels and a frame rate of no less than 30fps; the grating pattern projected by the structured light projection unit includes a four-step phase-shifting sinusoidal grating and a specific coded composite grating, wherein the light intensity distribution of the four-step phase-shifting sinusoidal grating is as follows: ;in, It is the background light intensity. It's about stripe contrast. It is the phase field to be solved. The number of phases is and the phase difference is . .

Images