Deep learning semiconductor defect microscopic super-resolution method based on physical guidance

Abstract

The invention provides a deep learning semiconductor defect microscopic super-resolution method based on physical guidance, and the method comprises the steps: obtaining an accurate point spread function through a microscopic imaging system, building an accurate mathematical degradation model through the point spread function, and calculating the difference between an image and a condition image based on the degradation model. And the obtained gradient information is used for restraining diffusion model image prediction. And finally, gradient information is used as a guide item to be introduced into backward diffusion in the denoising diffusion implicit model, and an offset item is introduced into mean value prediction in the diffusion process of each step so as to prevent the image from generating artifacts and non-existing information.

Description

Technical Field

[0001] This invention relates to the field of image processing technology for optical microscopic detection of semiconductor defects, and particularly to a physical-guided deep learning-based super-resolution method for semiconductor defect microscopy. Background Technology

[0002] Microscopic imaging systems are crucial tools for detecting semiconductor defects and improving chip yield. Currently, numerous microscopes are used for semiconductor inspection. Acoustic scanning microscopes can scan large areas of IC chip wafers, but their imaging resolution is insufficient (>20µm). While scanning electron microscopes and transmission electron microscopes can achieve nanoscale detection, their high equipment cost and complex sample preparation processes reduce semiconductor inspection efficiency. Dark-field microscopy systems are primarily sensitive to numerous scratch defects, posing a greater challenge for patterned IC chip wafers. Wide-field and confocal microscopes, with their advantages of speed, non-destructive testing, and high system integration, have become the most widely used microscopes for semiconductor defect detection. In recent years, with the continuous miniaturization and complexity of chip processes, defect sizes have gradually decreased, and the chip manufacturing industry has continuously increased its requirements for semiconductor inspection accuracy. Image quality remains a key factor determining defect detection accuracy. However, while confocal and wide-field microscopes pursue high-quality imaging, they sacrifice the imaging field of view, significantly increasing the inspection time for large-size IC chip wafers (200mm and 300mm), making it difficult to meet the current demands for high precision and throughput in semiconductor inspection. Furthermore, semiconductor materials inherently suffer from low optical imaging contrast, and structural details are easily and significantly attenuated during the imaging process. Therefore, improving the imaging quality of current wide-field and confocal microscopy used for semiconductor inspection has become a key technological requirement for enhancing the accuracy of semiconductor inspection. Images carry crucial information in machine vision, and image processing is the core of semiconductor inspection. Traditional optical microscopy often relies on deconvolution and regularization techniques to address imaging degradation; however, these methods struggle to recover complex image details. Deep neural networks, with their ability to process highly nonlinear mappings, can capture the structural texture of high-resolution optical images.

[0003] Therefore, a pressing technical problem for those skilled in the art is that existing purely data-driven deep learning super-resolution or image reconstruction methods, lacking physical imaging constraints, often fail to guarantee consistency between the reconstructed results and the real physical scene, easily introducing artifacts or unreliable details. This not only reduces the credibility of the reconstructed image but also limits the practical application value of these methods in scenarios such as industrial inspection, semiconductor wafer defect detection, and high-precision optical imaging. Therefore, how to improve the accuracy and stability of reconstruction by introducing physical model constraints while maintaining the efficient learning capabilities of deep learning models has become an urgent problem to be solved. Summary of the Invention

[0004] To address the problems in existing technologies, this invention proposes a physics-guided deep learning-based super-resolution method for semiconductor defect microscopy. This method uses high-frequency information from a physical imaging degradation model to guide a denoising diffusion implicit model, enabling rapid reconstruction of high-quality defect images while enhancing physical consistency, thus achieving super-resolution microscopic image prediction.

[0005] This invention is achieved through the following technical solution: This invention proposes a physical-guided deep learning-based super-resolution microscopic method for semiconductor defects, the method specifically being: Step 1: Based on the actual optical microscope system, establish an imaging model and obtain the point spread function (PSF) of the microscope to accurately represent the imaging effect and blurring characteristics of the optical system on light. Step 2: Acquire images using low- and high-power objectives of an optical microscope, and amplify the data by rotation and random cropping to obtain image pairs for network training and validation; Step 3: Using the point spread function obtained in Step 1, establish a degradation model for the low-resolution image, calculate the high-frequency gradient information of the image, and use the gradient information as a physical constraint to guide the subsequent diffusion process. Step 4: In the reverse reconstruction process of the HP-DDIM model, the gradient information obtained in Step 3 is added as the mean offset term to the image update in each step to guide the image generation to be more in line with the physical imaging law and reduce false structures or artifacts. Step 5: Complete network training through multiple backdiffusion iterations; at this point, input a low-resolution semiconductor image and output a high-resolution reconstructed image that meets physical degradation constraints.

[0006] Furthermore, in step one, standard samples or waveplates are made using polystyrene micro / nanoparticles. By acquiring imaging data from tiny point light sources under a microscope, the point spread function of the system is measured and calculated to accurately characterize the imaging characteristics of the microscope.

[0007] Furthermore, in step three, the point spread function obtained in step one is used to model the image pairs acquired in step two to construct a degradation model for low-resolution images. The main purpose of constructing this degradation model is to calculate the loss of detail in the image during the imaging degradation process and extract the high-frequency gradient information. The obtained high-frequency gradient information truly reflects the changes in edge and texture features and is used as a physical constraint to guide image reconstruction in the subsequent diffusion process.

[0008] Furthermore, in step four, during the reverse reconstruction of the HP-DDIM model, the gradient information obtained in step three is added as a mean offset term to each image update. DDIM will gradually remove noise and generate a clear image during iteration. Introducing gradient information is equivalent to providing a correction direction based on the physical imaging law in each update.

[0009] Furthermore, the HP-DDIM backward process uses a multi-scale residual U-net network model to learn the conditional transition distribution, and the mean squared error loss function optimizes the training process, as expressed by the formula: (8) (8) Substituting into the DDIM diffusion formula, we can obtain (9) Among them, hyperparameters Record To simplify the model, It is set to a constant.

[0010] Furthermore, to avoid numerical instability caused by the guiding gradient at different time steps, an adaptive guiding coefficient is introduced. ,in It involves adjusting the gradient weight parameters.

[0011] The beneficial effects of this invention are: This invention proposes a physics-guided deep learning-based super-resolution method for semiconductor defect microscopy. The method first proposes using high-frequency information from a physical imaging degradation model to guide a denoising diffusion implicit model (HP-DDIM), which enhances physical consistency while rapidly reconstructing high-quality defect images. The method obtains an accurate point spread function through a microscopic imaging system, establishes a precise mathematical degradation model based on this function, and calculates the difference between the image and the conditional image based on this degradation model. The acquired gradient information is used to constrain the image prediction of the diffusion model. Finally, the gradient information is introduced as a guiding term into the backdiffusion of the denoising diffusion implicit model. In each diffusion step, the mean prediction incorporates an offset term (gradient information) to prevent image artifacts and missing information. Attached Figure Description

[0012] Figure 1 This is the overall system flowchart of the present invention.

[0013] Figure 2 This is an overall flowchart of the method described in this invention.

[0014] Figure 3 This is a network structure diagram of the invention predicting the noise of the diffusion model.

[0015] Figure 4This is a schematic diagram of the wide-field image reconstruction and confocal reconstruction results achieved by the method of this invention.

[0016] Figure 5 This is a schematic diagram of the super-resolution results of wide-field and confocal microscopy in this invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] This invention provides a physics-guided deep learning-based super-resolution method for semiconductor defect microscopy. First, based on an optical microscope system used in semiconductor imaging applications, a complete optical imaging model is established, and the point spread function (PSF) of the optical microscope is obtained to accurately characterize the system's imaging properties. Then, a degradation model combined with the PSF is used to describe the formation process of a low-resolution image, and the difference between the reconstructed image and the image under observation conditions is calculated to obtain gradient information, which is used to physically constrain the image prediction of the deep generation model. Finally, the gradient information is embedded as a guiding term in the backdiffusion process of the Denoising Diffusion Implicit Model (DDIM), introducing gradient offset in the mean prediction of each diffusion step, effectively limiting the solution space of the generated image. This method not only suppresses artifacts and non-existent image information but also improves the resolution and structural detail reconstruction accuracy of semiconductor microscopic images while maintaining the high expressive power of the deep learning model, providing a reliable high-resolution imaging means for industrial inspection.

[0019] Specifically, see Figures 1-5 This invention proposes a physical-guided deep learning-based super-resolution method for semiconductor defect microscopy, the method specifically being: Step 1: Based on the actual optical microscope system, establish an imaging model and obtain the point spread function (PSF) of the microscope to accurately represent the imaging effect and blurring characteristics of the optical system on light. Step 2: Acquire images using low- and high-magnification objectives of an optical microscope, and augment the data through rotation and random cropping to obtain image pairs for network training and validation. The acquired image pairs include low-resolution images and high-resolution reference images. Low-resolution images are obtained through low-magnification objectives or downsampling, while high-resolution images are obtained through high-magnification objectives or existing super-resolution methods. To enhance data diversity, the acquired images undergo augmentation processing, such as rotation, flipping, cropping, and noise perturbation, thereby obtaining more training samples and enriching image features.

[0020] Step 3: Using the point spread function obtained in Step 1, establish a degradation model for the low-resolution image, calculate the high-frequency gradient information of the image, and use the gradient information as a physical constraint to guide the subsequent diffusion process; Step 3 introduces the gradient information as a guiding term into the backward diffusion of DDIM, and in each diffusion process, the mean prediction introduces an offset term (gradient information) to prevent the generation of image artifacts and non-existent information.

[0021] Step 4: In the reverse reconstruction process of the HP-DDIM model, the gradient information obtained in Step 3 is added as the mean offset term to the image update in each step to guide the image generation to be more in line with the physical imaging law and reduce false structures or artifacts. Step 5: Complete network training through multiple backdiffusion iterations; at this point, input a low-resolution semiconductor image and output a high-resolution reconstructed image that meets physical degradation constraints. Specifically, inputting a low-resolution semiconductor image at this point can output an image that retains clear details while satisfying physical degradation constraints, thereby improving the identifiability of semiconductor wafer defects or microstructures.

[0022] In step one, in order to obtain the point spread function of the optical microscopy system, standard samples or waveplates are made using polystyrene micro-nano particles. By collecting imaging data of tiny point light sources under the microscope, the point spread function of the system is measured and calculated to accurately characterize the imaging characteristics of the microscope.

[0023] In step three, the point spread function obtained in step one is used to model the image pairs acquired in step two, and a degradation model of the low-resolution image is constructed. The main purpose of constructing this degradation model is to calculate the loss of detail in the image during the imaging degradation process and extract the high-frequency gradient information. The obtained high-frequency gradient information truly reflects the changes in edge and texture features and is used as a physical constraint to guide image reconstruction in the subsequent diffusion process.

[0024] In step four, during the reverse reconstruction of the HP-DDIM model, the gradient information obtained in step three is added as a mean offset term to each image update. DDIM iterates to gradually remove noise and generate a clear image; introducing gradient information effectively provides a correction direction based on physical imaging principles at each update. Thus, image generation no longer relies entirely on the model's statistical learning results but is also constrained by the real optical system. Throughout multiple iterations, the gradient information continuously plays a role, enabling the generated image to gradually avoid false structures or unreasonable details and achieve more accurate restoration in edge and texture regions. The resulting high-resolution image, while maintaining improved resolution, more closely resembles actual imaging characteristics.

[0025] Example To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Specific embodiments include the following: like Figure 1 The diagram illustrates the overall workflow of HP-DDIM for improving image quality in semiconductor inspection optical microscopy. This system possesses dual-mode high-definition imaging capabilities, including confocal and wide-field modes, with PSF and FWHM reaching 350nm and 150nm respectively. To achieve efficient dual-modal image acquisition, a beam splitter (BS) is designed in front of the shared objective lens, enabling rapid switching between wide-field and confocal modes and common-path imaging. The acquired paired image datasets drive HP-DDIM to recover missing structural information from the images.

[0026] The detailed principle of the method proposed in this invention is as follows: Figure 2 As shown, the original low-quality IC chip wafer image is used as input, and the output is a reconstructed high-quality image. HP-DDIM can be divided into two parts: a diffusion network and a prediction network. The diffusion part uses high-frequency physical information from the degradation model to guide DDIM diffusion, which can improve the physical consistency and reconstruction speed of the IC chip wafer image. The prediction network uses... Figure 3 The lightweight multi-scale residual U-Net network shown learns noise and gradient information, which can balance local details and global structural information of the target image, reducing learning parameters while ensuring inference efficiency. The specific derivation of the gradient information and mean shift of the degradation model is as follows: Images observed by an optical microscope , The images are not representations of the true structure, but rather the result of image degradation caused by the optical system. The goal is to recover the true image structure from the observed images. , Degeneracy models are often used. Inversion can be represented as . It is a nonlinear response in the imaging system. Interference noise w and h represent the image's length and width.

[0027] (1) in It is the prior regularization term introduced for the microscopic system. It reconstructs semiconductor images. It is the gradient of the data likelihood. Represents network prediction. Utilizes functions. After converting the image to the frequency domain, high-frequency components are preserved, and an inverse transformation is performed to obtain the high-frequency structure of the image domain. HP-DDIM uses physical information to guide the unconditional DDIM backward process for conditional processing, generating samples related to x. Therefore, according to Bayes' theorem, it can be decomposed into: (2) (3) when The probability approaches 1, and the backward diffusion process ( →0) tends towards certainty, focusing on Nearby. The log-likelihood of the mean Taylor's expansion: (4) Backdiffusion is known to follow a deterministic Gaussian distribution. Therefore, it can Likelihood logarithm expansion: (5) (6) C is and The irrelevant constant term, when further rounded, still follows a Gaussian distribution, but its mean is shifted. : (7) in The forward propagation processes of DDIM and DDPM are identical, except that DDIM does not include random noise. Therefore, DDPM can still be used as a reference. Construct offset. It is a hyperparameter used to control the intensity of the offset.

[0028] The HP-DDIM backward process uses a multi-scale residual U-net network model to learn the conditional transition distribution, and the mean squared error loss function optimizes the training process. The formula is as follows: (8) (8) Substituting into the DDIM diffusion formula, we can obtain (9) Among them, hyperparameters Record To simplify the model, It is set as a constant. In this invention, the gradient term is redefined to more effectively characterize the degradation model of the optical microscope system, thereby achieving high-quality inverse reconstruction optimization. It is assumed that the noise introduced by the microscope degradation model... For an approximate Gaussian distribution, according to Bayes' theorem, the gradient G can be written as: (10) (11) It is the variance. The final gradient term, derived mathematically, is shown in formula (4). To avoid numerical instability caused by the guiding gradient at different time steps, an adaptive guiding coefficient is introduced. ,in It involves adjusting the gradient weight parameters.

[0029] Raw image data of semiconductor wafers were acquired using optical microscopes such as confocal and wide-field microscopes, covering samples at different magnifications and imaging conditions. Subsequently, the acquired images underwent preprocessing operations, including image spatial alignment, to ensure consistency of feature locations. The semiconductor wafer images were standardized and cropped, and data augmentation operations such as rotation, flipping, and scaling were performed to enrich the diversity of training samples. This data was ultimately used as the training dataset for HP-DDIM.

[0030] like Figure 2 As shown, during the network training phase, the HP-DDIM model proposed in this invention is used to complete image training and prediction, compared with traditional pure data-driven deep learning methods. This method improves both the network structure and training mechanism. First, a multi-scale residual U-net network (such as...) is used. Figure 3 As shown, the diffusion noise distribution is modeled at each step, and high-frequency gradient information of the physical degradation model is learned simultaneously. This gradient information can accurately reflect the details and edge features lost during actual imaging, thus playing a role in constraining and correcting the model's backdiffusion. Specifically, in each mean update, this high-frequency gradient information is used as an offset guide, so that the model does not completely follow the network statistical results, generating a super-resolution image that conforms to physical laws.

[0031] Figure 4 The results of wide-field image reconstruction and confocal reconstruction using the method of this invention are shown. (a) are the WF (input), confocal (GT), and virtual confocal (WVC) images of the logic functional area sample of the display chip, arranged from left to right. The following figure is a magnified image within the blue dashed box, clearly showing that HP-DDIM achieves high-resolution virtual confocal reconstruction of the WF image. (b) is the Fourier transform spectrum of the optical microscopic image. Figure 5These are the super-resolution results of wide-field and confocal microscopy in this invention. (c) and (d) show the contrast in line profile intensity along the white dashed line in (a) and (b), respectively. (e) and (f) are the Relative Scale Pearson (RSP) evaluation metrics for assessing the resolution scale correlation between conventional WF, confocal images, and HP-DDIM reconstructed images, with resolution calibrated using average scaling ratio (ASR). (g) and (h) are the structural similarity (SSIM) and peak signal-to-noise ratio (PSNR) metrics for WF and confocal microscopy reconstructed images. (i) compares the Local Defect Quality (LDQ) defect quality evaluation metrics using blue boxes.

[0032] The above provides a detailed description of the physical-guided deep learning-based semiconductor defect microscopic super-resolution method proposed in this invention. Specific examples have been used to illustrate the principles and implementation methods of this invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this invention. Therefore, the content of this specification should not be construed as a limitation of this invention.

Claims

1. A physically-guided deep learning method for semiconductor defect microscopic super-resolution, characterized in that, The method is specifically: Step one, according to the actual optical microscope system, an imaging model is established, and the point spread function (PSF) of the microscope is obtained to accurately represent the imaging effect and blur characteristics of the optical system on light; Step two, images are collected using low-power and high-power objectives of the optical microscope, and image pairs for network training and verification are obtained by rotating and randomly cropping and amplifying the data; Step three, the point spread function obtained in step one is used to establish a degradation model of the low-resolution image, and the high-frequency gradient information of the image is calculated, which is used as a physical constraint to guide the subsequent diffusion process; Step four, in the reverse reconstruction process of the HP-DDIM model, the gradient information obtained in step three is added as a mean shift term to the image update of each step, guiding the image generation to be more consistent with the physical imaging law and reducing false structures or artifacts; Step five, network training is completed through multiple reverse diffusion iterations; at this time, a low-resolution semiconductor image is input, and a high-resolution reconstructed image consistent with the physical degradation constraint is output.

2. The method of claim 1, wherein, In step one, polystyrene micro-nanoparticles are used to make standard samples or wave plates, and the imaging data of the micro point light source is collected under the microscope to measure and calculate the point spread function of the system to accurately represent the imaging characteristics of the microscope.

3. The method of claim 1, wherein, In step three, the point spread function obtained in step one is used to model the image pairs collected in step two, and a degradation model of the low-resolution image is constructed; the main purpose of constructing this degradation model is to calculate the detail loss of the image in the imaging degradation process and extract the high-frequency gradient information; the obtained high-frequency gradient information truly reflects the changes of edge and texture features, which is used as a physical constraint condition to guide the image reconstruction in the subsequent diffusion process.

4. The method of claim 1, wherein, In step four, in the reverse reconstruction process of the HP-DDIM model, the gradient information obtained in step three is added as a mean shift term to the image update of each step; DDIM gradually removes noise and generates a clear image during iteration, and after introducing the gradient information, it is equivalent to giving a correction direction based on the physical imaging law at each update.

5. The method of claim 4, wherein, The HP-DDIM backward process is to learn the conditional transfer distribution using a multi-scale residual U-net network model, and the mean square error loss function is used to optimize the training process, which is represented by the formula: （8） (8) Substituting the DDIM diffusion formula gives （9） where the hyperparameters are set to constants. , to simplify the model, are set to constants.

6. The method of claim 1, wherein, To avoid numerical instability caused by the guiding gradient on different time steps, an adaptive guiding coefficient is introduced wherein is the adjusted gradient weight parameter.

Images