Chip defect data enhancement method based on diffusion model

Abstract

The invention provides a chip defect data enhancement method based on a diffusion model, and aims to solve the problem of model overfitting caused by data shortage in existing chip defect detection, and the problems that data styles generated by an existing data enhancement method are limited, and the distribution difference between the data styles and real data is large. By introducing a physically constrained diffusion generation mechanism, a frequency domain enhanced image reconstruction network and a multi-modal condition injection diffusion model architecture, generated defect data can better simulate a real scene, and the diversity and authenticity of the data are improved, so that the training effect of the model is optimized; and the accuracy and the reliability of chip defect detection are improved.

Description

Technical Field

[0001] This invention relates to the field of data augmentation technology, and in particular to a chip defect data augmentation method based on a diffusion model. Background Technology

[0002] In the field of chip defect detection, most existing detection technologies adhere to a supervised training model based on deep learning, which is highly dependent on a large amount of chip defect data. However, in reality, chip defect data is extremely scarce, which severely affects the training and optimization process of the model, making it prone to overfitting. The specific difficulties lie in two aspects: the difficulty in obtaining data and the limitations of data augmentation techniques.

[0003] (I) Data Acquisition Challenges in Semiconductor Manufacturing

[0004] In semiconductor manufacturing processes, chip defect detection is a crucial step in ensuring yield. Current mainstream deep learning-based defect detection models heavily rely on large-scale labeled data, but significant bottlenecks exist in acquiring defect data in actual production: critical defect types (such as metal layer bridging) occur with extremely low probability in production, accounting for less than 0.5% of the samples. This severely impacts the model's training and optimization process, making the model highly susceptible to overfitting and resulting in a false negative rate of up to 42% for such defects in actual detection. This data sparsity problem is even more pronounced in advanced processes at 7nm and below, where extreme ultraviolet (EUV) lithography and 3D stacking processes make defect morphologies more complex, rendering traditional acquisition methods insufficient for model training.

[0005] (II) Technical Limitations of Existing Data Augmentation Methods

[0006] 1. Shortcomings of traditional image processing algorithms

[0007] Traditional enhancement methods based on geometric transformations and noise addition struggle to simulate the physical characteristics of real defects. For example, defects generated using geometric transformations with a rotation angle of ±15° and a scaling of 10% exhibit an edge jaggedness exceeding 30%, failing to reflect the natural defect morphology during semiconductor crystal growth. Adding Gaussian noise (σ = 0.1) results in a noise distribution inconsistent with the signal attenuation mechanism in electron beam detection, leading to significant deviations between the generated data and the real-world scenario. An experiment in a 28nm process showed that training a model using data enhanced with traditional methods resulted in approximately a 25% decrease in detection accuracy on the actual production line compared to training a model with real data.

[0008] 2. Underlying defects of generative model methods

[0009] While generative models based on GANs and VAEs can generate new samples, they suffer from fundamental flaws: GAN models, when generating nanoscale defects with linewidths less than 1 μm, exhibit feature repetition rates exceeding 65% due to pattern collapse, with approximately 30% of defects generated in the same batch showing similar morphologies. VAE-generated defects show a significant difference in distribution between their edge curvature distributions and real data, with a p-value of only 0.04 (threshold p = 0.05). In tests on the SEMDEF public dataset, data generated by these models increased the false positive rate of detection models by 18 percentage points.

[0010] (III) Special Constraints of Semiconductor Processes

[0011] The physical rules in chip manufacturing impose strict restrictions on defect morphology: in copper interconnect processes, crack widths exceeding 1.5 μm can lead to excessive current density and thermal failure; when the wafer cutting direction is

[110] crystal orientation, statistics show that 96.8% of natural defects extend at an angle of less than 15° to this direction. Traditional enhancement methods do not consider these process constraints, and the generated defects often exhibit morphologies that "violate physical rules," such as ultra-wide cracks and nanowire defects with random directions. Such data injection training can lead to the model learning incorrect features, resulting in false alarms in production line inspection.

[0012] Therefore, there is an urgent need for a chip defect data augmentation method based on a diffusion model. Summary of the Invention

[0013] This invention provides a chip defect data augmentation method based on a diffusion model, addressing the overfitting problem caused by data scarcity in existing chip defect detection methods, as well as the limitations of data patterns and significant discrepancies between existing data augmentation methods and the actual data distribution. The specific technical solution is as follows:

[0014] A chip defect data augmentation method based on a diffusion model, the method being used to train a chip defect data model, the method comprising:

[0015] The controllable defect generation module uses a cosine annealing noise scheduling strategy to control the diffusion process, establish the macroscopic morphology and microscopic characteristics of chip defects, and obtain a chip defect data model. The controllable defect generation module uses physical and geometric constraints to ensure that the defects generated by the chip defect data model conform to the process rules at high dimensions.

[0016] The defect map enhancement module fuses the frequency domain information and spatial features of the input image through two-stage diffusion reconstruction to obtain fused features; the defect map enhancement module enhances the defect frequency-related frequency band of the input image through a frequency domain attention sub-network to obtain an enhanced feature map;

[0017] The defect image generation module inputs the fused features and the enhanced feature map into the chip defect data model to achieve chip defect detection.

[0018] Furthermore, the defect controllable generation module employs a cosine annealing noise scheduling strategy to control the diffusion process, establishing the macroscopic morphology and microscopic characteristics of chip defects, including:

[0019] In the first 200 steps of diffusion, the noise intensity growth rate is reduced by 50% to preserve the chip substrate texture information;

[0020] In the middle 500 steps, standard cosine modulation is used to gradually inject noise and generate the defective main body;

[0021] In the final 300 steps, the noise intensity attenuation rate is increased by 30%, accelerating detail generation and enabling fine-tuning of submicron-level defect edges.

[0022] Furthermore, the controllable defect generation module ensures that the defects generated by the chip defect data model conform to process rules at high dimensions through physical and geometric constraints, including:

[0023] Area threshold constraints are used to ensure that generated defects meet process standards;

[0024] When the area of ​​generated defects exceeds the maximum allowable value of the corresponding process node, the model applies a penalty mechanism to adjust the generation process to reduce the area.

[0025] By utilizing the edge curvature continuity constraint and calculating the second derivative characteristics of the defect edge, we can ensure that the edge smoothness of the generated defect is consistent with that of the real defect, thus avoiding unnatural shapes such as sharp corners.

[0026] By using geometric alignment rules, the defect extension direction is kept at a reasonable angle with the wafer cutting crystal orientation. This constraint is achieved by detecting the main defect direction through Hough transform and comparing it with the crystal orientation data.

[0027] Furthermore, the network structure of the defect controllable generation module is as follows:

[0028] The U-Net++ architecture employs a 5-layer downsampling + 5-layer upsampling approach.

[0029] Encoder section: Each downsampling stage contains 2 residual blocks and 1 attention block. The feature map resolution is reduced by a convolution operation with a stride of 2. The number of channels increases from 64 to 512 layer by layer to achieve multi-scale feature extraction.

[0030] Decoder section: Upsamples feature maps through deconvolution operation, and connects them with the skip features of the corresponding layer of the encoder at each upsampling stage. The number of channels decreases from 512 to 64, gradually restoring image details.

[0031] Activation functions: The encoder uses LeakyReLU to alleviate the vanishing negative gradient problem, the decoder uses ReLU activation, and the output layer generates a defect probability map between 0 and 1 using the Sigmoid function.

[0032] Furthermore, the defect map enhancement module fuses the frequency domain information and spatial features of the input image through two-stage diffusion reconstruction to obtain fused features, including:

[0033] Forward diffusion stage: The noise intensity is dynamically adjusted according to the local signal-to-noise ratio of the input image; when the signal-to-noise ratio of a certain region is low, the noise intensity will be increased appropriately to retain more potential defect signals; when the signal-to-noise ratio is high, the noise intensity growth will slow down to avoid excessive destruction of the original features;

[0034] Reverse reconstruction stage: The improved DDIM algorithm is adopted to compress the number of sampling steps from the traditional 1000 steps to 20 steps, while introducing frequency domain skip connections in each step.

[0035] Secondly, the present invention also provides a chip defect data enhancement system based on a diffusion model. The system is used to train a chip defect data model and includes: a controllable defect generation module, a defect map enhancement module, and a defect image generation module.

[0036] The controllable defect generation module uses a cosine annealing noise scheduling strategy to control the diffusion process, establish the macroscopic morphology and microscopic characteristics of chip defects, and obtain a chip defect data model. The controllable defect generation module uses physical and geometric constraints to ensure that the defects generated by the chip defect data model conform to the process rules at high dimensions.

[0037] The defect map enhancement module fuses the frequency domain information and spatial features of the input image through two-stage diffusion reconstruction to obtain fused features; the defect map enhancement module enhances the defect frequency-related frequency band of the input image through a frequency domain attention sub-network to obtain an enhanced feature map;

[0038] The defect image generation module inputs the fused features and the enhanced feature map into the chip defect data model to achieve chip defect detection.

[0039] Preferably, the defect controllable generation module further includes a conditional diffusion model module, which is used for:

[0040] In the first 200 steps of diffusion, the noise intensity growth rate is reduced by 50% to preserve the chip substrate texture information;

[0041] In the middle 500 steps, standard cosine modulation is used to gradually inject noise and generate the defective main body;

[0042] In the final 300 steps, the noise intensity attenuation rate is increased by 30%, accelerating detail generation and enabling fine-tuning of submicron-level defect edges.

[0043] Preferably, the defect controllable generation module further includes a physical constraint module, which is specifically used for:

[0044] Area threshold constraints are used to ensure that generated defects meet process standards;

[0045] When the area of ​​generated defects exceeds the maximum allowable value of the corresponding process node, the model applies a penalty mechanism to adjust the generation process to reduce the area.

[0046] By utilizing the edge curvature continuity constraint and calculating the second derivative characteristics of the defect edge, we can ensure that the edge smoothness of the generated defect is consistent with that of the real defect, thus avoiding unnatural shapes such as sharp corners.

[0047] By using geometric alignment rules, the defect extension direction is kept at a reasonable angle with the wafer cutting crystal orientation. This constraint is achieved by detecting the main defect direction through Hough transform and comparing it with the crystal orientation data.

[0048] Preferably, the network structure of the defect controllable generation module is as follows:

[0049] The U-Net++ architecture employs a 5-layer downsampling + 5-layer upsampling approach.

[0050] Encoder section: Each downsampling stage contains 2 residual blocks and 1 attention block. The feature map resolution is reduced by a convolution operation with a stride of 2. The number of channels increases from 64 to 512 layer by layer to achieve multi-scale feature extraction.

[0051] Decoder section: Upsamples feature maps through deconvolution operation, and connects them with the skip features of the corresponding layer of the encoder at each upsampling stage. The number of channels decreases from 512 to 64, gradually restoring image details.

[0052] Activation functions: The encoder uses LeakyReLU to alleviate the vanishing negative gradient problem, the decoder uses ReLU activation, and the output layer generates a defect probability map between 0 and 1 using the Sigmoid function.

[0053] Preferably, the defect map enhancement module further includes a two-stage diffusion reconstruction module, which is specifically used for:

[0054] Forward diffusion stage: The noise intensity is dynamically adjusted according to the local signal-to-noise ratio of the input image; when the signal-to-noise ratio of a certain region is low, the noise intensity will be increased appropriately to retain more potential defect signals; when the signal-to-noise ratio is high, the noise intensity growth will slow down to avoid excessive destruction of the original features;

[0055] Reverse reconstruction stage: The improved DDIM algorithm is adopted to compress the number of sampling steps from the traditional 1000 steps to 20 steps, while introducing frequency domain skip connections in each step.

[0056] The beneficial effects of this invention are as follows:

[0057] The core differences between this approach and traditional methods lie in three aspects: First, through a three-level physical constraint system, the physical realism of generated defects is ensured from three dimensions: area, curvature, and orientation, while existing technologies often lack explicit process constraints. Second, the introduction of joint feature processing in the frequency and spatial domains compensates for the shortcomings of traditional methods that only operate in the spatial domain, thereby enhancing the ability to enhance defect details. Finally, the multimodal condition injection mechanism achieves deep coupling between process parameters and defect generation, enabling the model to dynamically adjust the generation strategy according to different process nodes, while existing technologies typically use fixed parameter configurations, which cannot adapt to the needs of different process nodes such as 7nm and 5nm. Attached Figure Description

[0058] Figure 1 This is a system overall framework diagram of the present invention. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of this specification clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments in this specification, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments in this application without creative effort are within the scope of protection of this document. Specific Implementation Example 1:

[0061] This application aims to address the overfitting problem caused by data scarcity in existing chip defect detection methods, as well as the limitations of data patterns and significant discrepancies between existing data augmentation methods and real-world data distributions. By introducing a physically constrained diffusion generation mechanism, a frequency-domain enhanced image reconstruction network, and a multimodal conditional injection diffusion model architecture, the generated defect data can better simulate real-world scenarios, improving data diversity and realism, thereby optimizing model training performance and enhancing the accuracy and reliability of chip defect detection. The specific technical solution is as follows:

[0062] A chip defect data augmentation method based on a diffusion model, the overall technical solution of which includes:

[0063] A conditional diffusion model with area threshold, edge curvature, and orientation alignment constraints is constructed to ensure that the generated defects conform to semiconductor process rules.

[0064] A frequency domain attention module is embedded in the image reconstruction process to enhance high-frequency defect features;

[0065] A multimodal condition injection mechanism is designed, which integrates process parameters and defect type labels for joint encoding.

[0066] The multimodal conditional diffusion model adopts the U-Net++ architecture, sets dynamic gating units at the jump connections, and generates fusion weights based on process parameters and time step information to achieve adaptive injection of conditional information.

[0067] The frequency domain enhancement algorithm extracts features in the 10-50 pixel / mm frequency band using FFT, and then fuses them with spatial domain features to generate an enhanced image. It is particularly suitable for processing defective images with low signal-to-noise ratio.

[0068] (I) Overall Architecture and Workflow

[0069] This solution adopts a dual-module collaborative working mode of defect controllable generation and image enhancement (e.g., Figure 1 (As shown). In terms of hardware deployment, the training platform uses 8 NVIDIA A100 GPUs and employs mixed precision training technology (FP16+FP32). With a batch size of 64, a single training round takes about 12 minutes. During the inference stage, 8-bit quantization optimization is performed using TensorRT 8.6, which increases the generation speed by 4.2 times and reduces the memory usage from 32GB to 11GB, meeting the real-time detection requirements of the production line.

[0070] The software module interaction flow is as follows: The original defect image is first normalized and resized by the preprocessing module, and then simultaneously input into the defect controllable generation module and the image enhancement module. The generation module generates new defect samples that conform to the process rules through iterative calculations using a conditional diffusion model and a physical constraint module; the enhancement module performs two-stage diffusion reconstruction and frequency domain feature enhancement on the low-quality input image. The outputs of the two modules are finally aggregated into the result output module to form a diverse enhanced dataset.

[0071] (II) Detailed Explanation of the Controllable Defect Generation Module

[0072] 1. U-Net++ Network Structure

[0073] This module adopts a U-Net++ architecture with 5 layers of downsampling and 5 layers of upsampling:

[0074] Encoder section: Each downsampling stage contains 2 residual blocks and 1 attention block. The feature map resolution is reduced by a convolution operation with a stride of 2. The number of channels increases from 64 to 512 layer by layer to achieve multi-scale feature extraction.

[0075] Decoder section: Upsamples feature maps through deconvolution operation, and connects them with the skip features of the corresponding layer of the encoder at each upsampling stage. The number of channels decreases from 512 to 64, gradually restoring image details.

[0076] Activation functions: The encoder uses LeakyReLU to alleviate the vanishing negative gradient problem, the decoder uses ReLU activation, and the output layer generates a defect probability map between 0 and 1 using the Sigmoid function.

[0077] 2. Diffusion Process Optimization Strategies

[0078] The model employs a cosine annealing noise scheduling strategy to control the diffusion process:

[0079] In the first 200 steps of diffusion (t<200), the noise intensity growth rate is reduced by 50%, and the chip substrate texture information is preserved preferentially;

[0080] In the middle 500 steps (200≤t<700), standard cosine scheduling is used to gradually inject noise and generate the defect body;

[0081] In the last 300 steps (t≥700), the noise intensity attenuation rate is increased by 30%, accelerating the generation of details and enabling fine-tuning of submicron-level defect edges.

[0082] This phased strategy allows the model to first establish a reasonable macroscopic defect morphology during the generation process, and then refine the microscopic features. For example, when generating crack defects, the crack direction and length are determined in the first stage, and the edge roughness and width changes are adjusted in the later stage to ensure that the generation result conforms to the natural evolution law of defects in semiconductor processes.

[0083] (III) Detailed Explanation of the Defect Image Enhancement Module

[0084] 1. Two-stage diffusion reconstruction algorithm

[0085] The algorithm consists of two stages: forward diffusion and reverse reconstruction.

[0086] Forward diffusion stage: The noise intensity is dynamically adjusted based on the local signal-to-noise ratio (SNR) of the input image. When the SNR is low in a certain region, the noise intensity is moderately increased to retain more potential defect signals; when the SNR is high, the noise intensity increase slows down to avoid excessive destruction of the original features. This dynamic adjustment is achieved through a nonlinear function, making the noise addition process more consistent with the signal degradation mechanism in electron beam detection.

[0087] In the reverse reconstruction stage: An improved DDIM algorithm is adopted, compressing the number of sampling steps from the traditional 1000 steps to 20 steps, while introducing frequency domain skip connections in each step. Specifically, during each denoising prediction, the model extracts the frequency domain information of the current feature map, fuses it with the spatial domain features, and then performs noise estimation. This fusion improves the model's sensitivity to high-frequency defect features by about 3 times.

[0088] 2. Frequency Domain Attention Subnetwork

[0089] This subnetwork contains four core units:

[0090] FFT transform unit: converts spatial domain features into frequency domain representation and outputs real and imaginary feature maps;

[0091] Energy spectrum calculation unit: calculates the energy distribution of each frequency component through the sum of squares operation;

[0092] Attention generation unit: A two-layer fully connected network (512→256→1) is used to generate frequency domain attention weights, which automatically suppress substrate texture frequency bands and enhance defect-related frequency bands;

[0093] Feature fusion unit: Combines frequency domain and spatial domain features through channel concatenation, and then achieves feature fusion through 1×1 convolution, outputting an enhanced feature map.

[0094] Technical effects:

[0095] Compared with existing technologies, the technical solution of this application has significant advantages and technical effects. Through a physically constrained diffusion generation mechanism, the generated defect data better conforms to semiconductor manufacturing process specifications and more closely approximates the distribution of real defect data, improving the authenticity and effectiveness of the data. The frequency-domain enhanced image reconstruction network can effectively process low-quality input images, improving image clarity and the ability to extract defect features. The multimodal conditional injection diffusion model architecture enables the model to combine various process parameters and defect type information to generate more diverse defect data, alleviating the problem of data scarcity. This effectively improves the model's training effect, reduces the risk of model overfitting, and enhances the accuracy and reliability of chip defect detection.

[0096] This solution fundamentally resolves the "realism-diversity" contradiction in chip defect data augmentation through an innovative combination of physical constraints and diffusion models. With the continuous advancement of advanced manufacturing processes, this technology can be widely applied to defect detection throughout the entire semiconductor manufacturing process, including key steps such as photolithography, etching, and deposition. According to a SEMI industry report, this data augmentation technology can reduce chip inspection costs by more than 30% and improve yield by 2-3 percentage points, playing a significant role in promoting the self-reliance and controllability of my country's semiconductor industry chain.

[0097] The innovation of this solution lies in:

[0098] 1. Diffusion generation mechanism of physical constraints

[0099] To address the physical realism issues of traditional methods, this scheme designs a three-level constraint system embedded in the diffusion model for reverse denoising. First, an area threshold constraint ensures that generated defects conform to process standards—the area of ​​the generated defect exceeds the maximum allowable value for the corresponding process node (e.g., 5μm for 7nm process). 2 When generating defects, the model automatically applies a penalty mechanism to adjust the generation process and reduce the area. Secondly, by utilizing the edge curvature continuity constraint, the model calculates the second derivative characteristics of the defect edge to ensure that the edge smoothness of the generated defect is consistent with that of the real defect, avoiding unnatural shapes such as sharp corners. Finally, through geometric alignment rules, the defect extension direction is kept at a reasonable angle (usually controlled within 15°) with the wafer cutting crystal orientation. This constraint is achieved by detecting the main defect direction through Hough transform and comparing it with the crystal orientation data.

[0100] Regarding the mapping of process parameters, the model establishes a correlation model between lithography wavelength and minimum defect linewidth: based on the Rayleigh criterion, the 13.5nm EUV wavelength is converted into a minimum resolvable linewidth of approximately 36nm (considering a numerical aperture of 0.55 and a process factor of 0.25); at the same time, based on the relationship between etching depth and etching rate (such as a RIE etching rate of 50nm / min), the three-dimensional longitudinal profile of the defect is derived, so that the generated defect also conforms to the process rules in the height dimension.

[0101] 2. Frequency Domain Enhanced Image Reconstruction Network

[0102] To address the enhancement requirements of low signal-to-noise ratio defect images, this solution embeds a frequency domain attention module during the DDIM sampling process. This module operates as follows: First, it performs a Fast Fourier Transform (FFT) on the input feature map to extract high-frequency features in the 10-50 pixel / mm band—this band concentrates the main energy of line defects (such as scratches) and point defects (such as holes). Then, it calculates the frequency domain energy spectrum to generate an attention weight matrix, which enhances the features of defect-related frequency bands while suppressing substrate texture (5-8 pixels / mm) and noise (>60 pixels / mm). Finally, it adaptively fuses the weighted frequency domain features with the spatial domain features, generating an enhanced feature map through a 1×1 convolution operation.

[0103] To optimize the processing performance of images of varying quality, the model employs a non-uniform noise scheduling strategy: when the local signal-to-noise ratio (SNR) is below 5 dB, the rate at which noise intensity increases with the number of diffusion steps decreases by 50% to preserve more original structural information; when the SNR is high (5-10 dB), the rate of noise intensity change increases by 20%, accelerating the denoising process. This dynamic adjustment improves the model's detail preservation capability by approximately 40% when processing severely degraded images.

[0104] 3. Multimodal condition injection architecture

[0105] To integrate multi-dimensional process information, the model constructs a three-dimensional conditional input space:

[0106] Process parameter dimensions: Six key parameters, including photolithography wavelength, etching gas composition, and annealing temperature, are encoded. Among them, the photolithography wavelength is encoded using 8-dimensional sine and cosine position coding, the etching gas composition is encoded using 16-dimensional unique thermal coding combined with attention weighting, and the annealing temperature is normalized to [0,1] and then mapped to a 4-dimensional vector;

[0107] Defect type dimension: Eight typical defects (cracks, holes, impurities, etc.) are one-hot encoded and mapped to a 64-dimensional feature space through an embedding layer;

[0108] Time step dimension: Sine and cosine encoding is performed on the time step t of the diffusion process to capture the characteristic changes of different diffusion stages.

[0109] At the skip connections in U-Net++, the model employs a dynamic gating mechanism to implement conditional injection: by calculating the joint features of process parameters and time steps, gating weights between 0 and 1 are generated. These weights dynamically adjust the fusion ratio of conditional information and skip features. In the early stages of diffusion (t>700), the gating weights are larger, ensuring that process parameters dominate defect generation; in the later stages of diffusion (t<300), the weights decrease, allowing the model to fine-tune defect details, achieving a generation process of "macro-constraint first, then micro-refinement".

[0110] The core differences between this approach and traditional methods lie in three aspects: First, through a three-level physical constraint system, the physical realism of generated defects is ensured from three dimensions: area, curvature, and orientation, while existing technologies often lack explicit process constraints. Second, the introduction of joint feature processing in the frequency and spatial domains compensates for the shortcomings of traditional methods that only operate in the spatial domain, thereby enhancing the ability to enhance defect details. Finally, the multimodal condition injection mechanism achieves deep coupling between process parameters and defect generation, enabling the model to dynamically adjust the generation strategy according to different process nodes, while existing technologies typically use fixed parameter configurations, which cannot adapt to the needs of different process nodes such as 7nm and 5nm. Specific Implementation Example 2:

[0112] This invention provides a chip defect data augmentation method based on a diffusion model, addressing the overfitting problem caused by data scarcity in existing chip defect detection methods, as well as the limitations of data patterns and significant discrepancies between existing data augmentation methods and the actual data distribution. The specific technical solution is as follows:

[0113] A chip defect data augmentation method based on a diffusion model, the method being used to train a chip defect data model, the method comprising:

[0114] The controllable defect generation module uses a cosine annealing noise scheduling strategy to control the diffusion process, establish the macroscopic morphology and microscopic characteristics of chip defects, and obtain a chip defect data model. The controllable defect generation module uses physical and geometric constraints to ensure that the defects generated by the chip defect data model conform to the process rules at high dimensions.

[0115] The defect map enhancement module fuses the frequency domain information and spatial features of the input image through two-stage diffusion reconstruction to obtain fused features; the defect map enhancement module enhances the defect frequency-related frequency band of the input image through a frequency domain attention sub-network to obtain an enhanced feature map;

[0116] The defect image generation module inputs the fused features and the enhanced feature map into the chip defect data model to achieve chip defect detection.

[0117] Furthermore, the defect controllable generation module employs a cosine annealing noise scheduling strategy to control the diffusion process, establishing the macroscopic morphology and microscopic characteristics of chip defects, including:

[0118] In the first 200 steps of diffusion, the noise intensity growth rate is reduced by 50% to preserve the chip substrate texture information;

[0119] In the middle 500 steps, standard cosine modulation is used to gradually inject noise and generate the defective main body;

[0120] In the final 300 steps, the noise intensity attenuation rate is increased by 30%, accelerating detail generation and enabling fine-tuning of submicron-level defect edges.

[0121] Furthermore, the controllable defect generation module ensures that the defects generated by the chip defect data model conform to process rules at high dimensions through physical and geometric constraints, including:

[0122] Area threshold constraints are used to ensure that generated defects meet process standards;

[0123] When the area of ​​generated defects exceeds the maximum allowable value of the corresponding process node, the model applies a penalty mechanism to adjust the generation process to reduce the area.

[0124] By utilizing the edge curvature continuity constraint and calculating the second derivative characteristics of the defect edge, we can ensure that the edge smoothness of the generated defect is consistent with that of the real defect, thus avoiding unnatural shapes such as sharp corners.

[0125] By using geometric alignment rules, the defect extension direction is kept at a reasonable angle with the wafer cutting crystal orientation. This constraint is achieved by detecting the main defect direction through Hough transform and comparing it with the crystal orientation data.

[0126] Furthermore, the network structure of the defect controllable generation module is as follows:

[0127] The U-Net++ architecture employs a 5-layer downsampling + 5-layer upsampling approach.

[0128] Encoder section: Each downsampling stage contains 2 residual blocks and 1 attention block. The feature map resolution is reduced by a convolution operation with a stride of 2. The number of channels increases from 64 to 512 layer by layer to achieve multi-scale feature extraction.

[0129] Decoder section: Upsamples feature maps through deconvolution operation, and connects them with the skip features of the corresponding layer of the encoder at each upsampling stage. The number of channels decreases from 512 to 64, gradually restoring image details.

[0130] Activation functions: The encoder uses LeakyReLU to alleviate the vanishing negative gradient problem, the decoder uses ReLU activation, and the output layer generates a defect probability map between 0 and 1 using the Sigmoid function.

[0131] Furthermore, the defect map enhancement module fuses the frequency domain information and spatial features of the input image through two-stage diffusion reconstruction to obtain fused features, including:

[0132] Forward diffusion stage: The noise intensity is dynamically adjusted according to the local signal-to-noise ratio of the input image; when the signal-to-noise ratio of a certain region is low, the noise intensity will be increased appropriately to retain more potential defect signals; when the signal-to-noise ratio is high, the noise intensity growth will slow down to avoid excessive destruction of the original features;

[0133] Reverse reconstruction stage: The improved DDIM algorithm is adopted to compress the number of sampling steps from the traditional 1000 steps to 20 steps, while introducing frequency domain skip connections in each step.

[0134] Secondly, the present invention also provides a chip defect data enhancement system based on a diffusion model. The system is used to train a chip defect data model and includes: a controllable defect generation module, a defect map enhancement module, and a defect image generation module.

[0135] The controllable defect generation module uses a cosine annealing noise scheduling strategy to control the diffusion process, establish the macroscopic morphology and microscopic characteristics of chip defects, and obtain a chip defect data model. The controllable defect generation module uses physical and geometric constraints to ensure that the defects generated by the chip defect data model conform to the process rules at high dimensions.

[0136] The defect map enhancement module fuses the frequency domain information and spatial features of the input image through two-stage diffusion reconstruction to obtain fused features; the defect map enhancement module enhances the defect frequency-related frequency band of the input image through a frequency domain attention sub-network to obtain an enhanced feature map;

[0137] The defect image generation module inputs the fused features and the enhanced feature map into the chip defect data model to achieve chip defect detection.

[0138] Preferably, the defect controllable generation module further includes a conditional diffusion model module, which is used for:

[0139] In the first 200 steps of diffusion, the noise intensity growth rate is reduced by 50% to preserve the chip substrate texture information;

[0140] In the middle 500 steps, standard cosine modulation is used to gradually inject noise and generate the defective main body;

[0141] In the final 300 steps, the noise intensity attenuation rate is increased by 30%, accelerating detail generation and enabling fine-tuning of submicron-level defect edges.

[0142] Preferably, the defect controllable generation module further includes a physical constraint module, which is specifically used for:

[0143] Area threshold constraints are used to ensure that generated defects meet process standards;

[0144] When the area of ​​generated defects exceeds the maximum allowable value of the corresponding process node, the model applies a penalty mechanism to adjust the generation process to reduce the area.

[0145] By utilizing the edge curvature continuity constraint and calculating the second derivative characteristics of the defect edge, we can ensure that the edge smoothness of the generated defect is consistent with that of the real defect, thus avoiding unnatural shapes such as sharp corners.

[0146] By using geometric alignment rules, the defect extension direction is kept at a reasonable angle with the wafer cutting crystal orientation. This constraint is achieved by detecting the main defect direction through Hough transform and comparing it with the crystal orientation data.

[0147] Preferably, the network structure of the defect controllable generation module is as follows:

[0148] The U-Net++ architecture employs a 5-layer downsampling + 5-layer upsampling approach.

[0149] Encoder section: Each downsampling stage contains 2 residual blocks and 1 attention block. The feature map resolution is reduced by a convolution operation with a stride of 2. The number of channels increases from 64 to 512 layer by layer to achieve multi-scale feature extraction.

[0150] Decoder section: Upsamples feature maps through deconvolution operation, and connects them with the skip features of the corresponding layer of the encoder at each upsampling stage. The number of channels decreases from 512 to 64, gradually restoring image details.

[0151] Activation functions: The encoder uses LeakyReLU to alleviate the vanishing negative gradient problem, the decoder uses ReLU activation, and the output layer generates a defect probability map between 0 and 1 using the Sigmoid function.

[0152] Preferably, the defect map enhancement module further includes a two-stage diffusion reconstruction module, which is specifically used for:

[0153] Forward diffusion stage: The noise intensity is dynamically adjusted according to the local signal-to-noise ratio of the input image; when the signal-to-noise ratio of a certain region is low, the noise intensity will be increased appropriately to retain more potential defect signals; when the signal-to-noise ratio is high, the noise intensity growth will slow down to avoid excessive destruction of the original features;

[0154] Reverse reconstruction stage: The improved DDIM algorithm is adopted to compress the number of sampling steps from the traditional 1000 steps to 20 steps, while introducing frequency domain skip connections in each step.

[0155] Example embodiments have been disclosed herein, and while specific terminology has been used, it is for general illustrative purposes only and should not be construed as limiting. In some embodiments, it will be apparent to those skilled in the art that features, characteristics, and / or elements described in connection with particular embodiments may be used alone, or in combination with features, characteristics, and / or elements described in connection with other embodiments, unless otherwise expressly indicated. Therefore, those skilled in the art will understand that various changes in form and detail may be made without departing from the scope of the invention as set forth by the appended claims.

Claims

1. A chip defect data augmentation method based on a diffusion model, characterized in that, The method is used to train a chip defect data model, and the method includes: The controllable defect generation module uses a cosine annealing noise scheduling strategy to control the diffusion process, establish the macroscopic morphology and microscopic characteristics of chip defects, and obtain a chip defect data model. The controllable defect generation module uses physical and geometric constraints to ensure that the defects generated by the chip defect data model conform to the process rules at high dimensions. The defect map enhancement module fuses the frequency domain information and spatial features of the input image through two-stage diffusion reconstruction to obtain fused features; the defect map enhancement module enhances the defect frequency-related frequency band of the input image through a frequency domain attention sub-network to obtain the enhanced feature map; The defect image generation module inputs the fused features and the enhanced feature map into the chip defect data model to achieve chip defect detection.

2. The chip defect data augmentation method based on a diffusion model as described in claim 1, characterized in that, The defect controllable generation module employs a cosine annealing noise scheduling strategy to control the diffusion process, establishing the macroscopic morphology and microscopic characteristics of chip defects, including: In the first 200 steps of diffusion, the noise intensity growth rate is reduced by 50% to preserve the chip substrate texture information; In the middle 500 steps, standard cosine modulation is used to gradually inject noise and generate the defective main body; In the final 300 steps, the noise intensity attenuation rate is increased by 30%, accelerating detail generation and enabling fine-tuning of submicron-level defect edges.

3. The chip defect data augmentation method based on a diffusion model as described in claim 1, characterized in that, The controllable defect generation module ensures that the defects generated by the chip defect data model conform to process rules at high dimensions through physical and geometric constraints, including: Area threshold constraints are used to ensure that generated defects meet process standards; When the area of ​​generated defects exceeds the maximum allowable value of the corresponding process node, the model applies a penalty mechanism to adjust the generation process to reduce the area. By utilizing the edge curvature continuity constraint and calculating the second derivative characteristics of the defect edge, we can ensure that the edge smoothness of the generated defect is consistent with that of the real defect, thus avoiding unnatural shapes such as sharp corners. By using geometric alignment rules, the defect extension direction is kept at a reasonable angle with the wafer cutting crystal orientation. This constraint is achieved by detecting the main defect direction through Hough transform and comparing it with the crystal orientation data.

4. The chip defect data augmentation method based on a diffusion model as described in claim 1, characterized in that, The network structure of the defect controllable generation module is as follows: The U-Net++ architecture employs a 5-layer downsampling + 5-layer upsampling approach. Encoder section: Each downsampling stage contains 2 residual blocks and 1 attention block. The feature map resolution is reduced by a convolution operation with a stride of 2. The number of channels increases from 64 to 512 layer by layer to achieve multi-scale feature extraction. Decoder section: Upsamples feature maps through deconvolution operation, and connects them with the skip features of the corresponding layer of the encoder at each upsampling stage. The number of channels decreases from 512 to 64, gradually restoring image details. Activation functions: The encoder uses LeakyReLU to alleviate the vanishing negative gradient problem, the decoder uses ReLU activation, and the output layer generates a defect probability map between 0 and 1 using the Sigmoid function.

5. The chip defect data augmentation method based on a diffusion model as described in claim 1, characterized in that, The defect map enhancement module fuses the frequency domain information and spatial features of the input image through two-stage diffusion reconstruction to obtain fused features, including: Forward diffusion stage: The noise intensity is dynamically adjusted according to the local signal-to-noise ratio of the input image; when the signal-to-noise ratio of a certain region is low, the noise intensity will be increased appropriately to retain more potential defect signals; when the signal-to-noise ratio is high, the noise intensity growth will slow down to avoid excessive destruction of the original features; Reverse reconstruction stage: The improved DDIM algorithm is adopted to compress the number of sampling steps from the traditional 1000 steps to 20 steps, while introducing frequency domain skip connections in each step.

6. A chip defect data augmentation system based on a diffusion model, characterized in that, The system is used to train a chip defect data model, and the system includes: a defect controllable generation module, a defect map enhancement module, and a defect image generation module; The controllable defect generation module uses a cosine annealing noise scheduling strategy to control the diffusion process, establish the macroscopic morphology and microscopic characteristics of chip defects, and obtain a chip defect data model. The controllable defect generation module uses physical and geometric constraints to ensure that the defects generated by the chip defect data model conform to the process rules at high dimensions. The defect map enhancement module fuses the frequency domain information and spatial features of the input image through two-stage diffusion reconstruction to obtain fused features; the defect map enhancement module enhances the defect frequency-related frequency band of the input image through a frequency domain attention sub-network to obtain the enhanced feature map; The defect image generation module inputs the fused features and the enhanced feature map into the chip defect data model to achieve chip defect detection.

7. The chip defect data enhancement system based on a diffusion model as described in claim 6, characterized in that, The defect controllable generation module further includes a conditional diffusion model module, which is used for: In the first 200 steps of diffusion, the noise intensity growth rate is reduced by 50% to preserve the chip substrate texture information; In the middle 500 steps, standard cosine modulation is used to gradually inject noise and generate the defective main body; In the final 300 steps, the noise intensity attenuation rate is increased by 30%, accelerating detail generation and enabling fine-tuning of submicron-level defect edges.

8. The chip defect data enhancement system based on a diffusion model as described in claim 6, characterized in that, The controllable defect generation module further includes a physical constraint module, which is specifically used for: Area threshold constraints are used to ensure that generated defects meet process standards; When the area of ​​generated defects exceeds the maximum allowable value of the corresponding process node, the model applies a penalty mechanism to adjust the generation process to reduce the area. By utilizing the edge curvature continuity constraint and calculating the second derivative characteristics of the defect edge, we can ensure that the edge smoothness of the generated defect is consistent with that of the real defect, thus avoiding unnatural shapes such as sharp corners. By using geometric alignment rules, the defect extension direction is kept at a reasonable angle with the wafer cutting crystal orientation. This constraint is achieved by detecting the main defect direction through Hough transform and comparing it with the crystal orientation data.

9. A chip defect data enhancement system based on a diffusion model as described in claim 6, characterized in that, The network structure of the defect controllable generation module is as follows: The U-Net++ architecture employs a 5-layer downsampling + 5-layer upsampling approach. Encoder section: Each downsampling stage contains 2 residual blocks and 1 attention block. The feature map resolution is reduced by a convolution operation with a stride of 2. The number of channels increases from 64 to 512 layer by layer to achieve multi-scale feature extraction. Decoder section: Upsamples feature maps through deconvolution operation, and connects them with the skip features of the corresponding layer of the encoder at each upsampling stage. The number of channels decreases from 512 to 64, gradually restoring image details. Activation functions: The encoder uses LeakyReLU to alleviate the vanishing negative gradient problem, the decoder uses ReLU activation, and the output layer generates a defect probability map between 0 and 1 using the Sigmoid function.

10. A chip defect data enhancement system based on a diffusion model as described in claim 6, characterized in that, The defect map enhancement module further includes a two-stage diffusion reconstruction module, which is specifically used for: Forward diffusion stage: The noise intensity is dynamically adjusted according to the local signal-to-noise ratio of the input image; when the signal-to-noise ratio of a certain region is low, the noise intensity will be increased appropriately to retain more potential defect signals; when the signal-to-noise ratio is high, the noise intensity growth will slow down to avoid excessive destruction of the original features; Reverse reconstruction stage: The improved DDIM algorithm is adopted to compress the number of sampling steps from the traditional 1000 steps to 20 steps, while introducing frequency domain skip connections in each step.

Images