A wafer defect dynamic video generation method and device

Abstract

This disclosure provides a method and apparatus for generating dynamic videos of wafer defects, relating to the field of video generation. The method includes: acquiring a natural language description input by a user; inputting the natural language description into a semantically driven defect dynamic response synthesis model to generate a two-dimensional scattering intensity dynamic sequence, which is used to simulate the scattering response captured temporally by a dark-field camera; generating a wafer background image based on the natural language description; and inputting the two-dimensional scattering intensity dynamic sequence, the wafer background image, and the natural language description into a scattering-guided visual rendering model, generating a dynamic video of wafer defects through multimodal feature fusion and decoding. Applying this method, the two major modules of textual semantics and optical scattering physical response are separated, preserving the ability of natural language to freely customize defects while introducing optical priors through independent scattering modeling. The generated dynamic video of wafer defects can be directly used for training dark-field detection equipment.

Description

Technical Field

[0001] This disclosure relates to the field of video generation, and more particularly to a method and apparatus for generating dynamic videos of wafer defects. Background Technology

[0002] Current imageless wafer defect detection systems primarily employ three data generation or modeling methods: Physical simulation-driven, which uses electromagnetic field or optical scattering models (such as Finite-Difference Time-Domain (FDTD) and Rigorous Coupled-Wave Analysis (RCWA)) to simulate the dynamic response of specific defects under dark illumination. This method offers high physical accuracy but incurs significant computational overhead, struggles to cover open-domain defect types, and cannot be directly controlled by natural language. Real-world data playback, based on historical wafer scanning videos, involves playback or interpolation. This method offers high realism but lacks generalization ability for novel or rare defects and cannot generate defect dynamics with specific semantic descriptions on demand. End-to-end generation models directly generate wafer video sequences based on text. This method offers high flexibility, but due to the lack of explicit modeling of the physical scattering mechanism of the wafer surface, it easily generates artifacts or non-physical dynamics that do not conform to optical laws, affecting the reliability of downstream detection model training. Summary of the Invention

[0003] This disclosure provides a method, apparatus, electronic device, and storage medium for generating dynamic videos of wafer defects, in order to at least solve the above-mentioned technical problems existing in the prior art.

[0004] According to a first aspect of this disclosure, a method for generating a dynamic video of wafer defects is provided. The method includes: acquiring a natural language description input by a user, the natural language description containing semantic information about wafer defects; inputting the natural language description into a semantically driven defect dynamic response synthesis model to generate a two-dimensional scattering intensity dynamic sequence, the two-dimensional scattering intensity dynamic sequence being used to simulate the scattering response captured temporally by a dark-field camera; generating a wafer background image based on the natural language description; and inputting the two-dimensional scattering intensity dynamic sequence, the wafer background image, and the natural language description together into a scattering-guided visual rendering model, and generating a dynamic video of wafer defects through multimodal feature fusion and decoding; wherein... The semantically driven defect dynamic response synthesis model includes: a temporal scattering manifold encoder, used to compress an externally acquired two-dimensional scattering intensity dynamic sequence into a low-dimensional continuous latent manifold to output a low-dimensional latent sequence during the training phase, and to reconstruct the latent representation generated by text into a two-dimensional scattering intensity dynamic sequence during the inference phase; a defect synthesis learning model, used to perform a diffusion generation process on the low-dimensional continuous latent manifold constructed by the temporal scattering manifold encoder, generating a corresponding low-dimensional latent sequence based on the natural language description, and the generated low-dimensional latent sequence can be reconstructed into the two-dimensional scattering intensity dynamic sequence by the temporal scattering manifold encoder; and a semantic-scattering dual-projection alignment framework, used to enhance the semantic consistency between the natural language description and the generated scattering dynamics during the training phase.

[0005] In one embodiment, the temporal scattering manifold encoder compresses an externally acquired two-dimensional scattering intensity dynamic sequence into a low-dimensional continuous latent manifold to output a low-dimensional latent sequence, including: performing a one-dimensional temporal convolution on the input externally acquired two-dimensional scattering intensity dynamic sequence to extract inter-frame dynamic evolution features; compressing the spatial dimension through multi-layer convolution downsampling to obtain bottleneck layer features; outputting the mean and variance corresponding to the bottleneck layer features through variational coding branches; sampling the mean and variance using reparameterization techniques to obtain a low-dimensional latent sequence; and constraining the distribution of the low-dimensional latent sequence through a KL divergence regularization term to approximate a standard normal distribution to ensure the continuity of the latent space.

[0006] In one embodiment, the temporal scattering manifold encoder further includes a signal-to-noise ratio (SNR)-aware weighting mechanism, specifically comprising: calculating local SNR estimates for each pixel in the input two-dimensional scattering intensity dynamic sequence within its temporal and spatial neighborhoods, wherein the local SNR estimate is the ratio of the mean to the standard deviation of the pixel values ​​within the neighborhood window; comparing the local SNR estimate with a preset threshold to determine low SNR regions; and applying adaptive suppression to the low SNR regions during model training, wherein the adaptive suppression is achieved by reducing the weight of the low SNR regions in the reconstruction loss function.

[0007] In one embodiment, the defect synthesis learning model generates a corresponding low-dimensional latent sequence based on the natural language description, including: dividing the noisy low-dimensional latent sequence into several three-dimensional blocks according to the spatiotemporal dimension; flattening each three-dimensional block into a vector and mapping it to the embedding dimension of the Transformer through linear projection; fusing spatial position embedding, temporal rotation position encoding, and position offset based on text semantic mapping on the embedding dimension, and performing weighted suppression of the background region through a dynamic masking mechanism; performing feature transformation using a scattering chain perceptual attention mechanism in each layer of the multi-layer Transformer; and outputting a denoised low-dimensional latent sequence, wherein the denoised low-dimensional latent sequence and the low-dimensional latent sequence output by the temporal scattering manifold encoder are located in the same latent manifold space.

[0008] In one embodiment, the scattering chain perception attention mechanism includes: a local chain perception subprocess: constructing a physical prior map to model the propagation constraints of defect scattering in the spatial neighborhood, injecting a physical bias based on the diffraction feature length into the local window attention, wherein the physical bias attenuates as the spatial distance between pixels increases, to simulate the cascading propagation law of scattering energy at the defect edge; a global spatiotemporal coordination subprocess: concatenating the block sequences of all frames and adding global temporal position encoding, using sparse global attention to capture long-range dynamic patterns, and injecting Fourier features to explicitly capture periodic flickering patterns; and a gated fusion subprocess: dynamically weighting and summing the output features of the local chain perception subprocess and the global spatiotemporal coordination subprocess through a learnable gating network.

[0009] In one embodiment, generating a corresponding low-dimensional latent sequence based on the natural language description includes: extracting textual semantic features of the natural language description using a text encoder of a contrastive language-image pre-trained model, which are used as keys and values ​​in cross-attention calculation; using the scattering latent sequence in the defect synthesis learning model as a query; and in each layer of cross-attention calculation in the defect synthesis learning model, dynamically weighting the different spatiotemporal positions of the scattering latent sequence using the textual semantic features, so that the generation process focuses on the defect type, size, and dynamic behavior of the text description.

[0010] In one embodiment, the semantic-scattering dual-projection alignment framework enhances the semantic consistency between the natural language description and the generated scattering dynamics during the training phase, including: extracting features using independent text encoders and scattering dynamics encoders respectively; narrowing the embedding distance of matching text-scattering pairs and widening the embedding distance of non-matching text-scattering pairs through contrastive learning; extracting the latent representation of the intermediate layer of the defect synthesis learning model; mapping it to the semantic space of the pre-trained scattering encoder through a lightweight projection network; calculating the alignment loss between it and the corresponding text or scattering reference embedding; and optimizing the network parameters of the temporal scattering manifold encoder and the defect synthesis learning model based on the total loss function.

[0011] In one embodiment, the scattering-guided visual rendering model employs a spatiotemporal separation deep learning architecture, which includes alternately stacked spatial attention blocks and temporal attention blocks: the spatial attention block uses the latent features of the scattering sequence as the query and the wafer background image features as the key and value to calculate spatial cross-attention, used to process wafer texture details in a single frame; the temporal attention block calculates self-attention along the time axis and performs cross-attention verification using the text features described in the natural language as the key and value, used to model inter-frame motion coherence.

[0012] In one possible implementation, the scattering-guided visual rendering model includes multimodal feature fusion, which involves: mapping the wafer background image, each frame of the two-dimensional scattering intensity dynamic sequence, and the text features of the natural language description to a unified latent spatial dimension via an encoder; using the latent representation of the two-dimensional scattering intensity dynamic sequence as the query backbone, providing spatial texture keys and values ​​using the latent features of the wafer background image, and providing semantic guidance keys and values ​​using the text features; performing dynamic weighted interaction through a multimodal cross-attention mechanism, and then integrating the fused features with layer normalization and a feedforward network before inputting them into the decoder to reconstruct the dynamic video frame by frame.

[0013] According to a second aspect of this disclosure, an apparatus for generating dynamic video of wafer defects is provided. The apparatus includes: an acquisition module for acquiring a natural language description input by a user, the natural language description containing semantic information of wafer defects; a first generation module for inputting the natural language description into a semantically driven defect dynamic response synthesis model to generate a two-dimensional scattering intensity dynamic sequence, the two-dimensional scattering intensity dynamic sequence being used to simulate the scattering response captured temporally by a dark-field camera; a second generation module for generating a wafer background image based on the natural language description; and a third generation module for inputting the two-dimensional scattering intensity dynamic sequence, the wafer background image, and the natural language description into a scattering-guided visual rendering model, and generating a dynamic video of wafer defects through multimodal feature fusion and decoding. The video, wherein the first generation module includes: a temporal scattering manifold encoder module, used to compress an externally acquired two-dimensional scattering intensity dynamic sequence into a low-dimensional continuous latent manifold to output a low-dimensional latent sequence during the training phase, and to reconstruct the latent representation generated by text into a two-dimensional scattering intensity dynamic sequence during the inference phase; a defect synthesis learning model module, used to perform a diffusion generation process on the low-dimensional continuous latent manifold constructed by the temporal scattering manifold encoder, and generate a corresponding low-dimensional latent sequence according to the natural language description, wherein the generated low-dimensional latent sequence can be reconstructed into the two-dimensional scattering intensity dynamic sequence by the temporal scattering manifold encoder; and a semantic-scattering dual-projection alignment framework module, used to strengthen the semantic consistency between the natural language description and the generated scattering dynamics during the training phase.

[0014] According to a third aspect of this disclosure, an electronic device is provided, comprising: At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the methods of this disclosure.

[0015] According to a fourth aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions for causing a computer to perform the methods described in this disclosure.

[0016] This disclosure discloses a method, apparatus, electronic device, and storage medium for generating dynamic video of wafer defects. First, it acquires a natural language description input by a user, which contains semantic information about the wafer defects. Then, it inputs the natural language description into a semantically driven defect dynamic response synthesis model to generate a two-dimensional scattering intensity dynamic sequence. This sequence simulates the scattering response captured temporally by a dark-field camera, and a wafer background image is generated based on the natural language description. Finally, the two-dimensional scattering intensity dynamic sequence, the wafer background image, and the natural language description are input into a scattering-guided visual rendering model. Through multimodal feature fusion and decoding, a dynamic video of wafer defects is generated. This method separates the two main modules of text semantics and optical scattering physical response, preserving the ability of natural language to freely customize defects while introducing optical priors through independent scattering modeling. This resolves the contradiction in existing technologies where "flexibility leads to distortion, and distortion leads to inefficiency." Based on the scattering chain-aware attention mechanism, the local cascade and global periodicity of scattering are modeled. The generated wafer defect dynamic video can suppress non-physical flicker noise and tearing artifacts. Introducing the original text ensures that the defect morphology, scattering dynamics, and user commands in the final generated dynamic video are in a closed loop. The generated wafer defect dynamic video can be directly used for training dark field detection equipment.

[0017] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0018] The above and other objects, features, and advantages of this disclosure will become readily apparent from the following detailed description of exemplary embodiments, taken in conjunction with the accompanying drawings. Several embodiments of this disclosure are illustrated in the drawings by way of example and not limitation, in which: In the accompanying drawings, the same or corresponding reference numerals indicate the same or corresponding parts.

[0019] Figure 1 This illustration shows the implementation flow of a method for generating dynamic video of wafer defects according to an embodiment of the present disclosure. Figure 1 ; Figure 2 This illustration shows the implementation flow of a method for generating dynamic video of wafer defects according to an embodiment of the present disclosure. Figure 2 ; Figure 3 This illustration shows the implementation flow of a method for generating dynamic video of wafer defects according to an embodiment of the present disclosure. Figure 3 ; Figure 4 This illustration shows the implementation flow of a method for generating dynamic video of wafer defects according to an embodiment of the present disclosure. Figure 4 ; Figure 5 A schematic diagram of the module composition of a wafer defect dynamic video generation apparatus according to an embodiment of the present disclosure is shown; Figure 6 A schematic diagram of the composition structure of an electronic device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0020] To make the objectives, features, and advantages of this disclosure more apparent and understandable, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0021] Figure 1 This illustration shows the implementation flow of a method for generating dynamic video of wafer defects according to an embodiment of the present disclosure. Figure 1 It includes the following steps: Step 101: Obtain the natural language description input by the user, which contains semantic information about wafer defects.

[0022] Users input a natural language description via a graphical user interface or application programming interface. This description includes semantic information about wafer defects, such as their type, size, location, and dynamic response characteristics. Examples include "particle contamination with a diameter of approximately 80 nm appears on the metal layer, exhibiting a periodic scintillation response under rotating dark-field illumination" or "a bridging defect with a linewidth of 50 nm exhibits edge diffraction scintillation under oblique incident dark-field illumination." The system preprocesses the input natural language description, including word segmentation and removal of special characters. It then uses a Contrastive Language-Image Pre-training (CLIP) model to convert the text into tokenized text. Key semantic information is obtained, such as defect types (particles, bridging, scratches, or voids); key dimensional semantic information (linewidth, diameter, thickness, etc.); spatial coordinates (wafer center, edge, or local area); and optical dynamic response characteristics (rotating scintillation, spot translation, or brightness gradation).

[0023] Step 102: Input the natural language description into the semantically driven defect dynamic response synthesis model to generate a two-dimensional scattering intensity dynamic sequence. The two-dimensional scattering intensity dynamic sequence is used to simulate the scattering response captured by the dark field camera in time.

[0024] A semantically driven defect dynamic response synthesis model is input with natural language descriptions. During training, the model learns the mapping relationship from text semantics to scattering dynamic responses. During inference, the model receives text features as conditions and outputs a tensor of dimension F×h×w×1 through a diffusion generation process. Here, F represents the frame number (e.g., 64 frames), h and w are the spatial resolution (e.g., 128×128), and each pixel value represents the scattered light intensity at that spatiotemporal location. This two-dimensional dynamic sequence of scattering intensity simulates the scattering response captured temporally by a dark-field camera, including the spatial distribution of defect scattering and its variation over time.

[0025] Specifically, semantically driven defect dynamic response synthesis models include Temporal Scattering Manifold Encoder (TSME), DefectSynthFormer, a defect synthesis learning model integrating ScatteringChain-Aware Attention (SCA), and Semantic-Scattering Alignment via Dual-Projection (SSADP).

[0026] TSME is essentially a variational autoencoder. Its function is to compress the externally acquired two-dimensional scattering intensity dynamic sequence (dimension F×h×w×1) into a low-dimensional continuous latent manifold during the training phase, and output a low-dimensional latent sequence (dimension F'×h'×w'×d, where F'=F / 2, h'=h / 4, w'=w / 4, d=256). This compression process is achieved through one-dimensional temporal convolution and spatial downsampling. During the inference phase, the decoder part of TSME reconstructs the latent representation generated by DefectSynthFormer into a two-dimensional scattering intensity dynamic sequence in pixel space.

[0027] DefectSynthFormer is a diffusion-based Transformer architecture that performs a diffusion generation process on the low-dimensional latent manifold constructed by TSME. The architecture takes noisy low-dimensional latent sequences as input (training phase) or pure Gaussian noise (inference phase) and outputs denoised low-dimensional latent sequences. Its core innovation lies in replacing the multi-head self-attention in the standard Transformer with a Scattering Chain Aware Attention (SCA) mechanism, allowing the model to explicitly consider the physical propagation laws of scattering from the wafer surface during the generation process. DefectSynthFormer employs a 12-layer Transformer structure with embedded dimensions... =1024, attention count is 16.

[0028] SSADP intervenes only during the training phase, serving as an auxiliary training framework and not participating in the inference process. This framework strengthens the semantic consistency between natural language descriptions and generated scattering dynamics through two projection modules: the first projection narrows the embedding distance of matching text-scattering pairs through contrastive learning; the second projection constrains the generation details to conform to physical scattering laws by mapping the features of the DefectSynthFormer intermediate layer to the semantic space of the pre-trained scattering encoder. The overall training objective of SSADP comprises a weighted sum of diffusion loss, semantic projection loss, and latent projection loss.

[0029] The semantically driven defect dynamic response synthesis model has two stages: Training phase: Import the externally acquired two-dimensional scattering intensity dynamic sequence, TSME performs encoding and reconstruction, DefectSynthFormer performs diffusion noise addition and denoising training, SSADP synchronously calculates semantic contrast loss, latent space L2 alignment loss, diffusion mean squared error (MSE) loss, and backpropagation of the total loss updates all network weights of TSME and DefectSynthFormer. Inference phase: Close the SSADP loss calculation branch, cancel the input of true data, and only retain the TSME encoder and decoder and the DefectSynthFormer diffusion denoising forward calculation path. Rely only on text embedding as the generation condition, and output a two-dimensional scattering intensity gray-scale time sequence that only represents the defect light intensity distribution and does not contain wafer substrate texture.

[0030] The dynamic sequence of external two-dimensional scattering intensity required during the training phase is obtained in the following ways: the scattering response of different defect types such as particle contamination, bridging defects, scratches, and pits under dark field illumination is simulated using the finite-difference time-domain (FDTD) method; and the dynamic sequence of two-dimensional scattering intensity is extracted from actual wafer scanning videos collected by historical dark field inspection equipment.

[0031] Step 103: Generate a wafer background image based on the natural language description.

[0032] The system processes the natural language description input by the user, removes defect-related descriptions, and retains background-related descriptions, such as "clean silicon wafer surface, metal layer texture". The retained background-related descriptions are input into the pre-trained StableDiffusion text image model. The model masks the defect spot areas in keyframes and retains only the contour constraints of the imaging area, generating a clean, defect-free static background image containing the inherent texture of the real wafer. The image size is strictly aligned with the dynamic sequence of two-dimensional scattering intensity to ensure subsequent pixel-level fusion.

[0033] Step 104: Input the two-dimensional scattering intensity dynamic sequence, wafer background image and natural language description into the scattering-guided visual rendering model, and generate a wafer defect dynamic video through multimodal feature fusion and decoding.

[0034] The system inputs a two-dimensional scattering intensity dynamic sequence, a wafer background image, and a natural language description into a visual rendering model. The model uses the scattering sequence as the dynamic driving signal, the background image as the texture source, and the text as the semantic verification condition. Through multimodal feature fusion and decoding, it generates a high-resolution dynamic video of wafer defects frame by frame. The three inputs—the two-dimensional scattering intensity dynamic sequence, the wafer background image, and the natural language description—are encoded and mapped in parallel to a unified latent space. A spatiotemporal separation Transformer alternately stacks spatial attention blocks and temporal attention blocks, relying on multimodal cross-attention to achieve the fusion of wafer texture, scattering dynamics, and textual semantics under triple constraints. The fused temporal latent features are then fed into a video decoder for layer-by-layer upsampling to restore resolution. The output video has a resolution of 512×512 pixels. The generated dynamic video of wafer defects can be directly used for dark field detection algorithm training, equipment calibration, virtual defect reproduction, and system performance verification.

[0035] This application discloses a method for generating dynamic videos of wafer defects. First, it obtains a natural language description input by the user, which contains semantic information about the wafer defects. Then, it inputs the natural language description into a semantically driven defect dynamic response synthesis model to generate a two-dimensional scattering intensity dynamic sequence. This sequence simulates the scattering response captured temporally by a dark-field camera, and a wafer background image is generated based on the natural language description. Finally, the two-dimensional scattering intensity dynamic sequence, the wafer background image, and the natural language description are input into a scattering-guided visual rendering model. Through multimodal feature fusion and decoding, a dynamic video of wafer defects is generated. This method separates the textual semantics and the optical scattering physical response modules, preserving the ability to freely customize defects using natural language while introducing optical priors through independent scattering modeling. The generated dynamic video of wafer defects can be directly used for training dark-field detection equipment.

[0036] In one possible implementation, such as Figure 2 As shown, the temporal scattering manifold encoder compresses an externally acquired two-dimensional dynamic sequence of scattering intensity into a low-dimensional continuous latent manifold to output a low-dimensional latent sequence, including: Step 201: Perform one-dimensional temporal convolution on the input externally acquired two-dimensional scattering intensity dynamic sequence to extract inter-frame dynamic evolution features; Step 202: Compress the spatial dimension through multi-layer convolutional downsampling to obtain bottleneck layer features; Step 203: Output the mean and variance of the bottleneck layer features through variational coding branch; Step 204: Use the reparameterization technique to sample the mean and variance to obtain a low-dimensional latent sequence; Step 205: Constrain the distribution of the low-dimensional latent sequence by using the KL divergence regularization term to make it approximate the standard normal distribution, so as to ensure the continuity of the latent space.

[0037] Input original two-dimensional scattering intensity dynamic sequence First, a one-dimensional temporal convolution operation is performed along the time dimension F, with a kernel size of 3 and a stride of 2. This operation extracts high-order dynamic change features between adjacent frames, including the displacement trajectory of scattering spots, the increase and decrease distribution of pixel light intensity, the periodic flicker frequency, and the gradual change pattern of edge diffraction, outputting a multi-scale temporal feature map that simultaneously preserves the instantaneous changes in short frames and the periodic dynamic features in long frames. After the first temporal convolution layer, the time dimension is reduced from F to F / 2. Subsequently, each temporal convolution layer is followed by a spatial convolution layer with a kernel size of 3×3 and a stride of 2, gradually compressing the spatial dimension. After four layers of alternating temporal-spatial convolution downsampling, the tensor dimension becomes F / 2×h / 4×w / 4×C, where C is the number of channels (this application uses 512 channels as an example). This bottleneck layer feature contains the most essential scattering dynamic information of the input sequence. The bottleneck layer feature is input into two independent linear layers, with one output being the mean of the latent space. The other output is the logarithmic variance. Where F'=F / 2, h'=h / 4, w'=w / 4, d=256. A random noise sample is taken from a standard normal distribution. (0, I), compute low-dimensional latent sequences using reparameterization techniques: This operation makes the sampling process differentiable, facilitating gradient backpropagation. During training, KL divergence loss is introduced to calculate the latent distribution N( The KL distance between the potential vectors and the standard normal prior N(0,1) is incorporated into the loss optimization; during the training process, all potential vectors are continuously constrained to be uniformly distributed in the standard normal space, without local vector clustering and collapse; during the inference stage, linear interpolation can be performed on any two sets of potential vectors corresponding to different defects to directly generate a complete scattering time sequence of intermediate transition states, supporting dynamic simulation of continuously changing defects.

[0038] In one possible implementation, such as Figure 3 As shown, the temporal scattering manifold encoder also includes a signal-to-noise ratio-aware weighting mechanism, specifically including: Step 301: For each pixel in the input two-dimensional scattering intensity dynamic sequence, calculate the local signal-to-noise ratio estimate in its temporal and spatial neighborhoods. The local signal-to-noise ratio estimate is the ratio of the mean to the standard deviation of the pixel values ​​within the neighborhood window. Step 302: Compare the local signal-to-noise ratio estimate with a preset threshold to determine the low signal-to-noise ratio region; Step 303: During model training, adaptive suppression is applied to the low signal-to-noise ratio region. Adaptive suppression is achieved by reducing the weight of the low signal-to-noise ratio region in the reconstruction loss function.

[0039] TSME introduces a signal-to-noise ratio (SNR)-aware weighting mechanism during training to suppress the interference of low SNR regions on latent spatial modeling. For each pixel (i,j,t) in the input two-dimensional scattering intensity dynamic sequence X, the spatiotemporal neighborhood of that pixel is taken. Given (i,j,t), the neighborhood window has a spatial size of 7×7 pixels, and the temporal neighborhood consists of two frames before and after it. Calculate the mean μ of all pixel values ​​within this neighborhood window. ) and standard deviation σ( The local signal-to-noise ratio estimate is defined as SNR_{i,j,t}=μ( ) / (σ( The local signal-to-noise ratio (SNR) estimate is compared with a preset threshold γ. When SNR_{i,j,t} < γ, the pixel is determined to be a low SNR region, which typically corresponds to background noise regions or shadow regions under dark lighting. When SNR_{i,j,t} ≥ γ, it is determined to be a high SNR region, which typically corresponds to defect scattering signal regions. In the reconstruction loss function during model training, a reduced weight is applied to low SNR regions. Specifically, a dynamic weight matrix is ​​constructed. Where W_{i,j,t}=Sigmoid(τ·(SNR_{i,j,t}-γ)), and τ is a temperature coefficient. When SNR is much lower than γ, W approaches 0; when SNR is much higher than γ, W approaches 1. This mechanism forces the TSME encoder to focus its modeling capacity on the defect scattering signal region, ignoring background thermal noise.

[0040] In one possible implementation, such as Figure 4 As shown, the defect synthesis learning model generates corresponding low-dimensional latent sequences based on natural language descriptions, including: Step 401: Divide the noisy low-dimensional latent sequence into several three-dimensional blocks according to the spatiotemporal dimension; Step 402: Flatten each 3D block into a vector and map it to the embedding dimension of the Transformer through linear projection; Step 403: In the embedding dimension, spatial location embedding, temporal rotation location encoding and location offset based on text semantic mapping are fused, and the background region is weighted and suppressed through a dynamic masking mechanism. Step 404: Feature transformation is performed in each layer of the multi-layer Transformer using a scattering chain perceptual attention mechanism. Step 405: Output the denoised low-dimensional latent sequence. The denoised low-dimensional latent sequence and the low-dimensional latent sequence output by the temporal scattering manifold encoder are located in the same latent manifold space.

[0041] The Defect SynthFormer (TSME) transforms text semantics into low-dimensional latent sequences, and converts the low-dimensional latent sequences output by TSME into... The system is divided into several 3D blocks based on the spatiotemporal dimension. Each block has a spatial size of p×p pixels and a temporal depth of q frames. Each 3D block is flattened into a one-dimensional vector of length p×p×q×d, and then mapped to the embedding dimension of the Transformer through a trainable linear projection layer. =1024. Based on this, three location encoding methods are fused: learnable spatial location embedding, marking the position of each block in the spatial grid; temporal rotational location encoding (RoPE), marking the position of each block in the time series; and location offset based on text semantic mapping, which is generated by mapping Contrastive Language-Image Pre-training (CLIP) text features through a Multi-Layer Perceptron (MLP) and then superimposed onto the block embedding. Furthermore, a dynamic masking mechanism is used to weight background regions. For blocks identified as background regions (corresponding to areas with scattering intensity below a threshold), their participation in self-attention computation is reduced to decrease redundant computation. The embedding sequence is sequentially passed through a 12-layer Transformer encoder. In each layer, a Scatter Chain Aware Attention (SCA) mechanism is used instead of standard multi-head self-attention. Each layer also includes a Feed-Forward Neural Network (FFN), layer normalization, and residual connections. The hidden layer dimension of the feed-forward network is 4096. After 12 layers of Transformer encoding, the output sequence is mapped back to the original latent space dimension through a linear layer, resulting in a denoised low-dimensional latent sequence. .

[0042] The size of the 3D block is dynamically adjusted according to the semantic complexity of the natural language description: the first block size is used when the natural language description indicates that the defect distribution is sparse, and the second block size is used when the natural language description indicates that the defect distribution is dense. The second block size is smaller than the first block size. A larger block size helps the model capture global context information, while a smaller block size helps the model retain local details.

[0043] In one possible implementation, the scattering chain sensing attention mechanism includes: Local chain perception subprocess: Construct a physical prior graph to model the propagation constraints of defect scattering in the spatial neighborhood, and inject a physical bias based on the diffraction feature length into the local window attention. The physical bias decays as the spatial distance between pixels increases, so as to simulate the cascade propagation law of scattered energy at the defect edge. Global spatiotemporal coordination subprocess: concatenate the block sequences of all frames and add global temporal position encoding, use sparse global attention to capture long-range dynamic patterns, and inject Fourier features to explicitly capture periodic flicker patterns; Gated fusion subprocess: The output features of the local chain perception subprocess and the global spatiotemporal collaborative subprocess are dynamically weighted and summed through a learnable gating network.

[0044] The scattering chain-aware attention mechanism comprises three sub-processes. The first is a local chain-aware sub-process, which models the propagation constraints of defect scattering within the spatial neighborhood. Specifically, for each query location, the attention calculation is limited to covering only a local window. Instead of standard scaled dot-product attention, a physically-based bias term is injected when calculating the attention score. This bias term is calculated based on the spatial distance between pixels and the diffraction feature length determined by the incident wavelength and numerical aperture. As the spatial distance between two pixels increases, the bias term decays Gaussian, thus constraining a higher probability of cascading propagation of scattering energy between neighboring pixels and suppressing the propagation probability between distant pixels. This injection of physical bias allows the model to automatically follow the physical law of "cascading propagation of scattering energy from defect edges to the neighborhood" when generating scattering dynamics, fundamentally suppressing isolated noise artifacts.

[0045] A global spatiotemporal coordination subprocess is used to capture long-range dynamic patterns. All frame block sequences are concatenated into a long sequence along the temporal dimension, and global temporal positional encoding is added. A sparse global attention machine is employed, selecting interval frames along the temporal dimension with specific step sizes for cross-frame attention computation, rather than performing dense full attention on all frames, thus controlling computational complexity while capturing the periodic dependencies of multiple frames. To further explicitly capture the periodic flickering patterns under rotating illumination conditions, Fourier features γ(t) = [sin(2π] are injected into the query and key y. t),cos(2π t)], where The preset frequency base corresponds to the rotational speed of the rotary table, and t is the frame index.

[0046] A learnable gating network dynamically weights and sums the output features of the local chained perceptual subprocess and the global spatiotemporal collaborative subprocess. The gating network takes the embedding and text features of the current time step as input and outputs two weight coefficients α∈[0,1] and 1-α. The final output is O=α⊙ +(1-α)⊙ This gating mechanism enables the model to adaptively adjust the importance of local physical constraints and global temporal patterns based on the defect type—local constraints are more important for static defects (such as particles), while global temporal patterns are more important for dynamic defects (such as scratches under rotating lighting).

[0047] In one possible implementation, generating a corresponding low-dimensional latent sequence based on a natural language description includes: The text encoder of the contrastive language-image pre-trained model extracts the text semantic features of the natural language description, which are used as keys and values ​​in the cross-attention calculation. Use the scattering potential sequence in the defect synthesis learning model as the query; In each layer of cross-attention computation in the defect synthesis learning model, the different spatiotemporal positions of the scattering potential sequence are dynamically weighted using textual semantic features, so that the generation process focuses on the defect type, size and dynamic behavior described in the text.

[0048] The pre-trained CLIP text encoding branch is loaded to perform word segmentation, word embedding, and multi-layer Transformer encoding on the complete natural language defect text input by the user, outputting a fixed-dimensional global text semantic vector. Two identical text semantic vectors are copied and used as the Key and Value inputs of the cross-attention module, respectively, carrying all text semantic constraint information. The spatiotemporal scattering feature vector enhanced by SCA attention is transformed to the same dimension as the text features through a linear mapping layer, serving as the query vector for cross-attention, realizing cross-modal dimensional matching between scattering temporal features and text semantic features. Each Transformer layer of DefectSynthFormer is independently configured with a set of cross-attention calculation units; the similarity score between the Query (scattering feature) and the Key (text feature) is calculated, with higher similarity indicating a higher degree of matching between the scattering spot at the current spatiotemporal location and the text description defect; the Value feature weights are scaled according to the similarity score, automatically strengthening the defect spot regions of specified size, type, and dynamic in the text, and weakening the invalid scattering features in irrelevant regions. For example, when describing "50nm circular particle defects" in natural language, the model automatically increases the attention weight of the circular region in the center of the image and suppresses the generation of non-target light spots such as long strips and large-area scratches, thus achieving fine-grained and precise control of the text.

[0049] In one implementation, the semantic-scattering dual-projection alignment framework enhances the semantic consistency between natural language descriptions and generated scattering dynamics during the training phase, including: Features are extracted using independent text encoders and scattering dynamic encoders respectively. By contrastive learning, the embedding distance of matching text-scatter pairs is brought closer, while the embedding distance of non-matching text-scatter pairs is pushed further away. Extract the latent representation of the intermediate layer of the defect synthesis learning model, map it to the semantic space of the pre-trained scattering encoder through a lightweight projection network, and calculate the alignment loss between it and the corresponding text or scattering reference embedding. The network parameters of the temporal scattering manifold encoder and the defect synthesis learning model are optimized based on the total loss function.

[0050] The semantic-scattering dual-projection alignment framework enhances the semantic consistency between natural language descriptions and generated scattering dynamics during the training phase. First, it extracts features using independent text encoders and scattering dynamics encoders. Then, using the InfoNCE contrastive loss function, it narrows the embedding distance of matching text-scattering pairs and widens the embedding distance of mismatched text-scattering pairs. The text encoder reuses the CLIP basic coding structure, while the scattering dynamics encoder is a lightweight modification of the TSME coding branch. Within each batch of training samples, positive and negative sample pairs are manually constructed: positive sample pairs consist of text descriptions and corresponding matching scattering time sequences, while negative sample pairs consist of text descriptions and irrelevant defect scattering time sequences. The InfoNCE contrastive loss function maximizes the cosine similarity of the positive sample embedding vectors and minimizes the cosine similarity of the negative sample embedding vectors, forcing a one-to-one correspondence between text and scattering dynamics at a macro-semantic level.

[0051] Then, the temporal latent feature vectors output by the intermediate Transformer layer of DefectSynthFormer are extracted; a two-layer fully connected lightweight projection network is built to map the dimension of the latent vectors to the unified dimension of the pre-trained ScatterCLIP scattering semantic space; the L2 norm distance between the projected feature vectors and the standard scattering reference embedding vectors is calculated to generate the latent space alignment loss, which constrains the generation of scattering dynamics to fit the massive optical simulation priors from the micro-feature level, and eliminates the light spot dynamics that violate the laws of physics.

[0052] The total loss function consists of a weighted sum of three terms: ; To reduce the MSE loss of the DefectSynthFormer diffusion denoising, For the first projection InfoNCE semantic contrast loss, For the potential alignment loss of the second projection L2, The preset loss weight coefficients are used. The total loss is backpropagated as a whole, and the parameters of all networks in the TSME encoder, DefectSynthFormer, and dual projection branches are updated synchronously to achieve diffusion generation, cross-modal semantic alignment, and multi-objective joint optimization with optical and physical constraints.

[0053] In one implementation, the scattering-guided visual rendering model employs a spatiotemporal separation deep learning architecture, which includes alternately stacked spatial attention blocks and temporal attention blocks: Spatial attention blocks use latent features of the scattering sequence as queries and features of the wafer background image as keys and values ​​to calculate spatial cross attention, which is used to process wafer texture details in a single frame. The temporal attention block computes self-attention along the time axis and performs cross-attention verification using text features described in natural language as keys and values ​​to model inter-frame motion coherence.

[0054] The scattering-guided visual rendering model employs a spatiotemporal separation deep learning architecture, with the spatiotemporal separation Transformer architecture using an alternating stacking structure of spatial attention blocks and temporal attention blocks. Spatial attention blocks are used to process single-frame wafer texture details, establishing pixel-level alignment between scattering dynamic spots and background textures. The input consists of the scattering sequence features of the current frame (upsampled to the same resolution as the background image) and wafer background image features. Within the spatial attention block, latent features of the scattering sequence are used as the query, and latent features of the wafer background image are used as the key and value to calculate spatial cross-attention. This operation establishes pixel-level alignment between scattering dynamics and background textures; regions with high scattering intensity drive brightness enhancement and texture modulation in the corresponding regions of the background image, forming a physically believable dark-field scattering visual effect. Each layer of spatial attention blocks performs self-attention operations only on all pixels within a single video frame, calculating the spatial correlation of all pixels within the image; enhancing the rendering of single-frame details such as wafer metal fine textures, thin film bumps, and micro-particle contours, accurately superimposing defect scattering spots onto the corresponding base pixel positions, completely avoiding pixel alignment problems such as spot floating, misalignment, and edge pixel tearing.

[0055] Temporal attention blocks are used to model the motion trajectory of defect spots and the bright-dark flickering period between consecutive frames, ensuring the temporal continuity of defect dynamics and eliminating inter-frame jumps and stuttering artifacts. Each layer of temporal attention blocks calculates temporal correlations across pixels in the same spatial coordinates but different temporal frames, capturing the patterns of spot translation, scaling, and periodic bright-dark alternation. Attention weights are applied to suppress abrupt brightness changes and spot position jumps between adjacent frames, ensuring that the entire process of defect movement and flickering is smooth and continuous, without abrupt frame jumps or flickering breaks.

[0056] In one possible implementation, in a scattering-guided visual rendering model, multimodal feature fusion includes: The wafer background image, each frame of the two-dimensional scattering intensity dynamic sequence, and the text features of the natural language description are respectively mapped to a unified latent space dimension by an encoder; The query backbone is based on the latent representation of the dynamic sequence of two-dimensional scattering intensity, and spatial texture keys and values ​​are provided by the latent features of the wafer background image, while semantic guiding keys and values ​​are provided by text features. Dynamic weighted interaction is achieved through a multimodal cross-attention mechanism. The fused features are then integrated with a feedforward network after layer normalization and input into the decoder to reconstruct the dynamic video frame by frame.

[0057] The rendering model receives three inputs, each matched with a dedicated encoder, and uniformly mapped to a 512-dimensional shared latent space. The wafer background image is input to a dedicated VAE encoder, outputting latent features of the wafer substrate texture; the 2D scattering temporal grayscale sequence is input to a temporal modal variational autoencoder (VAE encoder), outputting dynamic latent features of defect scattering; and the CLIP text encoder, which reuses natural language descriptions, outputs global text semantic embedding features. The output dimensions of these three encoders are completely unified, eliminating cross-modal dimensional differences between images, temporal sequences, and text, ensuring compatibility for subsequent attention calculations.

[0058] The system uses the latent representation of the scattering sequence as the main query, background image features as the key / value pair providing spatial texture, and text embedding as the key / value pair providing semantic guidance. A fixed cross-attention triplet allocation rule ensures that the dynamics of defect scattering are the core constraint of the image: the temporal latent features of scattering serve as the main query throughout; the wafer background texture features are independently used as the first set of key / value pairs, responsible for constraining the details of the substrate texture; and the text semantic embedding is independently used as the second set of key / value pairs, responsible for constraining the defect type, size, and dynamic behavior.

[0059] Two independent cross-attention scores are calculated in layers: the scattering query and background key / value are weighted and fused with wafer substrate texture information; the scattering query and text key / value are weighted and corrected for defect spot morphology and temporal dynamics; the two sets of attention output features are added and fused channel by channel, simultaneously taking into account the triple constraints of wafer texture, scattering optical response and text semantics.

[0060] The fused multimodal features are sequentially normalized and then processed by two layers of GELU-activated feedforward networks to complete feature integration. The integrated temporal latent features are then fed into the video decoder, where multi-layer transposed convolutions are used to upsample and restore the original image resolution, and continuous wafer defect video images are output frame by frame.

[0061] Figure 5 This diagram illustrates the module composition of a device for generating dynamic video of wafer defects according to an embodiment of this application.

[0062] like Figure 5As shown, according to a second aspect of this application, an apparatus for generating a dynamic video of wafer defects is provided. The apparatus includes: an acquisition module 501, used to acquire a natural language description input by a user, the natural language description containing semantic information about wafer defects; a first generation module 502, used to input the natural language description into a semantically driven defect dynamic response synthesis model to generate a two-dimensional scattering intensity dynamic sequence, the two-dimensional scattering intensity dynamic sequence being used to simulate the scattering response captured temporally by a dark-field camera; a second generation module 503, used to generate a wafer background image based on the natural language description; and a third generation module 504, used to input the two-dimensional scattering intensity dynamic sequence, the wafer background image, and the natural language description together into a scattering-guided visual rendering model, and generate a dynamic video of wafer defects through multimodal feature fusion and decoding; wherein... The first generation module 502 includes: a temporal scattering manifold encoder module 5021, used to compress the externally acquired two-dimensional scattering intensity dynamic sequence into a low-dimensional continuous latent manifold to output a low-dimensional latent sequence during the training phase, and to reconstruct the latent representation generated by the text into a two-dimensional scattering intensity dynamic sequence during the inference phase; a defect synthesis learning model module 5022, used to perform a diffusion generation process on the low-dimensional continuous latent manifold constructed by the temporal scattering manifold encoder, and generate a corresponding low-dimensional latent sequence according to the natural language description, the generated low-dimensional latent sequence can be reconstructed into a two-dimensional scattering intensity dynamic sequence by the temporal scattering manifold encoder; and a semantic-scattering dual projection alignment framework module 5023, used to strengthen the semantic consistency between the natural language description and the generated scattering dynamics during the training phase.

[0063] In one embodiment, the temporal scattering manifold encoder module 5021 is specifically used to: perform one-dimensional temporal convolution on the input externally acquired two-dimensional scattering intensity dynamic sequence to extract inter-frame dynamic evolution features; compress the spatial dimension through multi-layer convolution downsampling to obtain bottleneck layer features; output the mean and variance corresponding to the bottleneck layer features through variational coding branches; sample the mean and variance using reparameterization techniques to obtain a low-dimensional latent sequence; and constrain the distribution of the low-dimensional latent sequence through KL divergence regularization terms to make it approximate a standard normal distribution to ensure the continuity of the latent space.

[0064] In one embodiment, the temporal scattering manifold encoder module 5021 further includes a signal-to-noise ratio (SNR) sensing weighting mechanism, specifically used to calculate the local SNR estimate in the temporal and spatial neighborhoods of each pixel in the input two-dimensional scattering intensity dynamic sequence. The local SNR estimate is the ratio of the mean to the standard deviation of the pixel values ​​within the neighborhood window. The local SNR estimate is compared with a preset threshold to determine low SNR regions. During model training, adaptive suppression is applied to low SNR regions, which is achieved by reducing the weight of low SNR regions in the reconstruction loss function.

[0065] In one implementation, the defect synthesis learning model module 5022 is specifically used to divide the noisy low-dimensional latent sequence into several three-dimensional blocks according to the spatiotemporal dimension; flatten each three-dimensional block into a vector and map it to the embedding dimension of the Transformer through linear projection; fuse spatial position embedding, temporal rotation position encoding and position offset based on text semantic mapping in the embedding dimension, and perform weighted suppression of the background region through a dynamic masking mechanism; perform feature transformation in each layer of the multi-layer Transformer using a scattering chain perceptual attention mechanism; and output the denoised low-dimensional latent sequence, which is located in the same latent manifold space as the low-dimensional latent sequence output by the temporal scattering manifold encoder.

[0066] In one possible implementation, the scattering chain perception attention mechanism includes: a local chain perception subprocess, used to construct a physical prior map to model the propagation constraints of defect scattering in the spatial neighborhood, injecting a physical bias based on the diffraction feature length into the local window attention, the physical bias attenuating as the spatial distance between pixels increases, to simulate the cascading propagation law of scattering energy at the defect edge; a global spatiotemporal coordination subprocess, used to concatenate the block sequences of all frames and add global temporal position encoding, using sparse global attention to capture long-range dynamic patterns, and injecting Fourier features to explicitly capture periodic flickering patterns; and a gated fusion subprocess, used to dynamically weight and sum the output features of the local chain perception subprocess and the global spatiotemporal coordination subprocess through a learnable gating network.

[0067] In one embodiment, the defect synthesis learning model module 5022 is further configured to extract textual semantic features of the natural language description using the text encoder of the contrastive language-image pre-trained model, as keys and values ​​in the cross-attention calculation; use the scattering latent sequence in the defect synthesis learning model as a query; and in each layer of cross-attention calculation in the defect synthesis learning model, dynamically weight the different spatiotemporal positions of the scattering latent sequence using textual semantic features, so that the generation process focuses on the defect type, size and dynamic behavior of the text description.

[0068] In one embodiment, the semantic-scattering dual-projection alignment framework module 5023 is specifically used to extract features using independent text encoders and scattering dynamic encoders respectively, to narrow the embedding distance of matching text-scattering pairs through contrastive learning, and to widen the embedding distance of non-matching text-scattering pairs; to extract the latent representation of the intermediate layer of the defect synthesis learning model, to map it to the semantic space of the pre-trained scattering encoder through a lightweight projection network, and to calculate the alignment loss between it and the corresponding text or scattering reference embedding; and to optimize the network parameters of the temporal scattering manifold encoder and the defect synthesis learning model based on the total loss function.

[0069] In one implementation, the scattering-guided visual rendering model employs a spatiotemporal separation deep learning architecture, which includes alternating stacked spatial attention blocks and temporal attention blocks: the spatial attention block uses the latent features of the scattering sequence as the query and the wafer background image features as the key and value to calculate spatial cross attention, which is used to process wafer texture details in a single frame; the temporal attention block calculates self-attention along the time axis and performs cross attention verification using text features described in natural language as the key and value, which is used to model inter-frame motion coherence.

[0070] In one possible implementation, the third generation module 504 is specifically used to map the wafer background image, each frame of the two-dimensional scattering intensity dynamic sequence, and the text features of the natural language description to a unified latent spatial dimension via an encoder; using the latent representation of the two-dimensional scattering intensity dynamic sequence as the query backbone, using the latent features of the wafer background image to provide spatial texture keys and values, and using the text features to provide semantic guidance keys and values; performing dynamic weighted interaction through a multimodal cross-attention mechanism, and after the fused features are integrated with layer normalization and a feedforward network, inputting them into the decoder to reconstruct the dynamic video frame by frame.

[0071] According to embodiments of this disclosure, this disclosure also provides an electronic device and a readable storage medium.

[0072] Figure 6 A schematic block diagram of an example electronic device 600 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0073] like Figure 6 As shown, the electronic device 600 includes a computing unit 601, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. The RAM 603 may also store various programs and data required for the operation of the electronic device 600. The computing unit 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0074] Multiple components in electronic device 600 are connected to I / O interface 605, including: input unit 606, such as keyboard, mouse, etc.; output unit 607, such as various types of displays, speakers, etc.; storage unit 608, such as disk, optical disk, etc.; and communication unit 609, such as network card, modem, wireless transceiver, etc. Communication unit 609 allows electronic device 600 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0075] The computing unit 601 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 601 performs the various methods and processes described above, such as a method for generating a dynamic video of wafer defects. For example, in some embodiments, a method for generating a dynamic video of wafer defects can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 608. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 600 via ROM 602 and / or communication unit 609. When the computer program is loaded into RAM 603 and executed by the computing unit 601, one or more steps of the method for generating a dynamic video of wafer defects described above can be performed. Alternatively, in other embodiments, the computing unit 601 may be configured by any other suitable means (e.g., by means of firmware) to perform a method for generating a dynamic video of wafer defects.

[0076] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0077] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0078] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0079] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0080] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0081] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0082] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this disclosure can be achieved, and this is not limited herein.

[0083] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this disclosure, "a plurality of" means two or more, unless otherwise explicitly specified.

[0084] The above description is merely a specific embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this disclosure should be included within the scope of protection of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

1. A method for generating a dynamic video of wafer defects, the method comprising: The method includes: Obtain a natural language description input by the user, the natural language description containing semantic information about wafer defects; The natural language description is input into a semantically driven defect dynamic response synthesis model to generate a two-dimensional scattering intensity dynamic sequence, which is used to simulate the scattering response captured by a dark field camera in time. Generate a wafer background image based on the natural language description; The two-dimensional scattering intensity dynamic sequence, the wafer background image, and the natural language description are input into the scattering-guided visual rendering model, and a wafer defect dynamic video is generated through multimodal feature fusion and decoding. The semantically driven defect dynamic response synthesis model includes: a temporal scattering manifold encoder, used to compress an externally acquired two-dimensional scattering intensity dynamic sequence into a low-dimensional continuous latent manifold during the training phase to output a low-dimensional latent sequence, and to reconstruct the latent representation generated by text into a two-dimensional scattering intensity dynamic sequence during the inference phase; a defect synthesis learning model, used to perform a diffusion generation process on the low-dimensional continuous latent manifold constructed by the temporal scattering manifold encoder, generating a corresponding low-dimensional latent sequence based on the natural language description, and the generated low-dimensional latent sequence can be reconstructed into the two-dimensional scattering intensity dynamic sequence by the temporal scattering manifold encoder; and a semantic-scattering dual-projection alignment framework, used to enhance the semantic consistency between the natural language description and the generated scattering dynamics during the training phase.

2. The method of claim 1, wherein, The temporal scattering manifold encoder compresses an externally acquired two-dimensional scattering intensity dynamic sequence into a low-dimensional continuous latent manifold to output a low-dimensional latent sequence, including: One-dimensional temporal convolution is performed on the input externally acquired two-dimensional scattering intensity dynamic sequence to extract inter-frame dynamic evolution features; Bottleneck layer features are obtained by compressing spatial dimensions through multi-layer convolutional downsampling. The mean and variance corresponding to the bottleneck layer features are output through the variational coding branch; By using reparameterization techniques to sample the mean and variance, a low-dimensional latent sequence is obtained; The distribution of the low-dimensional latent sequence is constrained by the KL divergence regularization term to approximate the standard normal distribution, thereby ensuring the continuity of the latent space.

3. The method of claim 2, wherein, The temporal scattering manifold encoder also includes a signal-to-noise ratio-aware weighting mechanism, specifically comprising: For each pixel in the input two-dimensional scattering intensity dynamic sequence, calculate the local signal-to-noise ratio estimate in its temporal and spatial neighborhoods. The local signal-to-noise ratio estimate is the ratio of the mean to the standard deviation of the pixel values ​​within the neighborhood window. The local signal-to-noise ratio estimate is compared with a preset threshold to determine the low signal-to-noise ratio region. During model training, adaptive suppression is applied to the low signal-to-noise ratio region, which is achieved by reducing the weight of the low signal-to-noise ratio region in the reconstruction loss function.

4. The method of claim 1, wherein, The defect synthesis learning model generates corresponding low-dimensional latent sequences based on the natural language description, including: The noisy low-dimensional latent sequence is divided into several three-dimensional blocks according to the spatiotemporal dimension; Each 3D block is flattened into a vector and mapped to the embedded dimension of the Transformer via linear projection; In the embedding dimension, spatial location embedding, temporal rotation location encoding, and location offset based on text semantic mapping are integrated, and the background region is weighted and suppressed through a dynamic masking mechanism; In each layer of a multi-layer Transformer, a scattering chain-based perceptual attention mechanism is used for feature transformation. The output is a denoised low-dimensional latent sequence, which is located in the same latent manifold space as the low-dimensional latent sequence output by the temporal scattering manifold encoder.

5. The method of claim 4, wherein, The scattering chain perception attention mechanism includes: Local chain perception subprocess: Construct a physical prior graph to model the propagation constraints of defect scattering in the spatial neighborhood, and inject a physical bias based on the diffraction feature length into the local window attention. The physical bias decays as the spatial distance between pixels increases, so as to simulate the cascade propagation law of scattered energy at the defect edge. Global spatiotemporal coordination subprocess: concatenate the block sequences of all frames and add global temporal position encoding, use sparse global attention to capture long-range dynamic patterns, and inject Fourier features to explicitly capture periodic flicker patterns; Gated fusion subprocess: The output features of the local chain perception subprocess and the global spatiotemporal collaborative subprocess are dynamically weighted and summed through a learnable gating network.

6. The method of claim 1, wherein, The step of generating the corresponding low-dimensional latent sequence based on the natural language description includes: The text encoder of the contrastive language-image pre-trained model extracts the text semantic features of the natural language description, which are used as keys and values ​​in the cross-attention calculation; Use the scattering potential sequence in the defect synthesis learning model as the query; In each layer of cross-attention calculation in the defect synthesis learning model, the text semantic features are used to dynamically weight the different spatiotemporal locations of the scattering potential sequence, so that the generation process focuses on the defect type, size and dynamic behavior described in the text.

7. The method according to claim 1, characterized in that, The semantic-scattering dual-projection alignment framework enhances the semantic consistency between the natural language description and the generated scattering dynamics during the training phase, including: Features are extracted using independent text encoders and scattering dynamic encoders respectively. By contrastive learning, the embedding distance of matching text-scatter pairs is brought closer, while the embedding distance of non-matching text-scatter pairs is pushed further away. Extract the latent representation of the intermediate layer of the defect synthesis learning model, map it to the semantic space of the pre-trained scattering encoder through a lightweight projection network, and calculate the alignment loss between it and the corresponding text or scattering reference embedding. The network parameters of the temporal scattering manifold encoder and the defect synthesis learning model are optimized based on the total loss function.

8. The method according to claim 1, characterized in that, The scattering-guided visual rendering model employs a spatiotemporal separation deep learning architecture, which includes alternately stacked spatial attention blocks and temporal attention blocks. The spatial attention block uses the latent features of the scattering sequence as the query and the features of the wafer background image as the key and value to calculate spatial cross attention, which is used to process single-frame wafer texture details. The temporal attention block calculates self-attention along the time axis and performs cross-attention verification using the text features described in the natural language as keys and values, in order to model inter-frame motion coherence.

9. The method according to claim 8, characterized in that, In the scattering-guided visual rendering model, multimodal feature fusion includes: The wafer background image, each frame of the two-dimensional scattering intensity dynamic sequence, and the text features of the natural language description are respectively mapped to a unified latent space dimension by an encoder; The latent representation of the two-dimensional scattering intensity dynamic sequence is used as the query backbone, the latent features of the wafer background image are used to provide spatial texture keys and values, and the text features are used to provide semantic guidance keys and values; Dynamic weighted interaction is achieved through a multimodal cross-attention mechanism. The fused features are then integrated with a feedforward network after layer normalization and input into the decoder to reconstruct the dynamic video frame by frame.

10. An apparatus for generating dynamic video of wafer defects, characterized in that, The device includes: The acquisition module is used to acquire a natural language description input by the user, wherein the natural language description contains semantic information about wafer defects; The first generation module is used to input the natural language description into a semantically driven defect dynamic response synthesis model to generate a two-dimensional scattering intensity dynamic sequence, which is used to simulate the scattering response captured by a dark field camera in time. The second generation module is used to generate a wafer background image based on the natural language description; The third generation module is used to input the two-dimensional scattering intensity dynamic sequence, the wafer background image and the natural language description into the scattering-guided visual rendering model, and generate a wafer defect dynamic video through multimodal feature fusion and decoding. The first generation module includes: a temporal scattering manifold encoder module, used to compress an externally acquired two-dimensional scattering intensity dynamic sequence into a low-dimensional continuous latent manifold to output a low-dimensional latent sequence during the training phase, and to reconstruct the latent representation generated from text into a two-dimensional scattering intensity dynamic sequence during the inference phase; a defect synthesis learning model module, used to perform a diffusion generation process on the low-dimensional continuous latent manifold constructed by the temporal scattering manifold encoder, and generate a corresponding low-dimensional latent sequence according to the natural language description, wherein the generated low-dimensional latent sequence can be reconstructed into the two-dimensional scattering intensity dynamic sequence by the temporal scattering manifold encoder; and a semantic-scattering dual-projection alignment framework module, used to enhance the semantic consistency between the natural language description and the generated scattering dynamics during the training phase.

Images