A scanning electron microscope image restoration method based on wavelet frequency domain adjusted diffusion model

Abstract

The application relates to the technical field of scanning electron microscope image processing, and discloses a scanning electron microscope image restoration method based on a wavelet frequency domain adjustment diffusion model, which comprises the following steps: a two-stage degradation pipeline containing random light path disturbance and fixed circuit acquisition restriction is constructed to generate a specific domain training data set; a frequency domain prior encoder is used to extract multi-scale high-frequency energy feature modulation noise distribution, a wavelet frequency domain adjustment diffusion model is trained; a target low-quality degraded image is input into a pre-trained model, an inverse denoising iteration is performed after initializing a latent noisy state, a wavelet domain consistency gradient is used to correct noise prediction, and the latent noisy state is updated until a restored image is generated. The application relieves data distribution mismatch through physical degradation modeling, enhances high-frequency detail perception with the aid of a frequency domain adjustment mechanism, improves structure fidelity by using a wavelet domain gradient guide, and realizes the balance between scanning electron microscope image noise suppression and detail recovery.

Description

A Scanning Electron Microscopy Image Restoration Method Based on Wavelet Frequency Domain Modulated Diffusion Model Technical Field

[0001] This invention relates to the field of scanning electron microscope (SEM) image processing technology, specifically to a method for restoring SEM images based on a wavelet frequency domain modulated diffusion model. Background Technology

[0002] Scanning electron microscopy (SEM) is an important tool for characterizing microstructures in fields such as materials science and biology. However, due to limitations in electron beam intensity, sample tolerance, and the physical characteristics of the imaging system, actual SEM images are often extremely susceptible to various degradation factors, leading to reduced signal-to-noise ratio and resolution loss.

[0003] Existing techniques for image restoration typically employ methods based on convolutional neural networks or general generative models. However, these methods largely rely on common assumptions about natural image degradation, such as using a single additive white Gaussian noise model or a simplified blur kernel to construct training data. The imaging mechanism of scanning electron microscopy involves the interaction between the electron beam and the sample, non-rotational symmetric astigmatism, distortion caused by charge effects, and quantization noise in the detector circuitry. Its degradation process is highly physically complex and nonlinear. Existing techniques neglect these specific physical optical path perturbations and circuit acquisition limitations, resulting in a significant deviation in the statistical properties of the simulated training data from the distribution of real-world scanning electron microscope images. This leads to insufficient generalization ability and robustness of the trained model when dealing with complex degraded images in real-world scenarios.

[0004] Furthermore, diffusion models, as the current mainstream generative restoration framework, are primarily trained by optimizing the denoising target within the pixel space. This isotropic training method often dominates the large-scale low-frequency structure of the image, and the network tends to neglect the reconstruction of high-frequency components. In the field of microscopic imaging, where extremely high detail is required, this deficiency manifests as the restored image removing noise, but micron and nanometer-level edge features and texture details being excessively smoothed, resulting in the loss of effective information.

[0005] Finally, in the reverse reasoning stage based on the generative model, because the image is recovered from noise through random sampling, the existing methods lack an effective constraint mechanism for intermediate states, which easily leads the generation process to deviate from the limitations of the original observation data. The restored result often exhibits false textures that do not match the actual sample structure, or destroys the original low-frequency topological structure of the image while removing noise, making it difficult to maintain the fidelity of the physical structure while ensuring noise suppression. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention provides a scanning electron microscope (SEM) image restoration method based on a wavelet frequency domain-adjusted diffusion model. This method solves the problems in existing SEM image restoration techniques, such as the mismatch between training data distribution and real-world scenes due to the complexity of physical degradation mechanisms, and the inability of diffusion models to simultaneously consider high-frequency texture details and low-frequency structural consistency during generation, which can easily lead to artifacts or over-smoothing.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a scanning electron microscope image restoration method based on a wavelet frequency domain modulated diffusion model, comprising the following steps:

[0008] Step S1: Construct a high-quality reference image pool based on real scanning electron microscope images. Process the high-quality reference image pool using a two-stage degradation pipeline that includes random optical path perturbation and fixed circuit acquisition limitations to generate corresponding low-quality images. This process constructs a domain-specific training dataset that includes semantic alignment between the images in the high-quality reference image pool and the low-quality images.

[0009] Step S2: Input the specific domain training dataset constructed in step S1 into a preset conditional diffusion network. During the forward training process, use the frequency domain prior encoder to extract multi-scale high-frequency energy features to modulate the noise distribution. Optimize the network parameters by minimizing the denoising prediction error to obtain a pre-trained wavelet frequency domain modulated diffusion model.

[0010] Step S3: Obtain the low-quality degraded image of the target to be processed, and use it as a conditional input to the pre-trained wavelet frequency domain modulated diffusion model obtained in step S2, and sample from the standard Gaussian distribution to initialize the potential noisy state at the current time.

[0011] Step S4: Use the pre-trained wavelet frequency domain modulated diffusion model to perform inverse denoising iteration on the potential noisy state initialized in step S3. Calculate the wavelet domain consistency gradient between the estimated clean image obtained based on the potential noisy state and the target low-quality degraded image in each time step. Inject the wavelet domain consistency gradient into the sampling process and correct the noise prediction results to update the potential noisy state.

[0012] Step S5: Repeat the reverse denoising iteration until the preset termination time step is reached, map the potential noisy state finally updated in step S4 back to the pixel space, and output the corresponding restored high-quality scanning electron microscope image.

[0013] Preferably, in step S1, the two-stage degradation pipeline processing of the high-quality reference image pool, which includes random optical path perturbation and fixed circuit acquisition constraints, specifically involves: configuring a system-preset first degradation operator set containing blur components, noise components, and artifact components; and using a random out-of-order strategy to superimpose the first degradation operator set onto the images in the high-quality reference image pool to generate an intermediate state image containing physical optical path features. The blur component is applied through a convolution operation, the noise component simulates statistical noise in the imaging process, and the artifact component is constructed based on the imaging characteristics of a scanning electron microscope. Subsequently, the analog signal is constrained by circuit characteristics during the detector's conversion to a digital signal, and deterministic signal link processing is performed on the intermediate state image, including applying dynamic range constraints, superimposing readout noise, and performing resizing operations according to a preset target resolution specification to generate the final low-quality image.

[0014] Preferably, the artifact components include: artifacts generated by simulating electron beam damage through the application of a local low-grayscale mask; artifacts generated by simulating the charging effect through the superposition of high-brightness saturation regions; and artifacts generated by simulating astigmatism using a non-rotationally symmetric elliptical Gaussian kernel.

[0015] Preferably, in step S2, the extraction of multi-scale high-frequency energy features using a frequency domain prior encoder specifically involves: performing a discrete wavelet transform on the high-quality reference image in the specific domain training dataset, decomposing it into a set of low-frequency approximate sub-bands and a set of high-frequency sub-bands containing horizontal, vertical, and diagonal details. Based on the set of high-frequency sub-bands, a multi-level pyramid decomposition is performed on the image, and the spatial high-frequency intensity at each decomposition scale is extracted using a single-layer wavelet high-frequency energy calculation formula. The extracted spatial high-frequency intensity is input into a multi-scale wavelet energy fusion formula to generate a global frequency domain guiding map; the global frequency domain guiding map serves as the multi-scale high-frequency energy feature to preserve micron and nanometer-level edge detail information.

[0016] Preferably, in step S2, the process of modulating the noise distribution specifically involves: performing mean-reduction processing on the global frequency domain guiding map to calculate the energy perturbation; combining the energy perturbation with a preset time-dependent modulation intensity, processing the standard noise using a frequency domain-aware noise modulation formula to generate modulated intermediate noise; renormalizing the intermediate noise using a modulation noise channel-level variance normalization formula to obtain the modulation noise for the final injection diffusion process, and using the modulation noise to generate training samples.

[0017] Preferably, the preset conditional diffusion network adopts a conditional diffusion model architecture based on residual displacement: the conditional distribution of the intermediate state at any time is defined by the ResShift conditional forward distribution formula; the specific intermediate state is generated by the ResShift conditional forward sampling formula using the modulation noise after modulation by the frequency domain prior encoder; the intermediate state is input into the network to calculate the predicted value, and the network parameters are updated by minimizing the general training error formula of the denoising network through the stochastic gradient descent algorithm.

[0018] Preferably, in step S4, the calculation of the estimated clean image based on the potential noisy state specifically involves: using the pre-trained wavelet frequency domain modulated diffusion model to receive the potential noisy state and outputting the noise prediction value in the current state; using the noise prediction value, through inverse reparameterization mapping, and employing the clean image estimation formula based on the current state to calculate the estimated clean image at the current time; the estimated clean image represents a single-point estimate of the original high-quality image by the model at the current noise level.

[0019] Preferably, in step S4, calculating the wavelet domain consistency gradient between the estimated clean image and the target low-quality degraded image specifically involves: acquiring the estimated clean image and the target low-quality degraded image; performing discrete wavelet transform on both to generate corresponding sub-band feature maps; constructing a wavelet domain consistency measure based on the generated sub-band feature maps using the wavelet domain negative log-likelihood loss function formula; calculating the differences between the sub-band feature maps at multiple scales using the wavelet domain consistency measure and setting sub-band importance weights to force structural consistency; and calculating the gradient relative to the estimated clean image based on the wavelet domain consistency measure to obtain the wavelet domain consistency gradient.

[0020] Preferably, in step S4, injecting the wavelet domain consistency gradient into the sampling process and correcting the noise prediction result specifically involves: mapping the wavelet domain consistency gradient from the estimated clean image back to the current sampling state using a chain rule; introducing a time-decaying guidance strategy and calculating the dynamic guidance step size at the current moment using the time-dependent strength formula of frequency domain guidance; weighting and fusing the wavelet domain consistency gradient and the dynamic guidance step size, injecting it into the original prediction of the denoising network, and calculating the corrected noise using the noise prediction formula after frequency domain consistency correction. By subtracting the weighted wavelet domain consistency gradient from the original predicted noise, components in the noise distribution that are inconsistent with the observed data structure are suppressed, thus completing the generation of the corrected noise.

[0021] Preferably, updating the potential noisy state specifically involves: using the corrected noise, solving for the potential noisy state at the previous moment through the inverse iterative equation of ResShift, until the sampling process is completed.

[0022] This invention provides a scanning electron microscope image restoration method based on a wavelet frequency domain modulated diffusion model. It has the following advantages:

[0023] 1. This invention solves the problem of inconsistent training data distribution with real scanning electron microscope images caused by a single noise model by constructing a two-stage degradation model that includes random optical path perturbation and fixed circuit acquisition constraints. The method simulates physical characteristics such as blurring, non-rotational symmetric astigmatism, charge effect and circuit quantization noise, so that the generated training data is closer to the actual imaging scene in terms of statistical characteristics, thereby improving the robustness and generalization ability of the restoration model when processing real degraded images.

[0024] 2. This invention utilizes the multi-resolution characteristics of wavelet transform to design a frequency domain adjustment mechanism, which solves the problem of easily ignoring high-frequency details in diffusion model training. By extracting multi-scale high-frequency energy features through a frequency domain prior encoder and encoding them into the training noise distribution, the network is guided to prioritize the recovery of micron and nanometer-level edge and texture information during the denoising process, effectively improving the phenomenon of overly smoothed or blurred details in the generated image.

[0025] 3. This invention introduces a wavelet domain consistency gradient guidance strategy in the inference stage, which solves the problem of false textures or low-frequency structural deviations that are easy to be generated in the generation process. In the reverse sampling step, the difference gradient between the estimated image and the observed data in the wavelet sub-band is calculated in real time and injected into the noise prediction stage to correct the sampling trajectory, ensuring that the restoration result retains clear high-frequency details and maintains consistency with the low-frequency topological structure of the original data. Attached Figure Description

[0026] Figure 1 is an example of a typical degradation phenomenon present in a real scanning electron microscope image;

[0027] Figure 2 is a schematic diagram of a two-stage physical degradation data generation pipeline according to an embodiment of the present invention;

[0028] Figure 3 is a schematic diagram of the structure and signal processing flow of the frequency domain prior encoder (FPE) in an embodiment of the present invention;

[0029] Figure 4 is a flowchart of the overall workflow for training and inference of the wavelet frequency domain modulated diffusion model according to an embodiment of the present invention.

[0030] Figure 5 is a comparison of the visual quality of the images restored by the embodiments of the present invention and the existing methods. Detailed Implementation

[0031] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0032] Referring to Figure 4, the present invention provides a scanning electron microscope image restoration method based on a wavelet frequency domain modulated diffusion model, comprising the following steps:

[0033] S1. A high-quality reference image pool is constructed based on real scanning electron microscope images. The high-quality reference image pool is processed using a two-stage degradation pipeline that includes random optical path perturbation and fixed circuit acquisition constraints to generate corresponding low-quality images. This results in the construction of a domain-specific training dataset containing semantically aligned high-quality reference images and low-quality degradation image pairs.

[0034] S2, input the specific domain training dataset into the pre-set conditional diffusion network, and use the frequency domain prior encoder to extract multi-scale high-frequency energy features of the image to modulate the noise distribution during the forward process of training. Then, optimize the network parameters by minimizing the denoising prediction error to obtain the pre-trained wavelet frequency domain modulated diffusion model.

[0035] S3, acquire the low-quality degraded image of the target to be processed, use it as a conditional input to pre-train a wavelet frequency domain modulated diffusion model, and sample from a standard Gaussian distribution to initialize the potential noisy state at the current time.

[0036] S4. The pre-trained wavelet frequency domain modulated diffusion model is used to perform inverse denoising iteration on the potential noisy state. In each time step, the wavelet domain consistency gradient between the clean image and the low-quality degraded image based on the current state estimate is calculated, and the gradient is injected into the sampling process to correct the noise prediction results and update the potential noisy state.

[0037] S5 repeats the reverse denoising iteration until the preset termination time step is reached, and finally maps the updated potential noisy state back to the pixel space, outputting the corresponding restored high-quality scanning electron microscope image.

[0038] The method proposed in this invention will now be described in detail based on the steps described above.

[0039] Referring to Figures 1 and 2, this embodiment of the invention proposes a method for constructing a specific domain training dataset based on physical degradation modeling, used to establish a mapping relationship from high-quality reference images to low-quality degraded images. The specific implementation process includes the following steps:

[0040] S101, Establish a high-quality reference image pool.

[0041] High signal-to-noise ratio (SNR) scanning electron microscope (SEM) images acquired during long integration times or in high beam intensity modes are selected as the base data, or high-resolution SEM images are acquired and downsampled. Downsampling utilizes spatial resolution compression to smooth high-frequency noise, thereby obtaining high-quality reference images with clear textures and extremely low noise, forming a high-quality reference image pool with consistent semantic benchmarks.

[0042] S102, Perform the first stage of degradation simulation: random optical path perturbation.

[0043] Referring to Figure 2, this step simulates physical interference in an electro-optical system. The system pre-sets a first set of degradation operators containing blur components, noise components, and artifact components, and uses a random disorder strategy to superimpose these components onto the input high-quality reference image.

[0044] Specifically, the blur components are applied through convolution operations, including: using spatially varying blur kernels to simulate defocus blur caused by the focal plane deviating from the sample surface; using isotropic Gaussian kernels to simulate Gaussian blur caused by the point spread function inherent in the lens system; and using linear motion blur kernels to simulate motion blur caused by pixel displacement due to slight deviation of the sample stage or environmental vibration during the imaging process.

[0045] The noise components mainly simulate statistical noise in the imaging process, including: Poisson noise caused by fluctuations in the number of incident electrons, whose noise intensity is dependent on the local brightness of the image; Gaussian noise caused by thermal noise of the background circuit; and stripe noise caused by instability of interline signals during raster scanning, which is a periodic horizontal or vertical intensity interference.

[0046] The artifact component models the imaging characteristics of the scanning electron microscope, including: simulating carbon buildup or melting of the sample surface caused by prolonged high-energy electron beam bombardment (electron beam damage) by applying a local low-grayscale mask; simulating local discharge (charge effect) caused by charge accumulation on the poorly conductive sample surface by superimposing high-brightness saturation regions; and simulating astigmatism caused by magnetic lens inhomogeneity using a non-rotationally symmetric elliptical Gaussian kernel, resulting in image stretching and distortion in specific directions. Through random combinations and parameterized adjustments of these operators, an intermediate-state image containing the physical characteristics of the optical path is generated.

[0047] S103, Perform the second stage of degradation simulation: fixed circuit acquisition limit.

[0048] Referring to the lower half of Figure 2, this step involves circuit characteristics limiting the process of converting the analog signal into a digital signal through the detector, and performing deterministic signal link processing on the intermediate state image.

[0049] First, a dynamic range limit is applied to the intermediate state image. The response range of the detector is simulated by a nonlinear truncation function or an S-shaped mapping curve, causing the highlight areas exceeding the threshold to overflow and the details in the dark areas to be compressed and lost, thereby reproducing the contrast-limited characteristics of real imaging.

[0050] Secondly, readout noise is superimposed on the image. The readout noise is an additive mixed Gaussian white noise that is independent of the signal content and is introduced by the analog signal amplifier and analog-to-digital converter.

[0051] Finally, based on the preset target resolution specifications, the image is resized, that is, spatial downsampling is performed through bicubic or bilinear interpolation to match the spatial sampling rate of the target low-quality image.

[0052] After processing through steps S101 to S103, the generated low-quality degraded images and the images in the high-quality reference image pool constitute a pixel-level strictly aligned training dataset, which covers the complete physical degradation features from electro-optical interactions to digital signal quantization.

[0053] Referring to Figures 3 and 4, this embodiment of the invention proposes a method for constructing and training a wavelet frequency domain modulated diffusion model based on the ResShift framework. This method introduces a frequency domain sensing mechanism during the training phase of image restoration, and modulates the noise distribution during the diffusion process by explicitly extracting high-frequency energy features of the image. The specific implementation process includes the following steps:

[0054] S201, Construct the ResShift conditional diffusion baseline model.

[0055] Referring to Figure 4, this embodiment uses a conditional diffusion model based on residual displacement (ResShift) as its basic architecture. The diffusion time step is set to... , To progress over time A monotonically increasing coefficient sequence satisfies and .

[0056] During the forward process of training, at any time intermediate state The conditional distribution is defined by the ResShift conditional forward distribution formula:

[0057] ;

[0058] in, Indicates a Gaussian normal distribution. It is the identity matrix. To control the standard deviation hyperparameter of diffusion randomness, Indicates from high-quality reference image Degrading images to low quality The deterministic displacement vector.

[0059] Based on the above distribution, the intermediate state Generated using the ResShift conditional forward sampling formula:

[0060] ;

[0061] in, This represents the standard Gaussian white noise used in the sampling.

[0062] S202, construct a frequency domain prior encoder and extract multi-scale high-frequency energy.

[0063] Referring to Figure 3, the frequency domain prior encoder (FPE) receives a high-quality reference image. (Identified as HR in the figure), a Discrete Wavelet Transform (DWT) is performed on it. The DWT uses Haar wavelets or Daubechies wavelets as basis functions and employs periodic or symmetrical padding to process the image boundaries. The image is decomposed into low-frequency approximate subbands. And a collection of high-frequency subbands containing horizontal, vertical, and diagonal details. .

[0064] For images Level 1 pyramid decomposition, for the 1st level Level decomposition ( The spatial high-frequency intensity at this scale is extracted using the single-layer wavelet high-frequency energy calculation formula:

[0065] ;

[0066] in, Indicates the first High-frequency energy diagram of the level, , , They represent the first High-frequency coefficient diagram of level, This represents the square of each pixel. This indicates the square root of each pixel.

[0067] After obtaining the energy maps at various scales, a global frequency domain guiding map is generated using the multi-scale wavelet energy fusion formula:

[0068] ;

[0069] in, This is the fused multi-scale high-frequency energy map. This means upsampling the feature map to the original image size using bilinear interpolation. . This is the summation symbol. The weighting coefficients for each scale are determined by the multi-scale fusion weight definition formula:

[0070] ;

[0071] This weight setting applies to shallow high-resolution features ( Smaller values ​​are assigned higher weights to preserve micron and nanometer-level edge detail information.

[0072] S203 executes a frequency-domain-aware noise modulation forward process.

[0073] Using the product generated in step S202 Standard noise Modulation is performed to generate a spatially non-uniform noise distribution.

[0074] The fused energy map is mean-removed, and the energy perturbation is calculated using the high-frequency energy-centered modulation term formula.

[0075] ;

[0076] in, That is Comprehensive energy diagram , This indicates the calculation of the spatial pixel mean on the image plane.

[0077] Define the modulation strength using a time-dependent modulation strength scheduling formula:

[0078] ;

[0079] in, For a moment modulation coefficients, For the initial intensity hyperparameter, The decay exponential hyperparameter is used to control the decay rate of the frequency-domain guided diffusion process.

[0080] Combining the energy perturbation and modulation intensity, the frequency domain sensing noise modulation formula is applied:

[0081] ;

[0082] in, This is the modulated intermediate noise. Due to the introduction of... The statistical characteristics of the noise have been altered, requiring renormalization using the modulation noise channel-level variance normalization formula:

[0083] ;

[0084] in, Modulation noise for the final injection diffusion process, This indicates the calculation of the variance of the tensor along the channel dimension. This represents the channel mean. In the sampling formula of step S201, the following is used: Replace standard noise , generate training samples.

[0085] S204, Denoising Network Optimization and Loss Function Definition.

[0086] Building a noise reduction network The network employs a U-Net architecture with residual connections and a self-attention mechanism. The noisy state generated in step S203... Time step embedding vector and low-quality condition images Input the data into the denoising network and calculate the loss using the general training error formula for denoising networks:

[0087] ;

[0088] Among them, in the formula In this step, it refers to the modulation noise that actually participates in the forward process. , Describing the L2 norm, This represents the expected value in terms of data distribution and time steps. Network parameters. Minimize the error formula using the stochastic gradient descent algorithm. Update.

[0089] Referring to Figure 4, this embodiment of the invention further proposes an inference and restoration process based on wavelet domain consistency gradient. In the inverse denoising inference stage, the system utilizes the trained denoising network as the image prior and calculates the consistency gradient by combining the structural features of the observed image in the wavelet domain, thereby correcting the sampling trajectory in real time. The specific implementation process includes the following steps:

[0090] S301, Inverse denoising sampling initialization and conditional input.

[0091] The reasoning and reconstruction process is executed from time step Iterate in reverse order to 0. Set the initial state. Sampled from low-quality observation images Centered on, with The variance follows a Gaussian distribution, thus establishing the initial approximate outline of the structure. Low-quality observation images are then used. As a global condition vector, at each time step Input to denoising network The network outputs the current noisy state based on this. The noise prediction value is given below.

[0092] S302, Estimate the clean image based on the current state.

[0093] Using the output of the denoising network, the potential clean image at the current time is estimated through inverse reparameterization mapping. Apply the clean image estimation formula based on the current state:

[0094] ;

[0095] in, The displacement coefficient for the current time step. These are preset noise scale parameters. This represents the noise component output by the denoising network. This step yields... This represents the model's best single-point estimate of the original high-quality image at the current noise level.

[0096] S303, construct wavelet domain consistency metric and gradient calculation.

[0097] To ensure that the generated image preserves both the low-frequency topology of the observed data and high-frequency details, the sampling score function is decomposed using Bayes' theorem. The posterior distribution score function decomposition formula is applied:

[0098] ;

[0099] The second term on the right-hand side of the equation is the data likelihood term. This step calculates the gradient of this term by constructing an explicit loss function. The wavelet domain negative log-likelihood loss function formula is applied:

[0100] ;

[0101] in, Indicates performing the first step on the image. Extracted after hierarchical decomposition Subband feature map These represent low-frequency approximations and horizontal, vertical, and diagonal high-frequency details, respectively. Set the importance weights for subbands. To ensure consistency with the low-frequency structure. This is the noise variance estimate for the corresponding frequency band.

[0102] Based on the above loss, its value relative to the estimated image is calculated. The gradient. Applying the wavelet domain likelihood gradient backpropagation formula (to the estimated image):

[0103] ;

[0104] in, This represents the Inverse Discrete Wavelet Transform (IDWT). The gradient scalar coefficients combine the weights and the normalization factor.

[0105] S304, Gradient approximation propagation and time-dependent correction.

[0106] Using the chain rule to... Gradient mapping back to the current sampling state Applying the wavelet domain likelihood gradient approximation propagation (to the current state) formula:

[0107] ;

[0108] To balance structural consistency guidance with the freedom of texture generation, a guidance strategy that decays over time is introduced. The time-dependent strength formula for frequency-domain guidance is applied:

[0109] ;

[0110] in, The dynamic guide step size for the current moment. Based on the lead rate, The attenuation control index is used. This scheduling strategy applies strong guidance in the early stages of the diffusion process to correct the contour, and weakens the guidance in the later stages to avoid smoothing out the generated fine textures.

[0111] S305, Execution status update.

[0112] The calculated and weighted physical consistency gradient is injected into the original prediction of the denoising network. The noise prediction formula after applying frequency domain consistency correction is as follows:

[0113] ;

[0114] in, This involves noise correction incorporating data consistency constraints. Based on a gradient-guided mechanism, by subtracting a weighted wavelet domain consistency gradient term from the original prediction noise, components in the noise distribution inconsistent with the observed data structure are suppressed, thereby improving the handling of potentially noisy states during the inverse iteration process. The trajectory is constrained and corrected.

[0115] Finally, the corrected noise is substituted into the inverse iterative equation of ResShift to calculate the state at the previous time step. , until the entire sampling process is completed.

[0116] S401, Experimental Setup and Evaluation Index System.

[0117] Regarding dataset preparation, this invention constructs a dedicated evaluation dataset based on SEM images accumulated over a long period in the laboratory. First, the BRISQUE no-reference image quality assessment metric is used to score the quality of the original images. BRISQUE evaluates the degree of image distortion by analyzing the statistical characteristics of natural scenes. Based on the BRISQUE scores, the top 10,000 images are selected as a high-quality reference image pool. Next, according to the two-stage degradation pipeline S101 to S103, semantically aligned high-quality and low-quality image pairs are generated. Two sub-datasets are constructed:

[0118] SEM-D Dataset: Designed for image denoising and artifact removal tasks. It contains 3000 pairs of images with a resolution of 256×256 pixels. The degradation pipeline introduces degradation factors such as blurring, noise, orientation artifacts, and contrast shift sequentially while maintaining the original resolution, simulating the image quality degradation during actual electron microscope acquisition.

[0119] SEM-SR dataset: Designed for 4x super-resolution reconstruction tasks. It contains 3000 pairs of images; the low-resolution input image size is 64×64 pixels, and the corresponding high-resolution reference image size is 256×256 pixels. The degradation pipeline introduces a 4x downsampling operation based on spatial integration in the second stage, building upon the first-stage random degradation.

[0120] Both datasets were divided into training, validation, and test sets in an 8:1:1 ratio, and all subsequent quantitative evaluation results were obtained on the test set.

[0121] Regarding the experimental environment, the method of this invention is implemented based on the PyTorch framework. All experiments were conducted on a workstation equipped with two NVIDIA GeForce RTX 4090 GPUs (24GB of video memory per card), with Ubuntu 22.04 operating system and CUDA version 12.1.

[0122] In terms of training strategy, a distributed data parallelism (DDP) strategy was adopted during the training phase, using the AdamW optimizer (an optimizer that combines the adaptive learning rate adjustment and weight decay mechanism of the Adam algorithm), with an initial learning rate set to 2×10. −4 The weight decay coefficient is 1×10 −4 A cosine annealing strategy was used for learning rate scheduling (this strategy gradually decreases the learning rate from high to low during training for smooth convergence). The batch size per GPU was set to 4 (total batch size is 8), and training lasted for 200 epochs. Data augmentation strategies included random horizontal / vertical flipping and 90° rotation to increase the model's generalization ability.

[0123] Regarding evaluation metrics, the following three image quality evaluation metrics are used:

[0124] Peak Signal-to-Noise Ratio (PSNR): This is an objective evaluation metric based on pixel error, reflecting the degree of image damage and distortion of the original image. A higher value indicates better image quality.

[0125] Structural Similarity Index (SSIM): Measures the similarity of image structure, brightness, and contrast. The higher the value, the higher the similarity between the two images in human visual perception.

[0126] Perceptual Image Quality Metric (LPIPS): Based on deep learning feature extraction, this perceptual loss better reflects the human eye's perception of differences in images; the lower the value, the better the image quality.

[0127] S402, a quantitative comparative analysis with existing mainstream methods.

[0128] The method of this invention was compared with existing mainstream denoising or image restoration methods on the SEM-D dataset, and the results are shown in Table 1.

[0129] Table 1. Comparison of quantitative results on the SEM-D dataset

[0130]

[0131] Conclusion: As shown in Table 1, the method of this invention outperforms other comparative methods in all three metrics: PSNR, SSIM, and LPIPS, indicating better performance in image denoising and artifact removal. This result verifies that the proposed wavelet frequency domain modulation mechanism can effectively enhance the diffusion model's ability to recover high-frequency textures and edge details in SEM images.

[0132] The quantitative comparison results of the method of the present invention on the SEM-SR dataset are shown in Table 2.

[0133] Table 2. Quantitative comparisons on the SEM-SR dataset

[0134]

[0135] Conclusion: The method of this invention also performs excellently in super-resolution reconstruction tasks. This demonstrates that the two-stage degradation pipeline designed for SEM imaging characteristics can more accurately simulate the real degradation distribution and effectively alleviate the domain difference problem between training and test data. Simultaneously, the effectiveness of wavelet domain consistency constraints is verified: by explicitly aligning the distribution of the reconstruction results and the observed images across multiple high-frequency subbands during the inference stage, the method of this invention can maintain or even improve reconstruction quality while significantly reducing the number of sampling steps.

[0136] S403, Subjective assessment of the visual quality of the restored image.

[0137] Referring to Figure 5, which shows a restoration example of different methods on the SEM-D dataset, from left to right: low-quality input image (LR), BM3D restoration result, DNCNN restoration result, SwinIR restoration result, restoration result of the method of this invention (Ours), and high-quality lossless image (GT).

[0138] Visual comparison reveals that low-quality input images (LR) exhibit significant noise and blurring. While BM3D and DNCNN suppress noise to some extent, they result in considerable loss of image detail and blurred textures. SwinIR recovers more detail, but still exhibits smearing or artifacts in high-frequency texture areas. In contrast, the image restored by the method of this invention (Ours) most closely resembles the original high-quality image (GT) in terms of overall sharpness and detail, especially in microstructure and edge textures. Noise suppression is significant while details are well preserved, and the visual quality evaluation results are consistent with the quantitative indicators.

[0139] S404, Ablation Experimental Analysis of Key Modules.

[0140] To verify the effectiveness of the proposed Frequency Domain Prior Encoder (FPE) and Wavelet Domain Consistent Gradient Guided (WCG) modules, a series of ablation experiments were conducted. Specifically, changes in model performance were observed by removing or replacing these core modules. The experiments were performed on the SEM-D test set, and the specific results are shown in Table 3.

[0141] Table 3. Ablation Experiment Results of Key Modules

[0142]

[0143] Conclusion: Removing the Frequency Prior Encoder (FPE) significantly degrades model performance. This indicates that explicitly modulating noise during training to guide the denoising network to focus on high-frequency textures is effective. Removing the Wavelet Domain Consistency Gradient (WCG) module also reduces performance, demonstrating that introducing data consistency constraints during inference and correcting the sampling path in the wavelet domain are crucial for generating high-quality images consistent with the observed data. The model achieves optimal performance when both the FPE and WCG modules are present. This verifies that the combination of these two innovative modules can synergistically improve image restoration results.

Claims

1. A scanning electron microscope image restoration method based on a wavelet frequency domain modulated diffusion model, characterized in that, Includes the following steps: S1. Construct a high-quality reference image pool based on real scanning electron microscope (SEM) images. Process the high-quality reference image pool using a two-stage degradation pipeline that includes random optical path perturbation and fixed circuit acquisition limitations to generate corresponding low-quality images. This constructs a specific domain training dataset that includes semantic alignment, pairing images from the high-quality reference image pool with the low-quality images. S2. Input the specific domain training dataset constructed in step S1 into a pre-defined conditional diffusion network. During the forward pass of training, use a frequency domain prior encoder to extract multi-scale high-frequency energy features to modulate the noise distribution. Optimize the network parameters by minimizing the denoising prediction error to obtain a pre-trained wavelet frequency domain modulated diffusion model. S3. Obtain the target low-quality degradation image to be processed as a conditional input to the step. The pre-trained wavelet frequency domain modulated diffusion model obtained in S2 samples from a standard Gaussian distribution to initialize the potential noisy state at the current time. In S4, the pre-trained wavelet frequency domain modulated diffusion model is used to perform inverse denoising iteration on the potential noisy state initialized in step S3. In each time step, the wavelet domain consistency gradient between the estimated clean image and the target low-quality degraded image obtained based on the potential noisy state is calculated. The wavelet domain consistency gradient is injected into the sampling process and the noise prediction result is corrected to update the potential noisy state. In S5, the inverse denoising iteration is repeated until a preset termination time step is reached. The potential noisy state finally updated in step S4 is mapped back to the pixel space, and the corresponding restored high-quality scanning electron microscope image is output.

2. The scanning electron microscope image restoration method based on a wavelet frequency domain modulated diffusion model according to claim 1, characterized in that, In step S1, the process of processing the high-quality reference image pool using a two-stage degradation pipeline that includes random optical path perturbation and fixed circuit acquisition limitations specifically involves: configuring a system-preset first degradation operator set containing blur components, noise components, and artifact components; using a random out-of-order strategy to superimpose the first degradation operator set onto the images in the high-quality reference image pool to generate an intermediate state image containing physical optical path features; wherein, the blur component is applied through a convolution operation, the noise component simulates statistical noise in the imaging process, and the artifact component models the imaging characteristics of the scanning electron microscope; the circuit characteristics limitation in the process of converting the analog signal into a digital signal by the detector; and performing deterministic signal link processing on the intermediate state image, including applying dynamic range limitation to the intermediate state image, superimposing readout noise, and performing resizing operations according to a preset target resolution specification to generate the final low-quality image.

3. The scanning electron microscope image restoration method based on a wavelet frequency domain modulated diffusion model according to claim 2, characterized in that, The artifact components include: artifacts generated by simulating electron beam damage by applying a local low-grayscale mask; artifacts generated by simulating the charging effect by superimposing a high-brightness saturation region; and artifacts generated by simulating astigmatism using a non-rotationally symmetric elliptical Gaussian kernel.

4. The scanning electron microscope image restoration method based on a wavelet frequency domain modulated diffusion model according to claim 1, characterized in that, In step S2, the extraction of multi-scale high-frequency energy features using a frequency domain prior encoder specifically involves: performing discrete wavelet transform on the high-quality reference image in the specific domain training dataset to decompose it into a set of low-frequency approximate subbands and a set of high-frequency subbands containing horizontal, vertical, and diagonal details; based on the set of high-frequency subbands, performing multi-level pyramid decomposition on the image, and extracting the spatial high-frequency intensity at each decomposition scale using a single-layer wavelet high-frequency energy calculation formula; inputting the extracted spatial high-frequency intensity into a multi-scale wavelet energy fusion formula to generate a global frequency domain guiding map; the global frequency domain guiding map serves as the multi-scale high-frequency energy feature to preserve micron and nanometer-level edge detail information.

5. The scanning electron microscope image restoration method based on a wavelet frequency domain modulated diffusion model according to claim 4, characterized in that, In step S2, the modulation of the noise distribution specifically involves: performing mean-reduction processing on the global frequency domain guide map and calculating the energy perturbation; combining the energy perturbation with a preset time-dependent modulation intensity, processing the standard noise using a frequency domain-aware noise modulation formula to generate modulated intermediate noise; The intermediate noise is renormalized using the modulation noise channel-level variance normalization formula to obtain the modulation noise of the final injection diffusion process, and the modulation noise is used to generate training samples.

6. The scanning electron microscope image restoration method based on a wavelet frequency domain modulated diffusion model according to claim 1, characterized in that, The preset conditional diffusion network adopts a conditional diffusion model architecture based on residual displacement: the conditional distribution of the intermediate state at any time is defined by the ResShift conditional forward distribution formula; the specific intermediate state is generated by the ResShift conditional forward sampling formula using the modulation noise after modulation by the frequency domain prior encoder; the intermediate state is input into the network to calculate the predicted value, and the network parameters are updated by minimizing the general training error formula of the denoising network through the stochastic gradient descent algorithm.

7. The scanning electron microscope image restoration method based on a wavelet frequency domain modulated diffusion model according to claim 1, characterized in that, In step S4, the calculation of the estimated clean image based on the potential noisy state specifically involves: using the pre-trained wavelet frequency domain modulated diffusion model to receive the potential noisy state and outputting the noise prediction value in the current state; Using the noise prediction value, the estimated clean image at the current time is calculated by inverse reparameterization mapping and a clean image estimation formula based on the current state; the estimated clean image represents a single-point estimate of the original high-quality image by the model at the current noise level.

8. The scanning electron microscope image restoration method based on a wavelet frequency domain modulated diffusion model according to claim 7, characterized in that, In step S4, calculating the wavelet domain consistency gradient between the estimated clean image and the target low-quality degraded image specifically involves: acquiring the estimated clean image and the target low-quality degraded image, performing discrete wavelet transform on both to generate corresponding sub-band feature maps; constructing a wavelet domain consistency measure based on the generated sub-band feature maps using the wavelet domain negative log-likelihood loss function formula; calculating the differences between the sub-band feature maps at multiple scales using the wavelet domain consistency measure, and setting sub-band importance weights to force structural consistency; Based on the wavelet domain consistency metric, the gradient relative to the estimated clean image is calculated to obtain the wavelet domain consistency gradient.

9. The scanning electron microscope image restoration method based on a wavelet frequency domain modulated diffusion model according to claim 8, characterized in that, In step S4, injecting the wavelet domain consistency gradient into the sampling process and correcting the noise prediction result specifically involves: using the chain rule to map the wavelet domain consistency gradient from the estimated clean image back to the current sampling state; introducing a time-decaying guidance strategy and calculating the dynamic guidance step size at the current moment using the time-dependent strength formula of frequency domain guidance; weighting and fusing the wavelet domain consistency gradient with the dynamic guidance step size and injecting it into the original prediction of the denoising network; calculating the corrected noise using the noise prediction formula after frequency domain consistency correction; and suppressing components in the noise distribution that are inconsistent with the observed data structure by subtracting the weighted wavelet domain consistency gradient from the original predicted noise, thereby completing the generation of the corrected noise.

10. The scanning electron microscope image restoration method based on a wavelet frequency domain modulated diffusion model according to claim 9, characterized in that, The process of updating the potential noisy state specifically involves using the corrected noise to solve for the potential noisy state at the previous moment through the inverse iterative equation of ResShift, until the sampling process is completed.

Images