Method and device for denoising scanning electron microscope image and storage medium

Abstract

The invention discloses a method and equipment for denoising a scanning electron microscope image, and a storage medium. The method comprises the following steps: acquiring a to-be-denoised scanning electron microscope image; performing local statistical characteristic analysis on the to-be-denoised scanning electron microscope image to obtain a minimum characteristic value to represent a first noise level of the scanning electron microscope image; performing linear mapping on the first noise level to obtain a second noise level, the second noise level corresponding to the denoising capability of a pre-training denoising diffusion model; and performing reasoning denoising on the scanning electron microscope image to be denoised by using a pre-training denoising diffusion model corresponding to the second noise level to obtain a denoised scanning electron microscope image. According to the scheme of the invention, the denoising model matched with the noise level of the image can be selected for denoising, and the denoising precision and flexibility can be improved.

Description

Technical Field

[0001] This application generally relates to the field of image processing technology. More specifically, this application relates to a method, an electronic device, and a computer-readable storage medium for denoising scanning electron microscope images. Background Art

[0002] In the processing of scanning electron microscope images, the noise can generally be assumed to be Gaussian additive white noise. However, the intensity of the noise is affected by multiple factors such as imaging equipment, environmental conditions, and sample characteristics, making it difficult to accurately determine in advance. Most of the existing denoising methods rely on the preset of noise type and intensity, or use models with fixed parameters, which often perform poorly in the case of changing noise intensity or complex noise types.

[0003] As a rapidly developing denoising technology in recent years, deep learning methods perform excellently in dealing with specific noise types. However, when a single deep neural network processes multiple noise levels, it faces the bottleneck of degraded generalization performance. At different noise levels, it is difficult for the network to balance all cases in terms of denoising effect, especially when the noise level is complex or the noise type is diverse, the denoising effect is poor, which easily leads to the loss of image details and even serious image distortion. In this case, the denoising ability of a single model is difficult to adapt to the complex noise characteristics of scanning electron microscope images and lacks the flexibility to handle multiple noises. In addition, although hyperspectral image denoising methods can process complex images with multiple bands, due to the diversity of noise types and the high-dimensional characteristics of hyperspectral data, the accuracy of noise estimation is greatly reduced, resulting in an unsatisfactory denoising effect. For hyperspectral images, multiple bands are simultaneously affected by noise interference, making it difficult for existing methods to accurately distinguish noise from the effective information of the image, thereby affecting the denoising effect.

[0004] In view of this, there is an urgent need to provide a solution for denoising scanning electron microscope images to improve the denoising accuracy and flexibility. Summary of the Invention

[0005] In order to solve at least one or more of the above-mentioned technical problems, this application proposes a solution for denoising scanning electron microscope images in the following aspects.

[0006] In a first aspect, the present application provides a method for denoising a scanning electron microscope image, including: obtaining a scanning electron microscope image to be denoised; performing local statistical characteristic analysis on the scanning electron microscope image to be denoised to obtain a minimum eigenvalue to represent a first noise level of the scanning electron microscope image; performing a linear mapping on the first noise level to obtain a second noise level, where the second noise level corresponds to the denoising ability of a pre-trained denoising diffusion model; using the pre-trained denoising diffusion model corresponding to the second noise level to perform inference denoising on the scanning electron microscope image to be denoised to obtain a denoised scanning electron microscope image.

[0007] In some embodiments, performing local statistical characteristic analysis on the scanning electron microscope image to be denoised to obtain a minimum eigenvalue to represent the first noise level of the scanning electron microscope image includes: dividing the scanning electron microscope image to be denoised to obtain a plurality of image blocks with the same dimension; obtaining a recombined two-dimensional matrix based on the plurality of image blocks with the same dimension; calculating a covariance matrix of the recombined two-dimensional matrix, and performing eigenvalue decomposition on the covariance matrix to obtain a plurality of eigenvalues; determining the minimum eigenvalue among the plurality of eigenvalues, and determining the non-negative square root of the minimum eigenvalue as the first noise level.

[0008] In some embodiments, obtaining a recombined two-dimensional matrix based on the plurality of image blocks with the same dimension includes: straightening the matrices of the plurality of image blocks with the same dimension to obtain a plurality of one-dimensional vectors, where straightening the matrix of the image block is to concatenate the rows in the matrix in ascending order of row numbers; performing matrix recombination on the plurality of one-dimensional vectors to obtain a recombined two-dimensional matrix, and one row in the recombined two-dimensional matrix corresponds to one one-dimensional vector.

[0009] In some embodiments, calculating the covariance matrix of the recombined two-dimensional matrix includes: calculating an average vector of the plurality of one-dimensional vectors; calculating the covariance matrix based on the average vector and the recombined two-dimensional matrix.

[0010] In some embodiments, calculating the covariance matrix based on the average vector and the recombined two-dimensional matrix can be implemented by the following formula:

[0011]

[0012] where, ∑ is the covariance matrix, s is the number of one-dimensional vectors, μ is the average vector, and x t is the t-th one-dimensional vector.

[0013] In some embodiments, linearly mapping the first noise level to obtain a second noise level includes: determining whether the first noise level is less than a preset threshold; in the case where the first noise level is less than the preset threshold, linearly mapping the first noise level using a preset linear relationship to obtain the second noise level; in the case where the first noise level is greater than or equal to the preset threshold, determining the second noise level as 1.

[0014] In some embodiments, the preset linear relationship can be expressed by the following formula:

[0015]

[0016] where σ1 is the first noise level and σ2 is the second noise level.

[0017] In some embodiments, the method further includes the following operations to fine-tune the pre-trained denoising diffusion model: capturing multiple sets of scanning electron microscope images with different noise level distributions under different shooting conditions, where the shooting conditions include dwell time, acceleration voltage, and / or number of image averages; annotating the corresponding noise level for each scanning electron microscope image in the multiple sets of scanning electron microscope images to obtain multiple sets of annotated scanning electron microscope images; inputting the multiple sets of annotated scanning electron microscope images as training data into the pre-trained denoising diffusion model to fine-tune it.

[0018] In a second aspect, the present application provides an electronic device, including: a processor; and a memory that stores program instructions for denoising scanning electron microscope images, and when the program instructions are executed by the processor, the methods and their multiple embodiments described in the foregoing first aspect are implemented.

[0019] In a third aspect, the present application provides a computer-readable storage medium, on which program instructions for denoising scanning electron microscope images are stored, and when the program instructions are executed by a processor, the methods and their multiple embodiments described in the foregoing first aspect are implemented.

[0020] Through the scheme for denoising scanning electron microscope images provided above, after analyzing the local statistical characteristics of the scanning electron microscope image to be denoised, the minimum eigenvalue can be obtained to represent the first noise level of the image, without the need to know the noise information of the image in advance, realizing adaptive noise level estimation. Then, through linear mapping, the first noise level can be mapped to the second noise level, which corresponds to the denoising ability of the pre-trained denoising diffusion model. Therefore, using the pre-trained denoising diffusion model with the denoising ability corresponding to the second noise level to perform inference denoising on the scanning electron microscope image to be denoised can improve the accuracy of denoising model selection, ensure that the denoising effect matches the actual noise level of the image, and improve the denoising accuracy. In addition, using the scheme of this application, a denoising model corresponding to the noise level of the image can be used for denoising, which not only overcomes the problem of poor denoising effect of a single model under complex noise conditions, but also improves the denoising flexibility, and can ensure that images with different noise levels can obtain high-quality denoising effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] By referring to the accompanying drawings and reading the following detailed description, the above and other objects, features, and advantages of the exemplary embodiments of this application will become readily understood. In the drawings, several embodiments of this application are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein:

[0022] Figure 1 An exemplary flowchart of a method for denoising scanning electron microscope images according to an embodiment of this application is shown;

[0023] Figure 2 An exemplary schematic diagram of linear mapping according to an embodiment of this application is shown;

[0024] Figure 3 An exemplary flowchart of a method for analyzing the local statistical characteristics of a scanning electron microscope image to be denoised according to an embodiment of this application is shown;

[0025] Figure 4 An exemplary denoising performance of different denoising models according to an embodiment of this application is shown;

[0026] Figure 5 An exemplary structural block diagram of an electronic device according to an embodiment of this application is shown. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0027] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative efforts shall fall within the protection scope of the present application.

[0028] It should be understood that the terms "including" and "comprising" used in the specification and claims of the present application indicate the presence of the described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or their combinations.

[0029] It should also be understood that the terms used in the specification of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the specification and claims of the present application, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include the plural forms. It should be further understood that the term "and / or" used in the specification and claims of the present application refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0030] As used in this specification and the claims, the term "if" can be interpreted as "when", "once", "in response to determining", or "in response to detecting" depending on the context. Similarly, the phrase "if determined" or "if [the described condition or event] is detected" can be interpreted as meaning "once determined", "in response to determining", "once [the described condition or event] is detected", or "in response to detecting [the described condition or event]" depending on the context.

[0031] For the sake of easy understanding, before the method for denoising scanning electron microscope images in the present application is described in detail, the content related to the denoising of scanning electron microscope images will be introduced first.

[0032] Traditional denoising methods usually rely on the preset of noise models (such as Gaussian noise, Poisson noise, etc.), so these methods can work effectively only when the noise type and intensity are known. However, in many practical applications, the noise sources are complex and difficult to model. Therefore, blind denoising technology analyzes the statistical characteristics of images, automatically estimates the noise level and performs denoising, solving the challenge that the noise type is unknown or the intensity cannot be accurately estimated. Blind Denoising is an image or signal processing technology that can automatically remove noise without prior knowledge of the noise type, distribution or intensity. It can adaptively estimate the noise intensity and automatically adjust the denoising strategy according to the estimation result, so as to achieve excellent denoising effects under different noise conditions.

[0033] The technical difficulty of blind denoising lies in: how to effectively estimate the noise intensity (also called the noise level) without preset noise information, and at the same time keep the useful information in the image or signal during the denoising process to prevent the loss of important details. The core of denoising is to balance the relationship between noise removal and detail preservation. Especially in images with low signal-to-noise ratio, the noise and image detail information may be intertwined, which makes accurate denoising more challenging.

[0034] The processing of scanning electron microscope images (also simply called SEM images) is a typical application scenario of blind denoising technology. SEM images are usually disturbed by various noises, such as Gaussian noise, shot noise and electronic signal fluctuations. Since these noises may exist simultaneously and have a complex distribution, but traditional denoising methods rely heavily on the noise type, so in the face of complex or unknown noise levels, traditional denoising methods perform poorly. In addition, in the actual processing of SEM images, sometimes the noise type is known (for example, Gaussian additive white noise), but the noise intensity or level (such as the standard deviation of the noise) is often unknown and affected by various factors, including imaging equipment, experimental conditions and sample characteristics.

[0035] Based on this, in view of the deficiencies of the existing SEM image processing methods in complex noise environments, this application combines noise level estimation and deep learning models to implement a method for pseudo-blind denoising of SEM images. This method can automatically estimate the noise level of SEM images through local statistical characteristic analysis, and can select the corresponding denoising model for denoising according to the noise level. This method can directly denoise SEM images with different noise characteristics without presetting parameters. Especially in images with high noise and rich details, the method of this application significantly enhances the noise suppression effect and achieves a balance between detail preservation and noise removal.

[0036] The following will describe the specific implementation manners of this application in detail with reference to the accompanying drawings.

[0037] Figure 1FIG. 0 shows an exemplary flowchart of method 100 for denoising a scanning electron microscope image according to an embodiment of the present application. It can be understood that method 100 can be executed by any suitable device with data processing capabilities, such as, but not limited to, a terminal device, a processor, a server, etc.

[0038] As Figure 1 shown, at step S101, method 100 can obtain a scanning electron microscope image to be denoised. Then, at step S102, method 100 can perform a local statistical feature analysis on the scanning electron microscope image to be denoised to obtain a minimum eigenvalue to represent the first noise level of the scanning electron microscope image to be denoised. Next, at step S103, method 100 can perform a linear mapping on the first noise level to obtain a second noise level. Here, the second noise level corresponds to the denoising ability of the pre-trained denoising diffusion model. Finally, at step S104, method 100 can use the pre-trained denoising diffusion model corresponding to the second noise level to perform inference denoising on the scanning electron microscope image to be denoised to obtain a denoised scanning electron microscope image. Here, the pre-trained denoising diffusion model corresponding to the second noise level refers to a pre-trained denoising diffusion model with a denoising ability corresponding to the second noise level.

[0039] In this embodiment, the obtaining of the first noise level mainly depends on the analysis of the matrix form of the scanning electron microscope image. In actual operation, it is assumed that the noise in the scanning electron microscope image can be approximated by a statistical model, usually Gaussian additive white noise. Thus, the statistical features of the image obtained through matrix operations can be used to reveal its noise level to estimate the noise level of the scanning electron microscope image. For ease of understanding, how to obtain the first noise level will be described in detail later in combination with Figure 3 method 300 for performing a local statistical feature analysis on the scanning electron microscope image to be denoised.

[0040] Since different denoising models usually have different denoising abilities, the denoising ability can be represented by the noise level that the denoising model can effectively remove. In addition, the value range of the denoising ability of the denoising model (for distinction, it can be called the first level range) is not the same as the value range of the noise level of the image (for distinction, it can be called the second level range). Therefore, a suitable denoising model cannot be directly matched according to the first noise level of the image. Based on this, after obtaining the first noise level, a noise level mapper can be introduced to adjust the first noise level based on past denoising experience, that is, map the first noise level to the second noise level in the second level range. Specifically, after performing a local statistical feature analysis on the scanning electron microscope image, the first noise level σ e of the image will be obtained. To select a suitable denoising model, the following linear relationship can be used to perform a linear mapping on the first noise level:

[0041] σ m = k·σ e + b(1)

[0042] where σ m is the second noise level, and σ e is the first noise level, and k and b are linear parameters. It can be understood that those skilled in the art can select specific values of k and b according to actual needs, as long as the values can map the first noise level to the value range of the noise reduction ability of the noise reduction model, and this application does not make specific limitations thereto. Additionally or optionally, the specific values of the linear parameters k and b can be determined through experiments or data analysis.

[0043] Furthermore, in order to select a more suitable noise reduction model according to the first noise level, when linearly mapping the first noise level, different mapping methods can be adopted according to the value of the first noise level. Specifically, it can be first determined whether the first noise level is less than a preset threshold. When the first noise level is less than the preset threshold, a preset linear relationship can be used to linearly map the first noise level to obtain the second noise level. Conversely, when the first noise level is greater than or equal to the preset threshold, the second noise level can be determined to be 1.

[0044] It can be understood that those skilled in the art can select specific values of the preset threshold and the linear coefficient in the preset linear relationship according to actual needs, and this application does not make specific limitations thereto. In one implementation scenario, the preset threshold can be 35, and the preset linear relationship can be expressed by the following formula:

[0045]

[0046] where σ1 is the first noise level and σ2 is the second noise level.

[0047] Accordingly, for the case where the first noise level is greater than or equal to the preset threshold, the following formula can be used for mapping:

[0048] σ2 = 1, σ1 ≥ 35(3)

[0049] Here, reference can also be made to Figure 2 to understand the process of linearly mapping the first noise level to the second noise level. As Figure 2As shown, when the first noise level is greater than or equal to 35, the first noise level is linearly mapped to a specific second noise level (i.e., 1). When the first noise level is greater than 0 and less than 35, the first noise level is linearly mapped to different second noise levels from 5 to 1, and the smaller the first noise level, the larger the linearly mapped second noise level. Thus, appropriate amplification of the noise is achieved so that the denoising model can more clearly identify and process the noise characteristics, effectively improving the denoising accuracy and enabling the denoising model to still maintain detailed information when facing a high noise level.

[0050] After obtaining the second noise level, the pre-trained denoising diffusion model corresponding to the second noise level can be used to denoise the scanning electron microscope image to be denoised. The diffusion model is a type of generative model that can achieve denoising by learning the transformation process of the image from noise to a clean image. The diffusion model adds noise to the image step by step and trains the model to reverse this process, that is, to recover from noise to a clean image. In the embodiments of this disclosure, the pre-trained denoising diffusion model can adopt the Denoising Diffusion Null-Space Model (DDNM) and use the pre-trained weights of OpenAI's open-source text-to-image (DALL·E). This model can be directly used for image denoising without additional training. Since the model has been pre-trained on a large-scale image dataset, it has strong generalization ability and can process images with different noise levels.

[0051] In the embodiments of this disclosure, in order to improve the performance of the pre-trained denoising diffusion model in the scanning electron microscope image denoising task, it can also be fine-tuned. Specifically, the following operations can be performed to fine-tune the pre-trained denoising diffusion model: capture multiple sets of scanning electron microscope images with different noise level distributions under different shooting conditions, where the shooting conditions include dwell time, acceleration voltage, and / or image averaging times; annotate the corresponding noise level for each scanning electron microscope image in the multiple sets of scanning electron microscope images to obtain multiple annotated sets of scanning electron microscope images; and input the multiple annotated sets of scanning electron microscope images as training data into the pre-trained denoising diffusion model to fine-tune it.

[0052] During the process of taking scanning electron microscope (SEM) images, the noise level of SEM images can be controlled by adjusting the dwell time, acceleration voltage, and image averaging times individually or in combination. The dwell time determines the residence time of the electron beam on each pixel. The longer the time, the more electron signals are accumulated on each pixel, the higher the signal-to-noise ratio (SNR), and the lower the noise. The acceleration voltage affects the energy and penetration depth of the electron beam. At low voltages, the electron beam is more concentrated on the sample surface, the signal is weaker but the noise may be higher; at high voltages, the signal is stronger, but the secondary electron yield may decrease and the noise distribution may change. The image averaging times represent the number of times of scanning the same area. The noise of each scan is randomly distributed and cancels each other out after averaging, thus effectively reducing random noise (such as electron counting statistical noise).

[0053] In the embodiments of this disclosure, the noise level of each SEM image in a plurality of SEM image sets can be obtained by the method described in the foregoing steps S102 and S103, that is, first perform local statistical feature analysis on each SEM image to obtain the minimum eigenvalue to represent the first noise level of the SEM image to be denoised, and then perform linear mapping on the first noise level to obtain the second noise level as the noise level of the SEM image.

[0054] The fine-tuned pre-trained denoising diffusion model can achieve denoising of SEM images with different noise levels. It directly associates the noise level with network parameters (such as convolution kernels, activation functions, loss weights, etc.) through conditional embeddings and dynamic weights, thereby realizing an explicit mapping of "noise level → model parameters". Essentially, the fine-tuned pre-trained denoising diffusion model is a dynamically parameterized unified model, that is, the main structure of the model remains unchanged, and all noise levels share the same network architecture, but are generated through conditional embeddings and dynamic weights, enabling the same model to dynamically adjust network parameters (such as convolution kernels, activation functions, loss weights, etc.) at different noise levels to achieve adaptability to multiple noise levels.

[0055] From the process of denoising the SEM image to be denoised described above, it can be seen that the pre-trained denoising diffusion model is not completely "blind" denoising. Specifically, the pre-trained denoising diffusion model requires the noise level provided externally as a guide, which is the key to the "pseudo-blind denoising" process. The accuracy of the noise level directly affects the denoising effect, and this noise level is exactly obtained through the local statistical feature analysis in the foregoing step S102 and the linear mapping in step S103. Compared with completely relying on deep learning models to estimate noise and denoise, the pseudo-blind denoising method of this application makes the denoising process more stable, and the noise level estimation provides more accurate initial conditions, which can reduce the dependence on complex model training.

[0056] The above combination Figure 1A method for denoising scanning electron microscope images is described. After analyzing the local statistical characteristics of the scanning electron microscope image to be denoised, the minimum eigenvalue can be obtained to represent the first noise level of the image, without the need to know the noise information of the image in advance, realizing an adaptive noise level estimation. Then, through linear mapping, the first noise level can be mapped to the second noise level, which corresponds to the denoising ability of the pre-trained denoising diffusion model. Therefore, using the pre-trained denoising diffusion model with the denoising ability corresponding to the second noise level to perform inference denoising on the scanning electron microscope image to be denoised can improve the accuracy of denoising model selection, ensure that the denoising effect matches the actual noise level of the image, and improve the denoising accuracy. In addition, using the solution of the present application, a denoising model corresponding to the noise level of the image can be used for denoising, which not only overcomes the problem of poor denoising effect of a single model under complex noise conditions, but also improves the denoising flexibility, and can ensure that images with different noise levels can obtain high-quality denoising effects.

[0057] Figure 3 FIG. shows an exemplary flowchart of a method 300 for analyzing the local statistical characteristics of a scanning electron microscope image to be denoised according to an embodiment of the present application. It can be understood that the description below in combination with Figure 3 is a specific implementation of the foregoing step S102. Therefore, the features described above in combination with Figure 1 can be similarly applied here.

[0058] As Figure 3 shown, at step S301, the method 300 can divide the scanning electron microscope image to be denoised to obtain a plurality of image blocks with the same dimension. At step S302, the method 300 can obtain a recombined two-dimensional matrix based on the plurality of image blocks with the same dimension. Further, at step S303, the method 300 can calculate the covariance matrix of the recombined two-dimensional matrix and perform eigenvalue decomposition on the covariance matrix to obtain a plurality of eigenvalues. Finally, at step S304, the method 300 can determine the minimum eigenvalue among the plurality of eigenvalues and determine the non-negative square root of the minimum eigenvalue as the first noise level. At the foregoing step S301, the method 300 can divide the scanning electron microscope image to be denoised (I ∈ R m×n×c , where m, n, and c are the height, width, and channels of the image respectively) to obtain a plurality of image blocks with the same dimension (or also referred to as matrix blocks) to ensure the local accuracy of noise estimation. Additionally or optionally, the scanning electron microscope image to be denoised is divided into a plurality of square image blocks with the same dimension. Here, the same dimension of different image blocks means that the width and height of different image blocks are the same, and each image block can be regarded as a local area, and it is assumed that the noise is uniformly distributed in these local areas.

[0059] At step S302 described above, method 300 may perform the following operations to obtain a recombined two-dimensional matrix: By straightening the matrices of multiple image patches with the same dimensions, multiple one-dimensional vectors can be obtained. In one embodiment, to straighten the matrix of an image patch into a one-dimensional vector, the rows in the matrix can be concatenated into a one-dimensional vector in ascending order of row numbers. After obtaining multiple one-dimensional vectors, a recombined two-dimensional matrix can be obtained by performing matrix recombination on the multiple one-dimensional vectors. One row in the recombined two-dimensional matrix corresponds to one one-dimensional vector and can be represented in the following form:

[0060]

[0061] where X s is the recombined two-dimensional matrix, x t is the t-th one-dimensional vector in X s , and s is the total number of one-dimensional vectors.

[0062] To facilitate understanding of the straightening and recombination operations, an example is used here for illustration. First, assume that the scanning electron microscope image to be denoised It can be understood that the numbers 1 to 16 here are only exemplary and illustrative, and do not limit the essence of the solution of this application. Then, by dividing image I, four image patches with the same dimensions can be obtained, and the width and height of each image patch are both 2, that is, and

[0063] Next, by straightening the matrices of multiple image patches, multiple one-dimensional vectors x1 = [1 2 5 6], x2 = [3 4 7 8], x3 = [9 10 13 14], and x4 = [11 12 15 16] can be obtained. Then, by performing matrix recombination on the obtained multiple one-dimensional vectors, a recombined two-dimensional matrix

[0064] After obtaining the recombined two-dimensional matrix X s , its covariance matrix ∑ of pixel values can be calculated, and the covariance matrix can reflect the statistical characteristics of the matrix. In actual operation, to calculate the covariance matrix of the recombined two-dimensional matrix, specifically, the average vector of multiple one-dimensional vectors can be calculated first. Then, based on the average vector and the recombined two-dimensional matrix, the covariance matrix is calculated.

[0065] Here, the average vector can be calculated using the following formula:

[0066]

[0067] Calculating the covariance matrix based on the average vector and the recombined two-dimensional matrix can be achieved using the following formula:

[0068]

[0069] Among them, ∑ is the covariance matrix, s is the number of one-dimensional vectors, μ is the average vector, and x t is the t-th one-dimensional vector.

[0070] Next, perform eigenvalue decomposition on the covariance matrix ∑, and r eigenvalues of the covariance matrix can be obtained In practical applications, the eigenvalue decomposition of the covariance matrix ∑ can be achieved by solving the following equation to obtain the eigenvalues:

[0071] det(Σ - λI) = 0 (7)

[0072] Among them, det() is the determinant operation, I is the identity matrix with the same dimension as ∑, and λ is the eigenvalue.

[0073] Among the eigenvalues of the covariance matrix, the smaller eigenvalues usually correspond to noise, while the larger eigenvalues usually represent the effective information in the image. Therefore, by analyzing these eigenvalues, the noise level can be estimated. In one embodiment, the minimum eigenvalue λ min , that is, the minimum value among the r eigenvalues is used to estimate the noise level. Assuming that the noise is Gaussian white noise, the first noise level, that is, the standard deviation σ of the noise, can be estimated by the formula to provide an accurate noise level estimate value for the subsequent denoising process.

[0074] Combined with the above Figure 3 The method 300 for analyzing the local statistical characteristics of the scanning electron microscope image to be denoised described above estimates the non-negative square root of the minimum eigenvalue as the first noise level by analyzing the local statistical characteristics of the noise in the scanning electron microscope image. Compared with the traditional preset noise model method, this statistical estimation method can adaptively estimate the noise level according to the image content without prior knowledge of the noise type and intensity.

[0075] Next, the method provided by the present application for denoising the scanning electron microscope image will be further described with specific examples. Here, reference can be made to Figure 4 , which shows the denoising performance of different denoising models for the same noisy image. In Figure 4 , the shown denoising models are the DDNM model, the DnCNN model, and the DenoiseAI model with three different denoising capabilities, and the denoising capabilities of the three DDNM models are σ = 0.15, σ = 0.25, and σ = 0.50 respectively.

[0076] The aforementioned DnCNN model (Denoising Convolutional Neural Network) is a deep learning model specifically used for image denoising. This model learns through training to restore a noise-free image from a noisy image, thereby achieving the denoising task. The core features of DnCNN include the use of convolutional layers, batch normalization layers, and the Rectified Linear Unit (ReLU) activation function. It adopts a deep convolutional neural network structure and can learn the characteristics of noise from a large number of noisy image pairs and remove the noise. The DeNoiseAI model is an image denoising tool based on artificial intelligence technology. It uses deep learning algorithms to remove noise in images and restore clear details. This model is particularly suitable for images taken under high ISO settings and can remove noise while preserving image details.

[0077] Figure 4 In a, the original noisy image is shown. This image contains obvious noise interference, exhibits a high noise level, and the areas with higher noise intensity cover the details and structures in the image, making it difficult to recognize the image.

[0078] Figure 4 In b, the denoising performance of the DDNM model with σ = 0.15 is shown. It can be seen from this figure that the DDNM model can significantly remove noise while retaining more image details. The details of the denoised image are relatively clear, and the noise is effectively suppressed. At this time, the DDNM model shows excellent noise suppression effect, with almost no obvious noise residue.

[0079] Figure 4 In c, the denoising performance of the DDNM model with σ = 0.25 is shown. It can be seen from this figure that the DDNM model can still remove most of the noise. However, compared with the denoising performance of the DDNM model with σ = 0.15, some image details are lost, especially in the edge areas, and the denoised image appears smoother. Nevertheless, the overall contour of the image is still clear, and the noise processing effect is significant.

[0080] Figure 4 In d, the denoising performance of the DDNM model with σ = 0.50 is shown. It can be seen from this figure that the denoising effect of the DDNM model has decreased. Although most of the noise has been removed, the smoothness of the image has increased significantly, and some details are lost, especially the fine structural information is difficult to retain. This indicates that there is a trade-off between detail retention and denoising effect in the denoising model with a high noise level.

[0081] Figure 4Figure e shows the denoising performance of the DenoiseAI model. As can be seen from the figure, compared with the DDNM model, the DenoiseAI model has a general noise removal effect when processing images, but the texture and detail loss of the image is more obvious at higher noise levels. Although the noise is reduced, the image smoothing effect is strong, especially in the background and edge areas, which shows that the DenoiseAI model has limited performance in high noise environments.

[0082] Figure 4 Figure f shows the denoising performance of the DnCNN model. As can be seen from the figure, compared with the DDNM model, the DnCNN model has a poor denoising effect in a high-noise environment. The noise in the image is still obvious, and the denoised image shows more artifacts, and the edges and details are lost more seriously. Although the processing result of DnCNN removes some noise, the residual noise and image blur still exist.

[0083] By comparing these models, we can see that the DDNM model performs particularly well in image denoising, and can effectively remove noise while retaining details and structures. In contrast, the DnCNN model and DenoiseAI model perform poorly, with insufficient denoising effects and easily leading to image blurring or loss of details.

[0084] In addition, by comparison, it can be seen that for the same noisy image, the denoising performance of denoising models with different denoising capabilities is not the same. Therefore, the method of the present application first accurately estimates the noise level of the noisy image, and then selects the denoising model corresponding to the noise level according to the estimated noise level for pseudo-blind denoising. Compared with the traditional blind denoising method that relies on a single deep learning model, it can provide more accurate denoising results under different noise conditions and show stronger denoising capabilities, especially in high noise and complex noise environments. It still maintains excellent detail retention and denoising effects.

[0085] Next, combine Figure 5 An electronic device 500 provided in an embodiment of the present application is exemplarily introduced. Figure 5 As shown, the electronic device 500 of the embodiment of the present application may include a processor 501 , a memory 502 and a communication bus 503 .

[0086] In the process of a specific embodiment, the above-mentioned processor 501 may be at least one of an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing image processing device (DSPD), a programmable logic image processing device (PLD), a field programmable gate array (FPGA), a CPU, a controller, a microcontroller, and a microprocessor. It can be understood that for different devices, the electronic devices for implementing the above-mentioned processor functions may also be others, and the present embodiment does not make specific limitations.

[0087] In the embodiment of the present application, the above-mentioned communication bus 503 is used to realize the connection and communication between the processor 501 and the memory 502; the memory 502 stores program instructions for denoising the scanning electron microscope image; when the above-mentioned processor 501 executes the program instructions stored in the memory 502, it realizes the present application in combination with Figures 1 to 4 the method for denoising the scanning electron microscope image described.

[0088] The above combination Figure 5 describes the electronic device that can be used to execute the method for denoising the scanning electron microscope image of the present application. It should be understood that the device structure or architecture here is only exemplary, and the implementation manner and implementation entity of the present application are not limited by it, but can be changed without departing from the spirit of the present application. It can be understood that the present disclosure emphasizes the differences between the various embodiments, and the same or corresponding parts can be referred to each other. For the sake of brevity, the present disclosure will not elaborate one by one.

[0089] According to the above description in combination with the drawings, those skilled in the art can also understand that the embodiments of the present application can also be implemented by software programs. Therefore, the present application also provides a computer-readable storage medium. The computer-readable storage medium stores program instructions for denoising the scanning electron microscope image, and the program instructions can be used to implement the method for denoising the scanning electron microscope image described in combination with Figures 1 to 4 the present application.

[0090] It should be noted that although the operations of the method of the present application are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. On the contrary, the order of the steps depicted in the flowchart can be changed. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step for execution, and / or one step may be decomposed into multiple steps for execution.

[0091] Although multiple embodiments of the present application have been shown and described herein, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Those skilled in the art can conceive of many changes, alterations, and alternative ways without departing from the spirit and scope of the present application. It should be understood that various alternatives to the embodiments of the present application described herein may be employed in the practice of the present application. The appended claims are intended to define the scope of protection of the present application and thus cover equivalents or alternatives within the scope of these claims.

Claims

1. A method for denoising scanning electron microscope images, characterized in that, Including: Obtain a scanning electron microscope (SEM) image to be denoised; Perform local statistical property analysis on the SEM image to be denoised to obtain a minimum eigenvalue to represent the first noise level of the SEM image; Perform a linear mapping on the first noise level to obtain a second noise level, where the second noise level corresponds to the denoising ability of a pre-trained denoising diffusion model; Use the pre-trained denoising diffusion model corresponding to the second noise level to perform inference denoising on the SEM image to be denoised to obtain a denoised SEM image.

2. The method according to claim 1, wherein Performing local statistical property analysis on the SEM image to be denoised to obtain a minimum eigenvalue to represent the first noise level of the SEM image includes: Divide the SEM image to be denoised to obtain multiple image blocks with the same dimension; Obtain a reconstructed two-dimensional matrix based on the multiple image blocks with the same dimension; Calculate the covariance matrix of the reconstructed two-dimensional matrix, and perform eigenvalue decomposition on the covariance matrix to obtain multiple eigenvalues; Determine the minimum eigenvalue among the multiple eigenvalues, and determine the non-negative square root of the minimum eigenvalue as the first noise level.

3. The method according to claim 2, wherein Obtaining a reconstructed two-dimensional matrix based on the multiple image blocks with the same dimension includes: Straighten the matrices of the multiple image blocks with the same dimension to obtain multiple one-dimensional vectors, where straightening the matrix of the image block means concatenating the rows in the matrix in ascending order of row numbers; Perform matrix recombination on the multiple one-dimensional vectors to obtain a reconstructed two-dimensional matrix, where one row in the reconstructed two-dimensional matrix corresponds to one one-dimensional vector.

4. The method according to claim 3, wherein Calculating the covariance matrix of the reconstructed two-dimensional matrix includes: Calculate the average vector of the multiple one-dimensional vectors; Calculate the covariance matrix based on the average vector and the reconstructed two-dimensional matrix.

5. The method according to claim 4, characterized in that Calculating the covariance matrix based on the average vector and the reconstructed two-dimensional matrix can be implemented using the following formula: where, ∑ is the covariance matrix, s is the number of one-dimensional vectors, μ is the mean vector, and x t is the t-th one-dimensional vector.

6. The method according to claim 1, characterized in that Performing a linear mapping on the first noise level to obtain a second noise level includes: Determine whether the first noise level is less than a preset threshold; In the case where the first noise level is less than the preset threshold, perform a linear mapping on the first noise level using a preset linear relationship to obtain the second noise level; In the case where the first noise level is greater than or equal to the preset threshold, determine the second noise level as 1.

7. The method according to claim 6, characterized in that, The preset linear relationship can be represented by the following formula: where, σ1 is the first noise level and σ2 is the second noise level.

8. The method according to claim 1, characterized in that The method further includes the following operations to fine-tune the pre-trained denoising diffusion model: Capture multiple sets of SEM images with different noise level distributions under different shooting conditions, where the shooting conditions include dwell time, acceleration voltage, and / or image averaging times; Annotate the corresponding noise level for each SEM image in the multiple sets of SEM images to obtain multiple sets of annotated SEM images; Use the multiple sets of annotated SEM images as training data and input them into the pre-trained denoising diffusion model to fine-tune it.

9. An electronic device, characterized in that, Including: A processor; And a memory that stores program instructions for denoising a scanning electron microscope image, which, when executed by a processor, enables the implementation of the method according to any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, Stored thereon are program instructions for denoising a scanning electron microscope image, which, when executed by a processor, implement the method according to any one of claims 1-8.

Images