Method for training de-noising model for de-noising scanning electron microscope image and related product

Abstract

The invention discloses a method for training a de-noising model for de-noising an image of a scanning electron microscope and a related product. The method comprises the following steps: acquiring a scanning electron microscope image and label noise data; performing noise estimation on the electron microscope image to obtain a noise estimation result; dividing the scanning electron microscope image into different noise intervals according to the noise estimation result; defining a first label and a second label based on a noise difference between a noise estimation result corresponding to each scanning electron microscope image in the corresponding noise interval and the corresponding label noise data; and combining the scanning electron microscope images of the corresponding noise intervals with the corresponding first labels or second labels to train a plurality of denoising models, and obtaining target denoising models adapted to different noise intervals. By using the scheme of the invention, accurate balance between a noise reduction effect and detail reservation can be realized.

Description

Technical Field

[0001] This application generally relates to the field of image processing technology. More specifically, this application relates to a method for training a denoising model for denoising scanning electron microscope images, a method for denoising scanning electron microscope images, electronic devices, and computer-readable storage media. Background Technology

[0002] Scanning electron microscopy (SEM), a core research tool in materials science, biology, electronics, and other fields, offers significantly higher resolution than optical microscopes. It clearly reveals crucial information such as the microscopic morphology, crystal structure, and elemental distribution of samples, providing indispensable technical support for scientific research and industrial testing. In practical applications, the quality of SEM images directly determines the accuracy of subsequent analytical results, and noise contamination is one of the main bottlenecks affecting image quality.

[0003] Existing denoising schemes for scanning electron microscope (SEM) images are mainly divided into two categories: traditional filtering and deep learning. Traditional methods rely on manually designed filtering rules, which suffer from problems such as cumbersome parameter adjustments, time-consuming computation, and a tendency to cause image blurring or loss of detail. While supervised deep learning models perform better, they lack adaptability to SEM images: some methods rely on preset noise types and intensities or use fixed-parameter models, making it difficult to cope with complex noise scenes; single deep neural networks experience decreased generalization performance when processing multiple noise levels, failing to balance detail preservation in low-noise scenes with noise suppression in high-noise scenes, and are prone to over-denoising or under-denoising.

[0004] In view of this, this application provides a scheme for training a denoising model for denoising scanning electron microscope images, so as to achieve a precise balance between denoising effect and detail preservation, and overcome the limitations of the prior art. Summary of the Invention

[0005] In order to at least address one or more of the technical problems mentioned above, this application proposes a scheme for training a denoising model for denoising scanning electron microscope images in several aspects.

[0006] In a first aspect, this application provides a method for training a denoising model for denoising scanning electron microscope (SEM) images, comprising: acquiring SEM images and label noise data; performing noise estimation on the SEM images to obtain noise estimation results; dividing the SEM images into different noise intervals based on the noise estimation results; defining a first label and a second label based on the noise difference between the noise estimation results corresponding to each SEM image in the corresponding noise interval and the corresponding label noise data; and combining each SEM image in the corresponding noise interval with the corresponding first label or second label to train multiple denoising models respectively, thereby obtaining a target denoising model adapted to different noise intervals.

[0007] In some embodiments, noise estimation of the scanning electron microscope image to obtain a noise estimation result includes: performing image feature value decomposition on the scanning electron microscope image; and obtaining the noise estimation result based on the minimum feature value after decomposition.

[0008] In some embodiments, image feature decomposition of the scanning electron microscope image includes: dividing the scanning electron microscope image into multiple sub-matrix blocks and straightening and recombining them to form new matrix blocks; calculating the covariance matrix of the new matrix blocks; and performing eigenvalue decomposition on the covariance matrix to obtain eigenvalues, thereby realizing image feature decomposition of the scanning electron microscope image.

[0009] In some embodiments, defining a first label and a second label based on the noise difference between the noise estimation result corresponding to each scanning electron microscope image in a corresponding noise range and the corresponding label noise data includes: defining the corresponding label noise data as the first label in response to the difference between the noise estimation result corresponding to the scanning electron microscope image in a corresponding noise range and the corresponding label noise data being greater than a preset value; and defining the corresponding label noise data as the second label in response to the difference between the noise estimation result corresponding to the scanning electron microscope image in a corresponding noise range and the corresponding label noise data being less than or equal to a preset value.

[0010] In some embodiments, the denoising model includes a deep convolutional neural network model, and the deep convolutional neural network model includes convolutional layers, batch normalization layers, and activation layers, and the deep convolutional neural network model employs a residual learning architecture.

[0011] In some embodiments, the method further includes: calculating the mean squared error loss function of the prediction results of each denoising model and the corresponding first label or second label during the training process; and minimizing the loss function to train multiple denoising models respectively.

[0012] In a second aspect, this application provides a method for denoising scanning electron microscope (SEM) images, comprising: acquiring an SEM image and a noise estimation result of the SEM image; inputting the SEM image into a target denoising model trained by the method described in the first aspect, based on the noise estimation result; and using the target denoising model to perform multi-step iterative denoising on the SEM image to obtain a denoised SEM image.

[0013] In some embodiments, the method further includes: sharpening the denoised scanning electron microscope image to obtain the final scanning electron microscope image.

[0014] In some embodiments, the denoised scanning electron microscope image is sharpened by the following operations: smoothing the denoised scanning electron microscope image to obtain a smoothed image; calculating the texture residual between the denoised scanning electron microscope image and the smoothed image; and superimposing the effective information in the texture residual whose intensity information meets preset conditions back into the denoised scanning electron microscope image.

[0015] In a third aspect, this application provides an electronic device, comprising: a processor; and a memory having computer instructions stored thereon for training a denoising model for denoising scanning electron microscope images, wherein when the computer instructions are executed by the processor, the method described in the first aspect is implemented; or having computer instructions stored thereon for denoising scanning electron microscope images, wherein when the computer instructions are executed by the processor, the method described in the second aspect is implemented.

[0016] In a fourth aspect, this application provides a computer-readable storage medium having stored thereon computer program instructions for training a denoising model for denoising scanning electron microscope images, wherein when executed by one or more processors, the computer program instructions cause the method described in the first aspect to be implemented; or having stored thereon computer program instructions for denoising scanning electron microscope images, wherein when executed by one or more processors, the computer program instructions cause the method described in the second aspect to be implemented.

[0017] By employing the above-described training scheme for denoising scanning electron microscope (SEM) images, this embodiment of the application obtains quantified noise levels by estimating the noise in the SEM images. This allows for the division of different noise intervals, grouping images with similar noise characteristics together. This enables the model to specifically learn the suppression rules of similar noise types, avoiding interference from different noise characteristics. By defining a first label (strong label) and a second label (weak label) based on the noise difference between the SEM images and labeled noise data, the denoising magnitude target of the model is clarified, making model training more directional. By combining images from different noise intervals with corresponding first or second labels to train multiple denoising models, a set of models adapted to different noise intervals and denoising needs is formed. This overcomes the generalization bottleneck of a single model, effectively solving the problems of excessive denoising in low-noise environments and insufficient denoising in high-noise environments. It achieves a flexible balance between denoising effect and detail preservation, improving the model's adaptability to complex noise environments and the reliability of denoising. Attached Figure Description

[0018] The above and other objects, features, and advantages of exemplary embodiments of this application will become readily understood by reading the following detailed description with reference to the accompanying drawings. In the drawings, several embodiments of this application are illustrated by way of example and not limitation, and the same or corresponding reference numerals denote the same or corresponding parts, wherein: Figure 1 This is an exemplary flowchart illustrating a method 100 for training a denoising model for denoising scanning electron microscope images according to an embodiment of this application; Figure 2 This is an exemplary schematic diagram illustrating the noise level distribution and the image proportion of each sample category according to embodiments of this application; Figure 3 This is an exemplary schematic diagram illustrating target denoising models trained according to embodiments of this application for different noise ranges; Figure 4 This is an exemplary flowchart illustrating a method 400 for denoising scanning electron microscope images according to an embodiment of this application; Figure 5 This is an exemplary overall flowchart illustrating the denoising of scanning electron microscope images according to embodiments of this application; Figure 6 This is an exemplary effect diagram illustrating iterative denoising using a target removal model according to an embodiment of this application; Figure 7 This is an exemplary schematic diagram showing a comparison of image sharpening after noise reduction according to an embodiment of this application; Figure 8 This is an exemplary structural block diagram illustrating an electronic device 800 according to an embodiment of this application. Detailed Implementation

[0019] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0020] It should be understood that the terms "comprising" and "including" used in the specification and claims of this application indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0021] It should also be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application. As used in this specification and claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this specification and claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations.

[0022] As used in this specification and claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if [described condition or event] is detected" may be interpreted, depending on the context, as "once determined," "in response to determination," "once [described condition or event] is detected," or "in response to detection of [described condition or event]."

[0023] As described in the background section above, existing scanning electron microscope image denoising schemes are mainly divided into two categories: traditional filtering and deep learning. Traditional methods can include spatial domain filtering (e.g., mean filtering, Gaussian filtering, median filtering, bilateral filtering, etc.) and transform domain filtering (e.g., Fourier transform, wavelet transform-based block matching, and 3D (BM3D) filtering algorithms, etc.). Spatial domain filtering suppresses noise through weighted or statistical processing of neighboring pixels, but mean filtering blurs image edges, median filtering is insufficient in preserving fine texture details, and bilateral filtering, while maintaining edge preservation, is parameter-sensitive and requires repeated manual adjustments. Transform domain filtering requires converting the image to the frequency domain for post-processing. While the BM3D algorithm is the optimal solution among traditional methods, overlapping block matching and iterative optimization are required during testing, resulting in processing times of several minutes per image, which is insufficient to meet industrial real-time requirements.

[0024] Existing supervised deep learning models (such as Denoising Convolutional Neural Networks (DnCNN) and Fast and Flexible Denoising Networks (FFDNet)) have improved denoising performance through optimizations such as residual learning and the introduction of noise level maps. For example, DnCNN uses residual learning and batch normalization to accelerate training and improve accuracy, while FFDNet combines noise level maps to optimize denoising performance under complex noise conditions. However, they are not well adapted to the complex noise characteristics of scanning electron microscope (SEM) images. Specifically, they rely on preset conditions, such as pre-setting the noise type or intensity. When the noise intensity of SEM images fluctuates dynamically due to changes in electron beam voltage and scanning magnification, the denoising effect deteriorates significantly.

[0025] Single-model generalization bottleneck: When using a single deep neural network to process multiple noise levels, the model needs to learn contradictory tasks such as low-noise fine-grained perturbation suppression and high-noise block artifact removal at the same time. This leads to over-denoising of low-noise images (loss of detail) and under-denoising of high-noise images (noise residue), and even image distortion.

[0026] Data utilization defects: Most existing models are trained directly using the full dataset without considering the differences in sample categories in scanning electron microscope images (e.g., solder balls are easier to obtain low-noise images, while chips have higher noise levels), which leads to overfitting of the model to the dominant sample category and poor generalization ability to other categories. At the same time, the strength of the labels is ignored, and the first label with the highest average number of frames is used for training, which cannot adapt to different noise reduction scenarios.

[0027] Based on this, this application provides a method for training a denoising model for scanning electron microscope images. By dividing the data and label strength into different noise ranges, models with various denoising capabilities are trained. This allows for the selection of an appropriate denoising model based on the noise level of the current image during inference deployment, enabling single or multiple iterations of denoising. This achieves a flexible balance between detail preservation and denoising capability.

[0028] The specific embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0029] Figure 1 This is an exemplary flowchart illustrating a method 100 for training a denoising model for denoising scanning electron microscope images according to an embodiment of this application. Figure 1As shown, step S101: acquire scanning electron microscope images and label noise data; step S102: perform noise estimation on the scanning electron microscope images to obtain noise estimation results; step S103: divide the scanning electron microscope images into different noise intervals according to the noise estimation results; step S104: define a first label and a second label based on the noise difference between the noise estimation results of each scanning electron microscope image in the corresponding noise interval and the corresponding label noise data; step S105: combine each scanning electron microscope image in the corresponding noise interval with the corresponding first label or second label to train multiple denoising models respectively, and obtain a target denoising model adapted to different noise intervals.

[0030] First, in step S101, scanning electron microscope (SEM) images and label noise data are acquired. In some implementation scenarios, SEM images are acquired using a scanning electron microscope (SEM) device, covering different experimental conditions (such as electron beam voltage, scanning magnification, and residence time) to obtain SEM images under different scenarios such as low voltage and high noise, high magnification detail, and fast scanning with low quality. The image size can be, for example, Tag noise data is collected at different frame averages (e.g., 1, 2, 4, 8, 16, 32, 64 times). In other implementation scenarios, sample categories may include, but are not limited to, solder balls, chips, copper mesh, metals, and trays.

[0031] By covering diverse experimental conditions and sample categories, the diversity of training data is ensured, laying the foundation for the model's generalization ability. Label images with different frame averages provide multi-gradient denoising targets, which can support the flexible definition of strong and weak labels, ensuring that the model can be trained efficiently.

[0032] Next, in step S102, noise estimation is performed on the scanning electron microscope image to obtain a noise estimation result. In some embodiments, image eigenvalue decomposition can be performed on the scanning electron microscope image, and then the noise estimation result can be obtained based on the minimum eigenvalue after decomposition. In other embodiments, the scanning electron microscope image is divided into multiple sub-matrix blocks and straightened and reassembled to form new matrix blocks. The covariance matrix of the new matrix blocks is calculated, and eigenvalue decomposition is performed on the covariance matrix to obtain eigenvalues, thereby realizing image eigenvalue decomposition of the scanning electron microscope image.

[0033] As an example, a scanning electron microscope image ( The noise is divided into multiple small matrix blocks to ensure local accuracy of the noise estimation. M , NLet and c represent the image height, width, and channels, respectively. Each block can be viewed as a local region, and noise is assumed to be uniformly distributed within these regions. Next, each small block is straightened out, resulting in several one-dimensional vectors, which are then used to form a new matrix block. : ,in This represents a set of submatrix blocks that have been decomposed. Let represent the straightened matrix block, i.e., the t-th one-dimensional vector, and s represent the total number of matrix blocks, s = (M-d+1)(N-d+1), where d represents the size of the sub-matrix block.

[0034] Obtain the reorganized new matrix block Calculate the covariance matrix of its pixel values. The covariance matrix reflects the statistical characteristics of a region. For a new matrix block... Its mean is the average of all one-dimensional vectors. The corresponding covariance matrix is ​​calculated as follows: .

[0035] Furthermore, regarding the covariance matrix Perform eigenvalue decomposition to obtain the eigenvalues ​​of the covariance matrix. In some implementation scenarios, algorithms such as Singular Value Decomposition (SVD) can be used to decompose the covariance matrix Σ into eigenvalues. It can be understood that smaller eigenvalues ​​in the covariance matrix often correspond to noise, while larger eigenvalues ​​typically represent valid information in the image. Therefore, by analyzing these eigenvalues, the noise level can be estimated.

[0036] In this embodiment, the minimum eigenvalue of the covariance matrix is ​​used. To estimate the intensity of the noise. As an example, assuming the noise is Gaussian white noise, the standard deviation of the noise... It can be estimated using the following formula: Therefore, noise estimation results for each electron microscope image are obtained. .

[0037] Sub-block partitioning ensures local accuracy of noise estimation and avoids misjudgment of local strong or low noise areas by global estimation. The covariance matrix and eigenvalue decomposition accurately separate noise from effective information, reducing the error in noise intensity calculation. The quantized σ value provides a unified standard for subsequent interval partitioning and label definition.

[0038] Further, in step S103, the scanning electron microscope image is divided into different noise intervals based on the noise estimation results. In some implementation scenarios, two key thresholds can be preset based on the noise level distribution statistics of the training data and the noise characteristics of the scanning electron microscope image, namely a low noise threshold. and high noise threshold Scanning electron microscope images were divided into low-noise, medium-noise, and high-noise ranges according to the following conditions: The low-noise range exhibits minimal noise impact and retains rich image details. The medium-noise range shows significant noise impact, but some details are still preserved. The high-noise range suffers severe noise impact and substantial loss of detail. In some implementation scenarios, multiple noise thresholds can be set based on the actual application scenario to divide more noise ranges; this application does not impose any restrictions on this.

[0039] By grouping images with significantly different noise features, images within the same interval have similar noise distributions and denoising requirements, avoiding the conflicting tasks of simultaneously learning fine-grained perturbation suppression and block artifact removal. This provides a structured data foundation for subsequent interval-label combination training, making model training more targeted.

[0040] Based on the different noise intervals, in step S104, a first label and a second label are defined based on the noise difference between the noise estimation results of each scanning electron microscope image in the corresponding noise interval and the corresponding label noise data.

[0041] In some embodiments, in response to the difference between the noise estimation result corresponding to the scanning electron microscope image in the corresponding noise range and the corresponding label noise data being greater than a preset value, the corresponding label noise data is defined as a first label; in response to the difference between the noise estimation result corresponding to the scanning electron microscope image in the corresponding noise range and the corresponding label noise data being less than or equal to a preset value, the corresponding label noise data is defined as a second label.

[0042] In one implementation scenario, the first and second labels can be defined using the following formula: ,in This represents the preset value, which is the threshold for the strength level of the noise reduction label pair. This represents the noise estimation result for the scanning electron microscope image within the corresponding noise range. This represents the corresponding label noise data.

[0043] As an example, suppose the scanning electron microscope image is in a low-noise region. =4, corresponding tag =2, =3, and Δσ=2 is less than 3, therefore it is defined as the second label, meaning that the noise in this interval is low, requiring slight noise reduction to avoid losing details; corresponding label =0.5, Δσ=3.5 is greater than 3, defined as the first label, meaning that noise reduction needs to be appropriately deepened, but more details still need to be preserved. Similarly, assuming that scanning electron microscope images in the high-noise region... =12, corresponding tag =10, =3, and Δσ=2 is less than 3, therefore it is defined as the second label, i.e., mild demand within the interval, and some noise is reduced first; corresponding label =7, Δσ=5 is greater than 3, and is defined as the first label, i.e., deep noise reduction.

[0044] In some implementation scenarios, the proportion of data from different sample categories can be adjusted for each noise range to avoid model bias caused by an excessive amount of data from one particular sample category. This is because scanning electron microscope images are greatly affected by the type of sample being photographed; using the same scanning parameters to photograph different samples may result in different noise levels. Therefore, to prevent the noise reduction model from becoming dependent on sample categories, class equalization can be performed on the training data under specific labels, retaining the category with the fewest numbers of images. For example, 130% of the images can be randomly sampled from the categories with the most images, ensuring that the difference in the number of images from each category does not exceed 30%.

[0045] Finally, in step S105, the scanning electron microscope images of each noise interval and the corresponding first label or second label are combined to train multiple denoising models respectively, so as to obtain target denoising models that are adapted to different noise intervals.

[0046] For example, suppose the image data in different noise ranges are , The label noise data is . The superscript indicates the noise level range [0,3), [3,6), [6,10), [10,-), '-' indicates All data greater than 10; The upper right subscript indicates the label data range [1,2), [2,3), [3,4), [4,5]. In this scenario, images in the medium to high noise range can be... and the first tag Combining these methods can train a medium-intensity noise reduction model, which can be applied to... High noise level data. Images in the medium-to-high noise range. Second label Combining these methods can train a weakly denoised model, which can be applied to... The noise level data.

[0047] In some embodiments, the denoising model includes a deep convolutional neural network (DNN) model, which comprises convolutional layers, batch normalization layers, and activation layers. Specifically, the convolutional layers extract local features from the image, each using a 3×3 kernel (balancing receptive field and computational cost); the batch normalization layers normalize the output of the convolutional layers, accelerating training convergence and mitigating gradient vanishing; and the activation layers employ the ReLU (Rectified Linear Unit) activation function, introducing non-linearity to enhance the model's feature representation capabilities. Furthermore, the deep convolutional neural network model incorporates residual learning. The model's output is the input noisy image minus the predicted noise, i.e., the denoised clean image, simplifying the learning task.

[0048] In some embodiments, the prediction results of each denoising model and the mean squared error loss function corresponding to the first or second label can be calculated during training to minimize the loss function and train multiple denoising models separately. For example, based on the mean squared error loss function, each denoising model can be trained backward using gradient descent to obtain a target denoising model adapted to different noise ranges. In some implementation scenarios, peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) can be used as performance metrics.

[0049] As described above, this application provides comprehensive data support for model training by collecting diverse scanning electron microscope images and labeled noise data with different average counts. By dividing each image into different noise intervals based on its noise level and defining strong and weak labels, images with similar noise characteristics are trained together. This allows the model to specifically learn the suppression rules of similar noise types and clearly define the model's noise reduction target, making training more directional and guiding the model to develop noise reduction capabilities adapted to different needs, significantly improving the model's generalization performance in complex noise environments. It also enables the model to adaptively adjust learning weights, achieving precise matching of noise features during training and effectively improving noise reduction accuracy. Furthermore, this multi-model combined training also breaks through the generalization bottleneck of a single model and can adapt to different noise scenarios.

[0050] Furthermore, sample class equalization eliminates the model's dependence on specific sample classes, ensuring stable performance across various sample images. Additionally, the architecture of a deep convolutional neural network enhances feature extraction and noise separation capabilities, the mean squared error loss function effectively reduces the difference between predicted and actual noise, and peak signal-to-noise ratio and structural similarity metrics ensure that denoising balances image quality and structural integrity. Ultimately, this achieves a flexible balance between denoising effectiveness and detail preservation, improving the model's adaptability to complex noisy environments and the reliability of denoising results.

[0051] Figure 2This is an exemplary schematic diagram illustrating the noise level distribution and the image proportion of each sample category according to embodiments of this application. Figure 2 Figure (a) shows the noise level distribution and number of scanning electron microscope (SEM) images for all different samples, where the horizontal axis represents the noise level distribution of the SEM images for different samples, and the vertical axis represents the corresponding number. As mentioned earlier, SEM images of different samples such as solder balls, chips, copper meshes, metals, and trays can be acquired using a scanning electron microscope. The image size is [missing information]. The average number of frames was set to 1, 2, 4, 8, 16, 32, and 64. To reduce GPU memory usage during model training, [the following can be done]: Cut into Subgraphs. Figure 2 Figure (a) in the figure exemplarily illustrates different sample sizes. The noise level distribution.

[0052] Figure 2 Figure (b) shows the category percentages for all samples, with solder balls, chips, copper mesh, metal, and trays accounting for 20%, 35.1%, 18.0%, 9.0%, and 17.9%, respectively. Figure 2 As shown in Figure (b), solder ball features are more likely to produce low-noise images compared to chip features. To prevent the training model from becoming dependent on sample categories, class equalization processing is required after dividing the data into noise level intervals.

[0053] Specifically, the number of samples in each category within the image data pairs is counted, and some data is removed to ensure that the number of samples between categories does not exceed 25%. This is done using labeled noise data. For example, Figure 2 Figure (c) in the diagram represents the noise level in the labeled data. The image percentages for each category are as follows: solder balls, chips, copper mesh, metal, and trays, at 19.8%, 36.9%, 17.7%, 9.9%, and 15.7%, respectively. Copper mesh has the lowest percentage, with 825 images. To ensure a total number of images is around 5000, 1000 images are initially retained for each of the other categories. For copper mesh, which has fewer than 1000 images (175), images are randomly sampled from the other four categories. The final balanced percentages for each category are as follows: Figure 2 As shown in Figure (d), the chip, metal, solder ball, tray, and copper mesh account for 20.9%, 20.9%, 20.9%, 20.9%, and 16.5%, respectively.

[0054] Figure 3This is an exemplary schematic diagram illustrating target denoising models trained for different noise ranges according to embodiments of this application. As described above, during the training phase, models with different noise levels, i.e., different denoising capabilities, can be trained based on the data noise level distribution range and label strength. Specifically, as... Figure 3 As shown in Figure (a), the initial noise distribution of the training set data is... The training data and labeled data with different noise levels are input into the model separately. Under the same training conditions, As a label, after training convergence, the noise level distribution of the training set is within the interval [range missing]. Inside, for example Figure 3 As shown in Figure (b). Similarly, As a label, after training convergence, the noise level distribution of the training set is within the interval [range missing]. Inside, for example Figure 3 As shown in Figure (c).

[0055] Figure 3 Figures (d) and (e) show, respectively, the use of As a label, after training convergence, the noise level distribution of the training set is within the interval [range missing]. Internal and Use After training convergence, the noise level of the training set is distributed in the interval [range]. The lower the noise level of the labels, the lower the range of noise level distribution in the data after model convergence, and the stronger the model's noise reduction capability.

[0056] Based on the target denoising models obtained from training in different noise ranges, inference denoising can be performed on newly arrived noisy images.

[0057] Figure 4 This is an exemplary flowchart illustrating a method 400 for denoising scanning electron microscope images according to an embodiment of this application. Figure 4 As shown, method 400 includes: step S401: acquiring a scanning electron microscope image and a noise estimation result of the scanning electron microscope image; step S402: inputting the scanning electron microscope image into a trained target denoising model based on the noise estimation result; step S403: using the target denoising model to perform multi-step iterative denoising on the scanning electron microscope image to obtain a denoised scanning electron microscope image.

[0058] First, in step S401, a scanning electron microscope (SEM) image and a noise estimation result of the SEM image are acquired. In some embodiments, the SEM image is a real-time image of the image to be denoised, captured using a scanning electron microscope (SEM) image. The noise estimation result of the SEM image can be obtained by sub-block partitioning, calculating the covariance matrix, eigenvalue decomposition, and using the smallest eigenvalue, as described above.

[0059] Next, in step S402, based on the noise estimation result, the scanning electron microscope image is input into the trained target denoising model. For example, if the image to be denoised is σ∈[0,3), a low-noise, weak denoising model is selected. Further, in step S403, the target denoising model is used to perform multi-step iterative denoising on the scanning electron microscope image to obtain the denoised scanning electron microscope image.

[0060] In some implementation scenarios, the target denoising model can be used to perform multi-step iterative denoising on scanning electron microscope images. After each iteration, the noise level of the current image is re-estimated until the noise level is lower than a preset stopping threshold, thus obtaining a denoised scanning electron microscope image. Based on this, noise is gradually reduced through multi-step iterations, avoiding the loss of details and artifacts caused by a single strong denoising. The noise re-estimation at each step is dynamically adjusted, ensuring that the denoising process is accurate and controllable.

[0061] In some embodiments, the denoised scanning electron microscope image can also be sharpened to obtain the final scanning electron microscope image. In some embodiments, the denoised scanning electron microscope image is sharpened by the following operations: smoothing the denoised scanning electron microscope image to obtain a smoothed image; calculating the texture residual between the denoised scanning electron microscope image and the smoothed image; and superimposing the effective information in the texture residual whose intensity information meets preset conditions onto the denoised scanning electron microscope image.

[0062] In some implementation scenarios, the denoised scanning electron microscope (SEM) image can be smoothed using methods such as Gaussian smoothing to obtain a smoothed image. The texture residual is calculated as: Denoised Image - Smoothed Image, ensuring the inclusion of high-frequency details such as edges and textures. Then, effective information from the residual image whose intensity (i.e., pixel value) is greater than a threshold (e.g., 20) can be selected and superimposed onto the denoised SEM image to obtain the final SEM image. Thus, thresholding effectively distinguishes edge details from noise, preventing the amplification of residual noise during sharpening, and the superposition operation enhances edge contrast, making image details clearer while maintaining the naturalness of the overall structure.

[0063] Figure 5 This is an exemplary overall flowchart illustrating denoising of scanning electron microscope images according to embodiments of this application. As an example, training set data is used... , Training yields the target denoising model For example, it is used to denoise scanning electron microscope images that need to be denoised.

[0064] like Figure 5As shown, in step S501, an arbitrary sample is obtained. The scanning electron microscope (SEM) image is obtained, and at step S502, the noise level of the SEM image is estimated to obtain the noise level result. Next, in step S503, the scanning electron microscope image is input into the target removal model. First, a noise reduction process is performed on the image. Then, in step S504, a second noise level estimation is performed on the denoised image to obtain the noise level result, denoted as... .

[0065] Further, at step S505, the noise level result is determined. Is the noise level greater than a given noise reduction threshold? If not, i.e., less than or equal to the noise reduction threshold, then image noise reduction ends, and in step S506, the image is sharpened. For example, the image is Gaussian smoothed, and then subtracted from the original image to obtain a texture residual image. If the intensity information of the residual image is greater than a given threshold, it is used as valid information and superimposed back onto the original image to sharpen the image edges. Specific details regarding sharpening can be found in the above description, and will not be repeated here. If the noise level result... If the value exceeds the noise reduction threshold, return to step S503 and use the target removal model. Continue denoising until the noise level is less than or equal to the denoising threshold. After sharpening, the final denoised scanning electron microscope image is output at step S507.

[0066] Figure 6 This is an exemplary effect diagram illustrating iterative denoising using a target-based model removal method according to an embodiment of this application. For example... Figure 6 Figures (a), (b), and (c) in the figure exemplify the original solder ball noise map, respectively, and demonstrate the noise removal using the target removal model ( The diagrams show the noise reduction results after one and two noise reduction cycles, respectively. , and .like Figure 6 Figures (d), (e), and (f) in the diagram exemplify the original chip noise map, followed by the noise reduction model ( ). The diagrams show the noise reduction results after one and two noise reduction cycles, respectively. , and It can be seen that the stepwise denoising process of the weak denoising model can ensure the denoising effect while avoiding the removal of image details.

[0067] Figure 7 This is an exemplary schematic diagram illustrating a comparison of image sharpening after noise reduction according to an embodiment of this application. Figure 7The left side of the image, from top to bottom, shows the original image, a diagram of one noise reduction step, and a diagram of two noise reduction steps. Figure 7 From top to bottom on the right side of the image, the images show the sharpened result of the original image, the sharpened result after one round of denoising, and the sharpened result after two rounds of denoising. As can be seen from the images, by introducing a post-processing sharpening algorithm, the edge details of the image can be effectively enhanced, thus significantly improving the blurring problem of the denoised image.

[0068] Figure 8 This is an exemplary structural block diagram illustrating an electronic device 800 according to an embodiment of this application. For example... Figure 8 As shown, the electronic device 800 of this application may include a processor 801 and a memory 802, wherein the processor 801 and the memory 802 communicate via a bus. The memory 802 stores program instructions for training a denoising model for denoising scanning electron microscope images. When the program instructions are executed by the processor 801, they enable the implementation of the denoising model according to the foregoing description in conjunction with the appendix. Figure 1 The method described herein is for training a denoising model for denoising scanning electron microscope images. Alternatively, memory 802 stores program instructions for denoising scanning electron microscope images, which, when executed by processor 801, enable the implementation of the denoising model described above in conjunction with the appendix. Figure 4 The method described is for denoising scanning electron microscope images.

[0069] Based on the foregoing description in conjunction with the accompanying drawings, those skilled in the art will understand that the embodiments of this application can also be implemented by software programs. Therefore, this application also provides a computer-readable storage medium. This computer-readable storage medium stores computer-readable instructions thereon for training a denoising model for denoising scanning electron microscope images. When these computer-readable instructions are executed by one or more processors, they implement the embodiments of this application in conjunction with the accompanying drawings. Figure 1 The method described herein for training a denoising model for denoising scanning electron microscope images; or the computer-readable storage medium having stored thereon computer-readable instructions for denoising scanning electron microscope images, which, when executed by one or more processors, implement the present application in conjunction with the appendix. Figure 4 The method described is for denoising scanning electron microscope images.

[0070] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0071] It should be noted that although the operations of the method of this application are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. On the contrary, the steps depicted in the flowchart can be performed in a different order. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0072] It should be understood that when the terms "first," "second," "third," and "fourth," etc., are used in the claims, specification, and drawings of this application, they are used only to distinguish different objects and not to describe a specific order. The terms "comprising" and "including" as used in the specification and claims of this application indicate the presence of the described features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or collections thereof.

[0073] It should also be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application. As used in this specification and claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this specification and claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations.

[0074] Although the embodiments of this application are described above, the content is merely an example adopted for the purpose of facilitating understanding of this application and is not intended to limit the scope and application scenarios of this application. Any person skilled in the art described in this application may make any modifications and changes in the form and details of the implementation without departing from the spirit and scope disclosed in this application, but the scope of patent protection of this application shall still be determined by the scope defined in the appended claims.

Claims

1. A method for training a denoising model for denoising scanning electron microscope images, comprising: Acquire scanning electron microscope images and label noise data; Noise estimation is performed on the scanning electron microscope image to obtain the noise estimation result; Based on the noise estimation results, the scanning electron microscope image is divided into different noise intervals; The first label and the second label are defined based on the noise difference between the noise estimation results of each scanning electron microscope image in the corresponding noise range and the corresponding labeled noise data. Multiple denoising models are trained by combining each scanning electron microscope image in the corresponding noise range with the corresponding first or second label, thereby obtaining a target denoising model that adapts to different noise ranges.

2. The method according to claim 1, wherein performing noise estimation on the scanning electron microscope image to obtain the noise estimation result includes: Image feature value decomposition is performed on the scanning electron microscope image; The noise estimation result is obtained based on the minimum eigenvalue after decomposition.

3. The method according to claim 2, wherein image feature value decomposition of the scanning electron microscope image comprises: The scanning electron microscope image is divided into multiple sub-matrix blocks and then straightened and recombined to form a new matrix block; Calculate the covariance matrix of the new matrix block; The covariance matrix is ​​decomposed into eigenvalues ​​to obtain eigenvalues, and the scanning electron microscope image is used for image eigenvalue decomposition.

4. The method according to claim 1, wherein defining the first label and the second label based on the noise difference between the noise estimation result corresponding to each scanning electron microscope image in the corresponding noise interval and the corresponding labeled noise data includes: If the difference between the noise estimation result of the scanning electron microscope image in the corresponding noise range and the corresponding label noise data is greater than a preset value, the corresponding label noise data is defined as the first label. If the difference between the noise estimation result corresponding to the scanning electron microscope image in the corresponding noise range and the corresponding label noise data is less than or equal to a preset value, the corresponding label noise data is defined as the second label.

5. The method according to claim 1, wherein the denoising model comprises a deep convolutional neural network model, and the deep convolutional neural network model comprises convolutional layers, batch normalization layers, and activation layers.

6. The method of claim 1, further comprising: During training, the prediction results of each denoising model and the mean squared error loss function of the corresponding first label or second label are calculated. Multiple denoising models are trained by minimizing the loss function.

7. A method for denoising scanning electron microscope images, comprising: Acquire scanning electron microscope (SEM) images and noise estimation results for the SEM images; Based on the noise estimation results, the scanning electron microscope image is input into the target denoising model trained by the method according to any one of claims 1-6; The target denoising model is used to perform multi-step iterative denoising on the scanning electron microscope image to obtain the denoised scanning electron microscope image.

8. The method of claim 7, further comprising: The denoised scanning electron microscope image is then sharpened to obtain the final scanning electron microscope image.

9. The method of claim 8, wherein the denoised scanning electron microscope image is sharpened by the following operation: The denoised scanning electron microscope image is smoothed to obtain a smoothed image; Calculate the texture residual between the denoised scanning electron microscope image and the smoothed image; The effective information in the intensity information of the texture residual that meets the preset conditions is superimposed back into the denoised scanning electron microscope image.

10. An electronic device, comprising: processor; as well as A memory storing computer instructions for training a denoising model for denoising scanning electron microscope images, wherein when the computer instructions are executed by a processor, the method according to any one of claims 1-6 is implemented; or a memory storing computer instructions for denoising scanning electron microscope images, wherein when the computer instructions are executed by a processor, the method according to any one of claims 7-9 is implemented.

11. A computer-readable storage medium storing computer program instructions thereon for training a denoising model for denoising scanning electron microscope images, wherein the computer program instructions, when executed by one or more processors, cause the implementation of the method according to any one of claims 1-6; or storing computer program instructions thereon for denoising scanning electron microscope images, wherein the computer program instructions, when executed by one or more processors, cause the implementation of the method according to any one of claims 7-9.

Images