Image processing method of critical dimension scanning electron microscope and electronic equipment

Abstract

The invention discloses a critical dimension scanning electron microscope image processing method and electronic equipment, and the method comprises the steps: employing a pre-trained image processing model to carry out the processing of an input image, and obtaining an optimized image; the pre-trained image processing model comprises a de-noising branch and an edge enhancement branch, the two branches are both formed based on a U-Net network architecture, the de-noising branch comprises a first encoder module and a first decoder module which are connected in sequence, and the first encoder module and the first decoder module both adopt an SC block module to extract local and global features of an image; the edge enhancement branch comprises an edge information extraction module, and a second encoder module and a second decoder module which are connected in sequence; and the feature fusion module is used for fusing all levels of output of the first encoder module and all levels of output of the second encoder module to serve as all levels of input of the first encoder module. Compared with a traditional image denoising algorithm, the algorithm solves the problem of edge blurring caused by excessive smoothness of the image while denoising.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a critical dimension scanning electron microscope image processing method and electronic equipment. Background Art

[0002] With the continuous advancement of high numerical aperture extreme ultraviolet lithography (High NA EUVL) technology, extensive research on thinner photoresists has become increasingly urgent. However, during critical dimension electron scanning microscope (CD-SEM) measurement, photoresist shrinkage has always been an important issue in lithography feature metrology, because it not only affects the accurate measurement of CD-SEM, but also may affect the process output due to the damage caused by the electron beam to the sample. The thinning of photoresist worsens the photoresist shrinkage ISSUE and brings great challenges. In order to reduce the impact of photoresist shrinkage, it is necessary to use lower voltage or current and fewer scanning frames to acquire SEM images, but this operation will inevitably cause image noise and reduce measurement accuracy. When the signal-to-noise ratio (SNR) is lower than 2, the estimation of unbiased line width roughness (LWR) becomes less accurate, and when the SNR reaches 1, the error is as high as 30%, so a high SNR is essential to obtain reliable LWR estimation. However, the method of obtaining high SNR images by changing the CD-SEM settings may result in long measurement time, reduced yield, and damage to the material. Under extreme conditions, low signal-to-noise ratio CD-SEM images may not be able to be correlated using edge profile-based algorithms. Therefore, designing effective image denoising methods is critical to obtain accurate metrology while minimizing photoresist shrinkage.

[0003] In recent years, with the rise of deep learning related technologies, deep learning methods have played an increasingly important role in the field of denoising. They overcome the limitations of traditional methods that rely on prior information and artificial features, and have strong generalization capabilities. For example, feedforward denoising convolutional neural network (DnCNN), convolutional blind denoising network (CBDNet), U-type Transformer network (Uformer), attention-guided denoising CNN (ADNet), multi-stage image denoising convolutional neural network (MWDCNN), etc. Although the existing CD-SEM image denoising algorithms have achieved good results, they still face problems such as over-smoothing of denoised images and loss of edge information. Summary of the invention

[0004] The embodiments of the present invention provide a critical dimension scanning electron microscope image processing method and electronic device to solve the problems of over-smoothing of denoised images and loss of edge information that still exist in the image denoising algorithms in the prior art.

[0005] According to a first aspect of the present invention, a critical dimension scanning electron microscope image processing method is provided, comprising: Use a pre-trained image processing model to process the input image to obtain an optimized image; Wherein, the pre-trained image processing model includes: A denoising branch, based on a U-Net convolutional neural network architecture, includes a first encoder module and a first decoder module connected in sequence, wherein the first encoder module and the first decoder module both use an SC block module to extract local and global features of the image; The edge enhancement branch is based on the U-Net convolutional neural network architecture, which includes an edge information extraction module, a second encoder module and a second decoder module connected in sequence; A feature fusion module is used to fuse the outputs of each level of the first encoder module and the second encoder module to serve as the inputs of each level of the first encoder module.

[0006] According to a second aspect of the present invention, an electronic device is provided, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the steps of the critical dimension scanning electron microscope image processing method of the first aspect mentioned above.

[0007] According to a third aspect of the present invention, there is provided a storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the critical dimension scanning electron microscope image processing method of the first aspect.

[0008] The critical dimension scanning electron microscope image processing method of the present invention can transmit the input image to the edge enhancement module and the denoising module in parallel, wherein the denoising module including the SC block enhances the ability to extract the global and local features of the input image; the edge enhancement branch first obtains the edge information of the input image through the edge information extraction module, and then extracts the global and local features of the edge information; the feature fusion module is also used to fuse the features of the encoder module and the decoder module of the two branches to make up for the deficiency of the denoising branch in edge feature extraction. Compared with the traditional image denoising algorithm, the algorithm solves the edge blurring problem caused by excessive image smoothing while denoising. The experimental results show that the algorithm achieves good denoising effect in the test image and improves the measurement accuracy of the critical dimension scanning electron microscope (CD-SEM). Furthermore, the critical dimension scanning electron microscope image processing method proposed in this patent provides an effective solution to the challenges brought by photoresist shrinkage, which is of great significance for promoting the development of high numerical aperture extreme ultraviolet lithography technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0010] Figure 1 A schematic diagram of the network structure of an image processing model used in a critical dimension scanning electron microscope image processing method according to an embodiment of the present invention; Figure 2 A block diagram of an SC block module in a critical dimension scanning electron microscope image processing method according to an embodiment of the present invention; Figure 3 It is a flow chart of processing an input feature map by an SC block module in a critical dimension scanning electron microscope image processing method according to an embodiment of the present invention; Figure 4 A schematic diagram of four trainable Soble operators in horizontal, vertical and diagonal directions used in edge enhancement processing in a critical dimension scanning electron microscope image processing method according to an embodiment of the present invention; Figure 5 A module diagram of a cross-attention fusion module in a critical dimension scanning electron microscope image processing method according to an embodiment of the present invention; Figure 6 A process for processing input features of a cross-attention fusion module in a critical dimension scanning electron microscope image processing method according to an embodiment of the present invention; Figure 7 A flowchart of the steps of performing noise simulation processing on an original SEM image in a critical dimension scanning electron microscope image processing method according to an embodiment of the present invention; Figure 8 A comparison diagram of the processing results of a critical dimension scanning electron microscope image processing method according to an embodiment of the present invention and the processing results of an existing traditional processing method; Fig. 9 A comparison diagram of the visual effect of a processing result of a critical dimension scanning electron microscope image processing method according to an embodiment of the present invention and the visual effect of a processing result of an existing traditional processing method; Fig.10 It is a comparison diagram of CD values ​​of three randomly selected CD-SEM images processed by a critical dimension scanning electron microscope image processing method according to an embodiment of the present invention and an existing traditional processing method; Fig.11 It is a result comparison diagram of LER values ​​of three randomly selected CD-SEM images processed by a critical dimension scanning electron microscope image processing method according to an embodiment of the present invention and an existing traditional processing method; Fig.12 It is a result comparison diagram of LWR values ​​of three randomly selected CD-SEM images processed by a critical dimension scanning electron microscope image processing method according to an embodiment of the present invention and an existing traditional processing method; Fig.13 for Figures 10 to 12 A line chart comparing the CD, LWR and LER values ​​under each treatment method; Fig.14 It is a schematic structural diagram of an embodiment of an electronic device of the present invention. DETAILED DESCRIPTION

[0011] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0012] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0013] In the description of the present invention, it should be understood that if the terms "center", "middle", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "axial", "radial", "circumferential" and the like appear, the orientation or position relationship indicated is based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. The features defined as "first" and "second" are used to distinguish the feature names, rather than having special meanings. In addition, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "multiple" means two or more.

[0014] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0015] It should also be noted that, in this article, the terms "include" and "comprise" include not only those elements, but also other elements not explicitly listed, or also include elements inherent to such processes, methods, articles or equipment. In the absence of further restrictions, the elements defined by the sentence "include..." do not exclude the existence of other identical elements in the process, method, article or equipment that includes the elements. The terms used in this article are generally commonly used by those skilled in the art. If there is any inconsistency with the commonly used terms, the terms in this article shall prevail.

[0016] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0017] The present invention will be further described in detail below in conjunction with the accompanying drawings.

[0018] Specifically, in the present invention, a pre-trained image processing model is used to process the input image to obtain an optimized image, thereby achieving denoising of the image and effectively reducing the problem of edge information loss. Among them, in the present invention, the pre-trained image processing model used includes a denoising branch and an edge enhancement branch. The denoising branch and the edge enhancement branch are both based on the U-Net convolutional neural network architecture. The denoising branch includes a first encoder module and a first decoder module connected in sequence, and the first encoder module and the first decoder module both use SCblock modules to extract local and global features of the image. The edge enhancement branch includes an edge information extraction module, a second encoder module and a second decoder module connected in sequence, and the edge information extraction module is used to obtain edge information of the input image, and then extract global and local features of the edge information. The image processing model also includes a feature fusion module for fusing the outputs of the first encoder module and the second encoder module at various levels as inputs of the first encoder at various levels.

[0019] Specifically, Figure 1 The network structure diagram of the image processing model used in the critical dimension scanning electron microscope image processing method according to one embodiment of the present invention is schematically shown. Figure 1 As shown, the image processing model adopted by the present invention includes a denoising branch and an edge enhancement branch. The denoising branch is the main branch, and the edge enhancement branch is the auxiliary branch. The outputs of the encoder module and the decoder module of the edge enhancement branch are fused with the outputs of the encoder module and the decoder module of the denoising branch through the feature fusion module to improve the deficiency of the denoising branch in edge extraction.

[0020] The denoising branch is built based on the U-Net convolutional neural network architecture, which includes the SC block module, the SConv module (strided convolution), and the TConv module (transposed convolution). Figure 1As shown, the first encoder module is composed of a repeated SC block module and an SConv module, and the first decoder module is composed of a repeated SC block module and a TConv module, and the SConv module and the TConv module are connected by a jump connection. SC block is a new type of neural network module that enhances the feature extraction capability by combining Swin Transformer (SwinT) and residual convolution (RConv). The design of this module aims to effectively fuse the features of SwinT block and RConvblock through convolution, Split function, Concat function and residual connection (Residual Connection) to improve the performance of the network when processing images. Specifically, Figure 2 The module diagram of the SC block module in the critical dimension scanning electron microscope image processing method according to one embodiment of the present invention is schematically shown. Figure 3 The schematic diagram shows the process of the SC block module processing the input feature map. Figure 2 and Figure 3 As shown, the process can be specifically implemented as including the following steps: Step S11: perform 1×1 convolution on the input feature map and divide it into a first feature map and a second feature map through the Split function; Step S12: Input the first feature map into the SwinT block to obtain a third feature map, and input the second feature map into the RConv block to obtain a fourth feature map; Step S13: The third feature map and the fourth feature map are combined by the Concat function to obtain a fifth feature map, and the number of channels of the fifth feature map is adjusted by a 1×1 convolution layer to match the acquired image; Step S14: Perform a residual connection between the fifth feature map with adjusted channel number and the input feature map to obtain an output feature map.

[0021] Specifically, when executing step S11, first input feature map X Perform a 1×1 convolution and then divide it into two parts through the Split function, which are respectively related to the feature map X The corresponding first feature map and second feature map are expressed as follows:

[0022] Among them, the obtained X 1 is the first feature map obtained in step S11, X 2 is the second feature map obtained in step S11.

[0023] Then, step S12 is performed to obtain the first feature map X 1 and the second figure X 2 are input to SwinTblock and RConv block respectively, and the first feature map is processed by SwinT block X 1 is processed to obtain the third feature map, and the second image is processed by RConv block X 2 is processed to obtain the fourth feature map. Its expression is as follows:

[0024] Among them, the obtained Y 1 is the third characteristic map obtained in step S12, Y 2 is the fourth feature map obtained in step S12.

[0025] Then, step S13 and step S14 are performed to obtain the third feature map. Y 1 and the fourth characteristic diagram Y 2 to obtain the fifth feature map, and adjust the number of channels of the fifth feature map through a 1×1 convolution layer, and finally perform residual connection to obtain the final SCblock output, that is, the output feature map. Its expression is as follows:

[0026] The edge enhancement branch is built based on the U-Net convolutional neural network architecture, which includes the Sobel convolution module, the RConvblock module (residual convolutional), the SConv module (strided convolution), and the TConv module (transposed convolution). Figure 1As shown, the edge information extraction module is set to a pre-trained Sobel convolution module, the second encoder module is composed of repeatedly set RConv block modules and SConv modules, the second decoder module is composed of repeatedly set RConv block modules and TConv modules, and the SConv module and the TConv module are connected by a jump connection. The adopted Sobel convolution can adaptively adjust the value of the Sobel factor α during the network model training process, so that the network model can dynamically adjust the sensitivity of edge detection for different training data and tasks during the training process, so as to extract edge information of different intensities. In the process of edge enhancement processing of the acquired image using the Sobel convolution module, the acquired image is convolved by the Sobel operator in the Sobel convolution module to obtain an edge information feature map. Specifically, in order to retain the edge features of the processed image as much as possible, in some possible implementations, four trainable Sobel operators including horizontal, vertical and diagonal directions (two diagonal directions) can be selected to convolve the acquired image. As shown Figure 4 As shown, Figure 4 The first one on the left is the trainable Sobel operator in the horizontal direction, the second one on the left is the trainable Sobel operator in the vertical direction, and the two on the right are trainable Sobel operators in different diagonal directions. The input image can obtain a four-channel edge information feature map corresponding to the above four Sobel operators through the edge information map module.

[0027] The feature fusion module includes the Cross-Attention Fusion Module (CAFM), see Figure 1 As shown, the cross attention fusion module fuses the outputs of the first-level SConv modules in the first encoder module and the second encoder module. Specifically, as the two input features Fc1 and Fc2 of the cross attention fusion module, both have the shape of C × H × W ,in, C is the number of channels of the feature map, H is the height of the feature map, W is the width of the feature map. Specifically, Figure 5 The module diagram of the cross attention fusion module in the critical dimension scanning electron microscope image processing method according to one embodiment of the present invention is schematically shown. Figure 6 The schematic diagram shows the process of processing the input features of the cross attention fusion module. Figure 5 and Figure 6 As shown, the process can be specifically implemented as including the following steps: Step S21: Processing the first input feature and the second input feature according to the channel attention mechanism of the cross attention fusion module to obtain a first channel weight coefficient corresponding to the first input feature and a second channel weight coefficient corresponding to the second input feature respectively; Step S22: obtaining a cross matrix according to the first channel weight coefficient and the second channel weight coefficient; Step S23: obtaining a first feature corresponding to the first input feature and a second feature corresponding to the second input feature according to the cross matrix, the first input feature and the second input feature; Step S24: Processing the first feature and the second feature according to the spatial attention mechanism of the cross attention fusion module to obtain a first spatial weight coefficient corresponding to the first feature and a second spatial weight coefficient corresponding to the second feature respectively; Step S25: obtaining a first fused feature corresponding to the first input feature and a second fused feature corresponding to the second input feature according to the first spatial weight coefficient, the second spatial weight coefficient, the first input feature and the second input feature; Step S26: Output the first fusion feature and the second fusion feature through the fully connected layer to obtain a fusion feature.

[0028] Specifically, when executing step S21, the channel attention mechanism of the cross attention fusion module is used to respectively perform the first input feature Fc 1 and the second input feature Fc 2 is processed to obtain the first input feature Fc 1 corresponds to the first channel weight coefficient Mc 1, and get the second input feature Fc 2 corresponds to the second channel weight coefficient Mc 2.

[0029] Then, step S22 is performed to set the first channel weight coefficient Mc 1 and the second channel weight coefficient Mc 2 multiply to get the cross matrix M cross The resulting cross matrix M cross Shape C ×1×1.

[0030] Then execute step S23, according to the cross matrix M cross , the first input feature Fc 1 and the second input feature Fc 2. Get the first input feature Fc 1 corresponds to the first feature F 1 and the second input feature Fc 2 corresponds to the second feature F 2. Specifically, its expression is as follows:

[0031] Get the first feature F 1 and second feature F 2, step S24 is executed to respectively perform the first feature F 1 and second feature F 2 is processed to obtain the first input feature Fc The first spatial weight coefficient corresponding to 1 Ms 1, and get the second input feature Fc 2 corresponds to the second spatial weight coefficient Ms 2.

[0032] Then, step S25 is executed to convert the first spatial weight coefficient Ms 1 and the first input feature Fc 1 and processed to obtain the same feature as the first input Fc 2 corresponds to the first fusion feature F out1 , the second spatial weight coefficient Ms 2 and the second input feature Fc 2 and processed to obtain the second input feature Fc 2 corresponds to the second fusion feature F out2 , add the residual connection to the fusion module. Specifically, its expression is as follows:

[0033] Finally, step S26 is executed to obtain the first fusion feature F out1 and the second fusion feature F out2 Through the fully connected layer output, the optimized image is finally obtained Y Specifically, its expression is as follows:

[0034] in, W is the weight of the output fully connected layer, B is the bias term of the output fully connected layer, W and B The value of is first obtained by random initialization, and then back-propagated B and W Update to confirm.

[0035] For the training of the image processing model of the present invention, the data set used in the training is a data set formed by the original SEM images simulated and generated based on the Thorsos method and the ARTIMAGEN library. Among them, in addition to these original SEM images, the data set also includes noise images corresponding to these original SEM images after noise simulation processing of the original SEM images. Then, when training these model modules, they can be trained based on the original SEM images and the noise images corresponding to them, so that each model module can be more suitable for denoising and optimizing SEM images, and improving the measurement accuracy of critical dimension scanning electron microscope (CD-SEM). Exemplarily, when the original SEM images are simulated and generated based on the Thorsos method and the ARTIMAGEN library, 10,800 original SEM images can be generated. When preprocessing these original SEM images, the data set can be divided into three parts: training set, test set and validation set. The training set is used to train the image processing model, the validation set is used for tuning during the training process, guiding the adjustment of model hyperparameters and preventing overfitting, and the test set is used to finally evaluate the performance of the model after the model training and tuning are completed, and to test the performance of the model on unseen data. The training set, validation set and test set account for 70%, 15% and 15% of the entire data set respectively.

[0036] Figure 7 The following schematically shows the steps of performing noise simulation processing on the original SEM image in the critical dimension scanning electron microscope image processing method according to one embodiment of the present invention, referring to Figure 7 As shown, the step flow can be specifically implemented as including the following steps: Step S31: cropping the original SEM image into non-overlapping images of a preset pixel size; Step S32: calculating the mean of the cropped original SEM image; Step S33: adjusting the contrast of the original image according to the calculated mean value to generate an image with reduced contrast; Step S34: Generate Gaussian noise with a mean of 0 and a standard deviation of σ; Step S35: Add the generated Gaussian noise to the image with reduced contrast to obtain a noisy image.

[0037] Since the size of the partially generated original SEM image is 64×1024, when performing step S31, the 64×1024 SEM image is firstly cropped into non-overlapping SEM images of size 64×64, and then steps S32 to S35 are further performed to obtain a noise image.

[0038] Step S32, calculating the mean of the original SEM image. Taking a single original SEM image as an example, its mean μ The calculation formula is as follows:

[0039] in, H is the height of the original SEM image, W is the width of the original SEM image, I(i , j ) is the pixel value of the original SEM image, i and j are the horizontal and vertical coordinate values ​​of the pixel respectively.

[0040] Calculate the mean of the SEM image μ After that, the step S33 is continued to be executed to adjust the contrast of the original SEM image to generate an image with reduced contrast. Specifically, when reducing the contrast of the original SEM image, the pixel value of the SEM image may be adjusted. I ( i , j ), applying a contrast factor β To calculate the new pixel value I’ ( i , j ). The specific formula is as follows:

[0041] in, β is the contrast factor, and its value range is [0,1]. Preferably, the contrast factor β The value is 0.35, and the visual effect displayed at this time can better simulate the low-dose SEM image.

[0042] Then, step S34 is performed to generate Gaussian noise. The mean of the generated Gaussian noise is 0 and the standard deviation is σ Gaussian noise with standard deviation σ The value of can be set to [40,70], and the visual effect presented can better simulate the low-dose SEM image. The specific distribution expression of the generated Gaussian noise is as follows:

[0043] Wherein, N represents Gaussian distribution.

[0044] Finally, step S35 is performed to add the generated Gaussian noise to the image with reduced contrast (pixel value is I’ ) to obtain a simulated noise image (pixel value is I’’ The specific formula is as follows:

[0045] After that, it is only necessary to generate a corresponding noise image for each SEM image to form a data set for training the edge enhancement module, denoising module and cross-attention fusion module.

[0046] Figure 8 The comparison between the processing results of the critical dimension scanning electron microscope image processing method according to one embodiment of the present invention and the processing results of the conventional processing method is schematically shown. Specifically, when performing the comparison of the processing results, the operating system used is Linux, the CPU model is Core i9-9900k, and the GPU model is NVIDIA GeForce RTX3090. During the training process, the contrast factor β Set to 0.35, standard deviation σ The value is set to [40, 70], the MSE loss function is used as the loss function, the Adam optimizer is selected for gradient descent, the epoch is set to 150, the initial learning rate is set to 0.0001, and the learning rate is reduced to 0.00001 after 130 epochs. The batch size is 64, and the model network is trained based on the Pytorch framework.

[0047] Figure 8 The PSNR shown in the figure is the peak signal-to-noise ratio, and SSIM is the structural similarity. PSNR is used to indicate the quality of the reconstructed image. The unit of PSNR is decibel. The larger the PSNR value, the smaller the distortion of the reconstructed image. Usually, when PSNR>30dB, the reconstructed image is considered to be a high-quality image. The calculation formula is as follows:

[0048] Among them, MAX represents the maximum possible value of the pixel value in the image, and MSE represents the size H × W The mean square error between the original SEM image and the optimized image is expressed as follows:

[0049] in, I ( i , j ) is the pixel value of the original image, K ( i , j ) is the pixel value of the denoised image.

[0050] SSIM measures the similarity between two images by comparing brightness, contrast and structure. Its value range is [-1, 1]. The closer it is to 1, the more similar the two images are. Its expression can be expressed by the following formula:

[0051] in, x and y is the pixel value of the original SEM image and the optimized image, μ x is an image x The average value of μ y is an image y The average value of σ x 2 is an image x The covariance of σ y 2 is an image y The covariance of σ xy is an image x and y The covariance of , represents the structural information. c 1 and c 2 are two constants introduced to avoid the denominator being zero, usually c 1=( k 1 l ) 2 and c 2=( k 2 l ) 2 ,in l is the dynamic range of pixel values, k 1 and k 2 are two very small constants, such as k 1=0.01, k 2=0.03.

[0052] Figure 8 The processing results comparison includes BM3D, WNNM, DnCNN, FFDNet, ADNet, SwinIR and MWDCNN, and the contrast factor β is 0.35, the noise level (standard deviation σ ) are 40, 50, and 60 respectively, and these algorithms are quantitatively compared. Figure 7It can be seen from the peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM) of each algorithm that the PSNR and SSIM of the traditional denoising methods BM3D and WNNM are significantly lower than those of the deep learning method at all noise levels, which shows that the deep learning method has significant advantages in processing high-noise images. Compared with the other five deep learning methods, the method proposed in this paper shows the best denoising effect at all noise levels. Specifically, when the noise level is 40, the PSNR of the method proposed in this paper is improved by 0.5186 dB relative to MWDCNN and the SSIM of SwinIR is improved by 0.0012; when the noise level is 50, the PSNR of the method proposed in this paper is improved by 0.4224dB relative to MWDCNN and the SSIM of SwinIR is improved by 0.0006; when the noise level is 60, the PSNR of the method proposed in this paper is improved by 0.327 dB relative to MWDCNN and the SSIM of SwinIR is improved by 0.0002. In summary, the method proposed in the present invention can achieve higher PSNR and SSIM when processing images with different noise levels, which verifies its effectiveness in the denoising task of low-dose CD-SEM images with low contrast and high noise.

[0053] Fig. 9 The following schematically shows the comparison between the visual effect of the processing result of the critical dimension scanning electron microscope image processing method according to one embodiment of the present invention and the visual effect of the processing result of the conventional processing method. Fig. 9 (a) is the original image, and (b) is the noise level (standard deviation σ ) is 30 and the contrast factor β The noise image is 0.35. (c) and (d) show the denoising effects of the BM3D algorithm and the WNNM algorithm respectively. It can be observed that these two traditional algorithms cause serious image edge blurring and detail loss during the denoising process, while reducing the overall contrast of the image.

[0054] Fig. 9 Figure (e) shows the visual effect of the processing results of the feedforward denoising convolutional neural network (DnCNN). Fig. 9 Figure (f) shows the visual effect of the processing results of the fast and flexible denoising network (FFDNet). Fig. 9 Figure (g) shows the visual effect of the processing results of the attention-guided denoising network (ADNet). Fig. 9 Figure (h) shows the visual effect of the processing results of the SwinIR network. Fig. 9 Figure (i) shows the visual effect of the processing results of the multi-stage image denoising convolutional neural network (MWDCNN). Fig. 9Figure (j) in the figure is a visual display of the processing result of the method of the present invention. It can be seen that the method of the present invention can better retain the edge details of the image and maintain a high image contrast while effectively removing noise. Compared with other denoising methods, the method of the present invention shows obvious advantages in the accuracy of edge restoration, which is particularly important for accurately obtaining edge information in CD-SEM images.

[0055] In order to verify the effectiveness of the denoising method in this paper in improving the metrology of low-dose CD-SEM images, this paper uses the open source software SMILE to obtain the critical dimension (CD, Critical Dimension), line edge roughness (LER, Line Edge Roughness), and line width roughness (LWR, Line Width Roughness) values ​​of the image. By comparing the CD, LER and LWR values ​​of different methods, it is verified that the method of the present invention is effective in improving the accuracy of CD-SEM metrology. Among them, CD usually refers to the minimum line width or gap width in the feature pattern of the lithography process. It is a very important parameter in semiconductor manufacturing because it directly affects the performance and function of the device. LER refers to the random slight changes or irregularities at the edge of the feature pattern, and LER is usually calculated by statistically calculating the deviation of the edge position from the ideal position. It is caused by the non-uniformity in the lithography and etching processes and is an important factor affecting device performance. LWR can represent the variation or irregularity of the line width of a feature pattern. It is closely related to LER because the variation of line width is usually caused by the variation of the edges on both sides. LWR is quantified by measuring the width of multiple locations on a line and calculating the standard deviation or root mean square value of these widths.

[0056] Figures 10 to 12 The results of CD, LWR, and LER values ​​of three randomly selected CD-SEM images processed by the critical dimension scanning electron microscope image processing method of one embodiment of the present invention and the existing traditional processing method are schematically shown. It should be pointed out that in this study, the original pixel setting of N. Chaudhary et al. is abandoned, and the pixel size is set to 1nm per pixel, which is conducive to data statistics when using SMILE. Fig.10 , Fig.11 and Fig.12 It can be clearly seen from the results that noise has a significant impact on the metrology of CD-SEM images. Compared with deep learning methods, traditional methods show obvious disadvantages in accurate metrology of low-dose CD-SEM images. Fig.13 Schematically shows Figures 10 to 12 The line chart comparing the CD, LWR, and LER values ​​of each treatment method. Fig.13It can be clearly seen that the image results restored using the DENet method are more stably close to the data of the original image. This shows that the method proposed in the present invention is effective in obtaining low-dose CD-SEM images, thereby providing a new solution to the problem of photoresist shrinkage.

[0057] The critical dimension scanning electron microscope image processing method of the present invention can transmit the input image to the edge enhancement module and the denoising module in parallel, wherein the denoising module including the SC block enhances the ability to extract the global and local features of the input image; the edge enhancement branch first obtains the edge information of the input image through Sobel convolution, and then extracts the global and local features of the edge information; the feature fusion module is also used to fuse the features of the encoder module and the decoder module of the two branches to make up for the deficiency of the denoising branch in edge feature extraction. Compared with the traditional image denoising algorithm, the algorithm solves the edge blurring problem caused by excessive image smoothing while denoising. The experimental results show that the algorithm achieves good denoising effect in the test image and improves the measurement accuracy of the critical dimension scanning electron microscope (CD-SEM). Furthermore, the critical dimension scanning electron microscope image processing method proposed in this patent provides an effective solution to the challenges brought by photoresist shrinkage, which is of great significance for promoting the development of high numerical aperture extreme ultraviolet lithography technology.

[0058] In some embodiments, an embodiment of the present invention provides a non-volatile computer-readable storage medium, which stores one or more programs including execution instructions, and the execution instructions can be read and executed by an electronic device (including but not limited to a computer, a server, or a network device, etc.) to execute the critical dimension scanning electron microscope image processing method of any of the above embodiments of the present invention.

[0059] In some embodiments, an embodiment of the present invention also provides a computer program product, which includes a computer program stored on a non-volatile computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer executes the critical dimension scanning electron microscope image processing method of any one of the above embodiments.

[0060] In some embodiments, an embodiment of the present invention also provides an electronic device, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the critical dimension scanning electron microscope image processing method of any of the above embodiments.

[0061] In some embodiments, the present invention further provides a storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the critical dimension scanning electron microscope image processing method of any of the above embodiments.

[0062] Fig.14 FIG. 1 is a schematic diagram of the hardware structure of an electronic device for executing a critical dimension scanning electron microscope image processing method provided by another embodiment of the present invention, such as Fig.14 As shown, the device includes: One or more processors 110 and memory 120, Fig.14 A processor 110 is taken as an example.

[0063] The apparatus for executing the critical dimension scanning electron microscope image processing method may further include: an input device 130 and an output device 140 .

[0064] The processor 110, the memory 120, the input device 130 and the output device 140 may be connected via a bus or other means. Fig.14 The example of connecting through bus is taken in the following.

[0065] The memory 120 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as program instructions / modules corresponding to the critical dimension scanning electron microscope image processing method in the embodiment of the present invention. The processor 110 executes various functional applications and data processing of the server by running the non-volatile software programs, instructions and modules stored in the memory 120, that is, the critical dimension scanning electron microscope image processing method in the above method embodiment is implemented.

[0066] The memory 120 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of a critical dimension scanning electron microscope image processing method, etc. In addition, the memory 120 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 120 may optionally include a memory remotely arranged relative to the processor 110, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0067] The input device 130 may receive input digital or character information and generate signals related to user settings and function control of the image processing device. The output device 140 may include a display device such as a display screen.

[0068] The one or more modules are stored in the memory 120, and when executed by the one or more processors 110, the critical dimension scanning electron microscope image processing method in any of the above method embodiments is executed.

[0069] The above product can execute the method provided by the embodiment of the present invention, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not described in detail in this embodiment, please refer to the method provided by the embodiment of the present invention.

[0070] The electronic device of the embodiment of the present invention exists in various forms, including but not limited to: (1) Mobile communication devices: These devices are characterized by their mobile communication functions and their main purpose is to provide voice and data communications. These terminals include: smart phones (such as iPhone), multimedia phones, functional phones, and low-end phones.

[0071] (2) Ultra-mobile personal computer devices: These devices fall into the category of personal computers, have computing and processing capabilities, and generally also have mobile Internet access features. These terminals include: PDA, MID and UMPC devices, such as iPad.

[0072] (3) Portable entertainment devices: These devices can display and play multimedia content. They include audio and video players (such as iPods), handheld game consoles, e-books, as well as smart toys and portable car navigation devices.

[0073] (4) Server: A device that provides computing services. The server consists of a processor, hard disk, memory, system bus, etc. The server is similar to a general computer architecture, but because it needs to provide highly reliable services, it has higher requirements in terms of processing power, stability, reliability, security, scalability, and manageability.

[0074] (5) Other electronic devices with data interaction functions.

[0075] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0076] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution, in essence, or the part that contributes to the relevant technology, can be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiment.

[0077] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A critical dimension scanning electron microscope image processing method, characterized in that: include: Use a pre-trained image processing model to process the input image to obtain an optimized image; Wherein, the pre-trained image processing model includes: A denoising branch, based on a U-Net convolutional neural network architecture, includes a first encoder module and a first decoder module connected in sequence, wherein the first encoder module and the first decoder module both use an SC block module to extract local and global features of the image; The edge enhancement branch is based on the U-Net convolutional neural network architecture, which includes an edge information extraction module, a second encoder module and a second decoder module connected in sequence; A feature fusion module is used to fuse the outputs of each level of the first encoder module and the second encoder module to serve as the inputs of each level of the first encoder module.

2. The critical dimension scanning electron microscope image processing method according to claim 1, characterized in that: The edge information extraction module includes a pre-trained Sobel convolution module, and the Sobel operators used in the Sobel convolution module include a horizontal Sobel operator, a vertical Sobel operator and two diagonal Sobel operators.

3. The critical dimension scanning electron microscope image processing method according to claim 1, characterized in that: The SCblock settings are: Perform a 1×1 convolution on the input feature map and divide it into the first feature map and the second feature map through the Split function; The first feature map is input into the SwinT block to obtain the third feature map, and the second feature map is input into the RConv block to obtain the fourth feature map; The third feature map and the fourth feature map are combined by the Concat function to obtain the fifth feature map, and the number of channels of the fifth feature map is adjusted by a 1×1 convolution layer to match the acquired image; The fifth feature map with adjusted channel number is residually connected to the input feature map to obtain the output feature map.

4. The critical dimension scanning electron microscope image processing method according to claim 1, characterized in that: When the feature fusion module is used to fuse the outputs of each level of the first encoder module and the second encoder module as the inputs of each level of the first encoder module, a cross-attention fusion module is used to fuse the first-level outputs of the first encoder module and the second encoder module.

5. The critical dimension scanning electron microscope image processing method according to claim 4, characterized in that: The cross attention fusion module is used to fuse the first-level outputs of the first encoder module and the second encoder module, comprising: Processing the first input feature and the second input feature according to the channel attention mechanism of the cross attention fusion module to obtain a first channel weight coefficient corresponding to the first input feature and a second channel weight coefficient corresponding to the second input feature, respectively; Obtaining a cross matrix according to the first channel weight coefficient and the second channel weight coefficient; According to the cross matrix, the first input feature and the second input feature, a first feature corresponding to the first input feature and a second feature corresponding to the second input feature are obtained; Processing the first feature and the second feature according to the spatial attention mechanism of the cross attention fusion module, respectively obtaining a first spatial weight coefficient corresponding to the first feature and a second spatial weight coefficient corresponding to the second feature; Obtaining a first fused feature corresponding to the first input feature and a second fused feature corresponding to the second input feature according to the first spatial weight coefficient, the second spatial weight coefficient, the first input feature, and the second input feature; The first fusion feature and the second fusion feature are output through the fully connected layer to obtain the fusion feature.

6. The critical dimension scanning electron microscope image processing method according to claim 1, characterized in that: The first encoder module is composed of SC block modules and SConv modules that are alternately arranged, and the first decoder module is composed of SC block modules and TConv modules that are alternately arranged. The SConv modules at each level in the first encoder module are connected to the TConv modules at each level in the first decoder using jump connections.

7. The critical dimension scanning electron microscope image processing method according to claim 1, characterized in that: The second encoder module is composed of RConv block modules and SConv modules that are alternately arranged, the second decoder module is composed of RConv block modules and TConv modules that are alternately arranged, and the SConv modules at each level in the second encoder module are connected to the TConv modules at each level in the second decoder using jump connections.

8. The critical dimension scanning electron microscope image processing method according to claim 1, characterized in that: The data set used in the training of the image processing model includes the original SEM image and the noise image which is obtained by performing noise simulation processing on the original SEM image and corresponds to the original SEM image one by one.

9. The critical dimension scanning electron microscope image processing method according to claim 8, characterized in that: The noise simulation process includes: The original SEM images are cropped into non-overlapping images of a preset pixel size; Calculate the mean of the cropped original SEM image; The contrast of the original image is adjusted according to the calculated mean value to generate an image with reduced contrast; Generate Gaussian noise with mean 0 and standard deviation σ; The generated Gaussian noise is added to the contrast-reduced image to obtain a noisy image.

10. An electronic device, characterized in that: include: At least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the steps of the critical dimension scanning electron microscope image processing method described in any one of claims 1 to 9.

Images