A photolithography alignment image enhancement method based on super-resolution generative adversarial network

Abstract

The present application relates to the field of photolithography alignment and its image processing technology, disclose a kind of photolithography alignment image enhancement method based on super-resolution generative adversarial network, comprising: the establishment and preprocessing of AlignNet-SR data set, overlay image super-resolution reconstruction and photolithography alignment template feature matching and center positioning calculation, carry out the overlay of photolithography machine multiple exposure. Construct photolithography alignment image super-resolution generative adversarial network (AlignGAN), with the adversarial training of generator and discriminator, and in combination with photolithography alignment visual image adaptive histogram equalization processing, to significantly improve the visual quality and resolution of photolithography alignment image. After the center positioning of alignment mark is obtained by template feature matching, the inner contour mark and the outer contour mark in the photolithography image are used for rapid and accurate alignment, which can adapt to various marks, effectively improve the accuracy of overlay alignment, and can more carefully process different regions of the image, thereby avoiding over-enhancement or detail loss.

Description

Technical Field

[0001] This invention relates to the field of image super-resolution technology, specifically to a method for enhancing lithographic alignment images based on super-resolution generative adversarial networks. Background Technology

[0002] With the development of integrated circuits, the demand for more advanced and mature process capabilities is increasing. Integrated circuits require mature and advanced processing solutions. Taking the field of autonomous driving as an example, for actuator control, mature signal-guided control is required, while central computing chips need high-speed processing capabilities to meet the needs of service-oriented operations. Therefore, integrated circuits are a key area of ​​demand and concentrated development for various industries. In this market environment, lithography machines, as key equipment in integrated circuit manufacturing, have huge application demand. With the continuous development of integrated circuits and the growth of demand, lithography machines will continue to play an important role in the future industry, meeting the industry's demand for higher-level process capabilities. In the entire integrated circuit manufacturing process, lithography is the most core and complex process step. Lithography technology uses photoresist, exposes it under a specific light source, and then uses development, etching, and other steps to remove the photoresist. The technology of transferring patterns from a film to a silicon wafer involves photolithography. Each circuit pattern layer requires exposure and a dedicated mask. To ensure precise alignment of each layer, the mask's position must perfectly match the previously exposed pattern. Since each circuit layer has different designs and materials, different masks are needed to achieve unique patterns for each layer. This ensures that the patterns formed on each layer are accurately superimposed, avoiding offset or misalignment, thus guaranteeing the performance and quality of the final chip. The overlay accuracy between the mask and the silicon wafer is one of the core performance indicators of a photolithography machine. Overlay accuracy is achieved through photolithography alignment. Therefore, photolithography alignment technology, as one of the three core technologies of photolithography, plays a crucial role in photolithography production. Improving the accuracy of photolithography alignment directly improves overlay accuracy, thereby improving product quality. Simultaneously, increasing the speed and efficiency of photolithography alignment also improves product productivity.

[0003] Overlay alignment technology is an alignment technique used in multiple exposures of proximity lithography machines. It widely adopts video image alignment methods based on geometric patterns. With the development of semiconductors, various overlay marks have emerged. Some lithography processes no longer design dedicated alignment marks, but directly use the lithographic pattern or the outline of the silicon wafer as alignment marks for alignment. The overlay alignment algorithm determines the key factor of the overlay accuracy of fully automatic exposure machines. However, after the wafer is coated and baked, interference such as low contrast, structural blurring, and nonlinear distortion of the alignment mark image directly leads to low positioning accuracy.

[0004] Based on this, the present invention provides a method for enhancing lithographic alignment images based on super-resolution generative adversarial networks. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a lithography alignment image enhancement method based on super-resolution generative adversarial networks. It possesses deep learning-based methods for alignment image denoising and blind image super-resolution reconstruction. By constructing an AlignGAN super-resolution generative adversarial network for lithography alignment images based on a 3D DenseNet fusion SE block (Squeeze-and-Excitation), consisting of a generator network and a discriminator network, and combining it with adaptive histogram equalization processing of the lithography alignment visual image, it effectively reduces various interferences during the overlay alignment process. This achieves the advantages of super-resolution and high-definition enhancement of alignment marks, solving the problem of low positioning accuracy caused by interference such as low contrast, structural blurring, and nonlinear distortion in the alignment mark image after wafer coating and baking, as mentioned in the background technology.

[0006] This invention provides the following technical solution: a method for enhancing lithographic alignment images based on super-resolution generative adversarial networks, comprising the following steps:

[0007] S1: The AlignNet-SR dataset was created by taking high-resolution overlay mark images and corresponding low-resolution simulation images that cover various overlay mark designs and wafer process conditions.

[0008] S2: Dataset preprocessing, including a series of operations such as pruning and standardization, data cleaning and augmentation, and dataset partitioning;

[0009] S3: Acquisition of template image and overlay image. The image acquisition system acquires the template image before exposure and the overlay image after each exposure.

[0010] S4: Adaptive histogram equalization is applied to process the acquired overlay pattern area. This technology can effectively eliminate the square overlay marks on the wafer after exposure by adjusting the brightness and contrast of the local area, which significantly improves the overlay image quality, thus making the subsequent multiple exposure process more accurate and reliable.

[0011] S5: Following S4, construct an AlignGAN (a generative adversarial network for super-resolution of lithographic aligned images) based on a 3D DenseNet fused SE block (Squeeze-and-Excitation). This network can perform deep learning-based alignment image denoising and blind image super-resolution reconstruction. The introduced compressed excitation SE block can complete adaptive learning of each channel of the feature map, which can not only improve the utilization rate of effective features, but also reduce the impact of network redundancy.

[0012] S6: Template feature matching. After the overlay mark is denoised and enhanced using a deep adversarial network in S5, the cv.matchTemplate() function of OpenCV is used for template matching. After obtaining the best matching position, its center positioning data is calculated. Finally, the motion control system achieves accurate overlay alignment by using the center positioning data of the mark.

[0013] Preferably, the overlay mark image in S1 includes an outer contour mark and an inner contour mark, wherein the outer contour mark is set on a mask, the inner contour mark is set on a wafer, and a low-resolution simulated image is generated by applying lithographic blurring such as downsampling, Gaussian blur and adding noise to the high-resolution image.

[0014] Preferably, the preprocessing operation of the dataset in S2 specifically includes: first, cropping the region of interest (ROI) containing the overlay mark from the original image pair and unifying the size of the image patch and the pixel value of the normalized image; then, after data cleaning, expanding the dataset by using data augmentation techniques such as geometric transformation, noise addition, brightness and contrast adjustment on the obtained image pair; and finally, dividing the preprocessed dataset into training set, validation set and test set according to the proportion.

[0015] Preferably, the application of adaptive histogram equalization technology in step S4 to reduce or eliminate the square overprinting marks on the wafer image after exposure by the mask specifically includes: first, converting the color image to a grayscale image; then applying Gaussian blur to reduce noise in the grayscale image; dividing the grayscale image into multiple non-overlapping and equally sized sub-image blocks, calculating the histogram of each sub-image block to obtain the frequency of the grayscale level, and then calculating the cumulative distribution function to obtain the equalized pixel value; finally, performing bilinear interpolation on each sub-image block to eliminate block artifacts, and then recombining all the enhanced sub-image blocks into a complete image.

[0016] Preferably, in step S5, an AlignGAN super-resolution generative adversarial network for lithographic alignment images based on 3D DenseNet fused SE blocks (Squeeze-and-Excitation) is introduced. Its training data originates from the AlignNet-SR dataset, including processed high-resolution overprinted marker images and their corresponding low-resolution simulated images. After initializing the parameters of the generator G and discriminator D, and defining the loss function and optimizer, the generator G and discriminator D are trained alternately to obtain the reconstructed super-resolution image. The specific training steps are as follows: The generator network takes the generated low-resolution simulated image as input, performs preliminary feature extraction on it through an initial 3D convolutional layer, and uses Leaky... The ReLU activation function enhances its non-linear expressive power. When an image is input into a Dense-SENet unit (set to 16 units) consisting of a 3D DenseNet and SE blocks (Squeeze-and-Excitation), the dimensionality and number of channels of the feature map are first reduced by a bottleneck layer (1x1 convolution) in the dense block. Then, spatial features are extracted using 3D convolution, and an SE block is added after the dense block for channel calibration. Important features are enhanced by weighting, while unimportant features are suppressed. The transition layer generates global features through pooling operations. Each Dense-SENet unit contains multiple 3D convolutional layers and can access the feature maps of all previous layers through dense connections. After multiple units are stacked, dimensionality is further reduced by a bottleneck layer. Finally, a super-resolution image is generated through two upsampling layers and one 3D convolutional layer. The discriminator network consists of stacked 3D convolutional layers, a Leaky ReLU activation function, and a normalization layer (Normalization layer). The system consists of basic units composed of blocks and Dense layers. First, it takes a real high-resolution template image and a generated high-resolution image as input, and converts the images to grayscale to simplify computation. Then, it extracts preliminary and high-level features of the image through four 3D convolutional layers and increases non-linear expressive power through the Leaky ReLU activation function to avoid the problem of neuron death. After the convolutional layers, a batch normalization layer is applied to standardize the features, reduce covariance shift, stabilize and accelerate the training process. Next, a fully connected layer Dense (1024) is used to flatten the extracted features and map them to a fixed-dimensional feature space. The final fully connected layer outputs a scalar, which is converted into a probability value between 0 and 1 through the Sigmoid activation function, representing the probability that the image is a real image. Finally, the discriminant loss is calculated through the cross-entropy loss function, and the backpropagation algorithm is used to update the parameters of the discriminator to improve its ability to distinguish between real and generated images.

[0017] Preferably, in step S6, the OpenCV template matching algorithm (cv.matchTemplate()) is used to match the super-resolution image obtained in step S5 with the average template generated by the marker image obtained by mask overlay. Matching is performed within the region of interest (ROI) of the target image. The normalized correlation coefficient between the template and the target image is calculated to find the position with the highest matching score. By extracting the matching points, the geometric transformation matrix is ​​estimated using the RANS AC algorithm to obtain the center location of the marker and correct the position of the target image, ensuring that the alignment markers on the mask and the wafer are accurately aligned. The effect of template matching and correction can be verified by visualizing the binary alignment mark contour, thereby achieving high-precision lithography alignment. The system sends motion signals to the motion system according to the alignment motion data to make the outer contour marker and the inner contour marker coincide, thereby achieving precise overlay.

[0018] Compared with the prior art, the present invention has the following beneficial effects:

[0019] (1) The present invention introduces adaptive histogram equalization (AHE) in the processing of overlay marks, which significantly improves the contrast and detail display of the image, equalizes the brightness and contrast of the image, weakens or eliminates the square overlay marks on the wafer image after exposure, and provides the necessary prerequisite for the realization of multiple exposures. Compared with traditional histogram equalization (HE), it can process different regions of the image more meticulously, thereby avoiding over-enhancement or loss of detail.

[0020] (2) This invention not only constructs a high-quality AlignNet-SR dataset suitable for overlay marking in the photolithography process, but also enables the deep generative adversarial network to efficiently generate realistic images that are almost indistinguishable from real images after adversarial training. This overcomes the dependence of traditional deep learning models on a large number of training samples and has the characteristics of unsupervised learning. Therefore, it has significant advantages in real-time performance and efficiency in generating realistic and diverse images.

[0021] (3) Compared with traditional super-resolution generative adversarial networks, this invention improves the three-dimensional dense block Dense Net structure by adding a transition layer to reduce the dimensionality of the feature maps in the dense block, thereby reducing the training difficulty of the network. Before the three-dimensional feature maps are passed into the dense block, a bottleneck layer is further added to perform feature map dimensionality reduction, which greatly reduces the feature map dimension, improves training efficiency, and alleviates the problem of computational consumption.

[0022] (4) In order to enable the network to fully learn the image features, a compressed excitation SE block is introduced in the 3D-DenseNet. Considering the characteristics of the SE block, we insert the SE block into the transport layer of 3D-DenseNet. This can filter out some unimportant features and squeeze and excite the effective features of the transport layer. Therefore, introducing the SE block in 3D-DenseNet can not only improve the utilization rate of effective features, but also reduce the impact of network redundancy. Attached Figure Description

[0023] Figure 1 This is a flowchart of the lithographic alignment method based on super-resolution generative adversarial networks of the present invention;

[0024] Figure 2 This is a schematic diagram of the AlignGAN super-resolution generative adversarial network for photolithographic alignment images in this invention.

[0025] Figure 3 This is a schematic diagram of the improved three-dimensional dense block DenseNet structure of the present invention;

[0026] Figure 4 This is a schematic diagram of the compressed excitation SE block structure integrated in this invention. Detailed Implementation

[0027] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0028] Please see Figures 1-4 A method for enhancing lithographic alignment images based on super-resolution generative adversarial networks includes the following steps:

[0029] S1: The AlignNet-SR dataset was created by taking high-resolution overlay mark images and corresponding low-resolution simulation images that cover various overlay mark designs and wafer process conditions.

[0030] S2: Dataset preprocessing, including a series of operations such as pruning and standardization, data cleaning and augmentation, and dataset partitioning;

[0031] S3: Acquisition of template image and overlay image. The image acquisition system acquires the template image before exposure and the overlay image after each exposure.

[0032] S4: Adaptive histogram equalization is applied to process the acquired overlay pattern area. This technology can effectively eliminate the square overlay marks on the wafer after exposure by adjusting the brightness and contrast of the local area, which significantly improves the overlay image quality, thus making the subsequent multiple exposure process more accurate and reliable.

[0033] S5: Following S4, construct an AlignGAN (Generative Adversarial Network) for super-resolution of lithographic aligned images based on a 3D DenseNet fused SE block (Squeeze-and-Excitation). This network can perform deep learning-based alignment image denoising and blind image super-resolution reconstruction. The introduced compressed excitation SE block can complete adaptive learning of each channel of the feature map, which can not only improve the utilization rate of effective features, but also reduce the impact of network redundancy.

[0034] S6: Template feature matching. After the overlay mark is denoised and enhanced using a deep adversarial network in S5, the cv.matchTemplate() function of OpenCV is used for template matching. After obtaining the best matching position, its center positioning data is calculated. Finally, the motion control system achieves accurate overlay alignment by using the center positioning data of the mark.

[0035] The overlay mark image in S1 includes an outer contour mark and an inner contour mark. The outer contour mark is set on the mask, and the inner contour mark is set on the wafer. A low-resolution simulated image is generated by applying lithographic blurring such as downsampling, Gaussian blur and adding noise to the high-resolution image.

[0036] The preprocessing operations of the dataset in S2 specifically include: First, cropping the region of interest (ROI) containing the overlay mark from the original image pairs and unifying the size of the image patches and the pixel values ​​of the normalized images; then, after data cleaning, expanding the dataset by using data augmentation techniques such as geometric transformation, noise addition, brightness and contrast adjustment; and finally, dividing the preprocessed dataset into training set, validation set and test set according to the proportion.

[0037] Specifically, S4 employs adaptive histogram equalization to reduce or eliminate the square overprinting marks on the wafer image after exposure. This includes: first, converting the color image to a grayscale image; then applying Gaussian blur to reduce noise in the grayscale image; dividing the grayscale image into multiple non-overlapping and equally sized sub-image blocks, calculating the histogram for each sub-image block to obtain the frequency of grayscale levels, and then calculating the cumulative distribution function to obtain the equalized pixel value; finally, performing bilinear interpolation on each sub-image block to eliminate block artifacts, and then recombining all the enhanced sub-image blocks into a complete image.

[0038] In S5, an AlignGAN (AlignGAN) super-resolution generative adversarial network for lithographic alignment images is introduced, based on a 3D DenseNet fusion SE block (Squeeze-and-Excitation). Its training data comes from the AlignNet-SR dataset, including processed high-resolution overlay marker images and their corresponding low-resolution simulated images. After initializing the parameters of the generator G and discriminator D, and defining the loss function and optimizer, the generator G and discriminator D are trained alternately to obtain the reconstructed super-resolution image. The specific training steps are as follows: The generator network takes the generated low-resolution simulated image as input, performs preliminary feature extraction through an initial 3D convolutional layer, and uses Leaky... The ReLU activation function enhances its non-linear expressive power. When an image is input into a Dense-SENet unit (set to 16 units) consisting of a 3D DenseNet and SE blocks (Squeeze-and-Excitation), the dimensionality and number of channels of the feature map are first reduced by a bottleneck layer (1x1 convolution) in the dense block. Then, spatial features are extracted using 3D convolution, and an SE block is added after the dense block for channel calibration. Important features are enhanced by weighting, while unimportant features are suppressed. The transition layer generates global features through pooling operations. Each Dense-SENet unit contains multiple 3D convolutional layers and can access the feature maps of all previous layers through dense connections. After multiple units are stacked, dimensionality is further reduced by a bottleneck layer. Finally, a super-resolution image is generated through two upsampling layers and one 3D convolutional layer. The discriminator network consists of stacked 3D convolutional layers, a Leaky ReLU activation function, and a normalization layer (Normalization layer). The system consists of basic units composed of blocks and Dense layers. First, it takes a real high-resolution template image and a generated high-resolution image as input, and converts the images to grayscale to simplify computation. Then, it extracts preliminary and high-level features of the image through four 3D convolutional layers and increases non-linear expressive power through the Leaky ReLU activation function to avoid the problem of neuron death. After the convolutional layers, a batch normalization layer is applied to standardize the features, reduce covariance shift, stabilize and accelerate the training process. Next, a fully connected layer Dense (1024) is used to flatten the extracted features and map them to a fixed-dimensional feature space. The final fully connected layer outputs a scalar, which is converted into a probability value between 0 and 1 through the Sigmoid activation function, representing the probability that the image is a real image. Finally, the discriminant loss is calculated through the cross-entropy loss function, and the backpropagation algorithm is used to update the parameters of the discriminator to improve its ability to distinguish between real and generated images.

[0039] In S6, OpenCV's template matching algorithm (cv.matchTemplate()) is used to match the super-resolution image obtained in S5 with the average template generated from the marker image obtained by mask overlay. Matching is performed within the region of interest (ROI) of the target image. The normalized correlation coefficient between the template and the target image is calculated to find the position with the highest matching score. By extracting the matching points, the geometric transformation matrix is ​​estimated using the RANSAC algorithm to obtain the center location of the marker and correct the position of the target image, ensuring accurate alignment of the alignment markers on the mask and the wafer. The effect of template matching and correction can be verified by visualizing the binary alignment marker contour, thereby achieving high-precision lithography alignment. The system sends motion signals to the motion system based on the alignment motion data to make the outer contour markers and inner contour markers coincide, thereby achieving precise overlay.

[0040] Please see Figure 1 A lithographic alignment method based on super-resolution generative adversarial networks is proposed, which consists of three parts: dataset establishment and preprocessing, reconstruction of overlay marker images, and extraction of alignment marker features and template matching. The specific steps are as follows:

[0041] Step 1: Establishing the AlignNet-SR dataset. High-precision microscopic imaging equipment is used to acquire unblurred original images of overlay markings to ensure that they have complete edge details and texture information. To meet the training requirements of GAN, at least thousands of high-resolution images are acquired, covering various overlay marking designs and wafer process conditions. Low-resolution images are generated from high-resolution images by downsampling, Gaussian blurring, and adding noise to simulate the blurring phenomenon caused by optical system imaging distortion, equipment vibration, and optical diffraction after exposure as much as possible. In addition, for training the generative adversarial network, the acquired high-resolution images and the generated low-resolution images must be strictly paired.

[0042] Step 2: Perform preprocessing steps on the dataset, including image cropping and standardization, data cleaning and enhancement, and dataset partitioning;

[0043] Specifically, firstly, regions of interest (ROIs) containing the overprinted markings are cropped from the original image pairs and uniformly adjusted to fixed-size image patches. The pixel values ​​are then normalized to the [0, 1] interval to enhance the stability of model training. Next, to ensure the quality of the training data, low-quality samples such as excessive blurring, missing markings, or severe background interference are removed during the imaging process. Data augmentation techniques, including geometric transformations, random noise addition, and brightness and contrast adjustments, are used to expand the dataset, increasing sample diversity and improving the model's generalization ability. During augmentation, it is crucial to ensure that the augmented image pairs maintain their registration relationship. Finally, the preprocessed dataset is divided proportionally into training, validation, and test sets: 70% is used as the training set for model optimization; 15% as the validation set for parameter tuning; and 15% as the test set for final performance evaluation.

[0044] Step 3: Using high-precision microscopic imaging equipment, acquire template images and overlay images. Template images refer to the images before exposure, while overlay images are the images after exposure. The acquired images cover the inner contour marks on the wafer and the outer contour marks on the mask.

[0045] Step 4: Perform adaptive histogram equalization on the acquired overprinted mark image to enhance the contrast of the exposed areas, as detailed below:

[0046] (1) Convert a color image to a grayscale image:

[0047] I gray (x,y)=0.299×R(x,y)+0.587×G(x,y)+0.114×B(x,y)

[0048] In the formula, R(x, y), G(x, y), and B(x, y) represent the red, green, and blue components of the original image at position (x, y), respectively. gray (x, y) is a grayscale image;

[0049] (2) Applying Gaussian blur to reduce noise and weaken high-contrast regions in grayscale images. Gaussian blur is a linear smoothing filter that can be implemented through convolution operations. For grayscale images I gray The Gaussian blur operation (x, y) can be represented as:

[0050]

[0051] In the formula, (i, j) traverses the set of pixels in the neighborhood centered at (x, y). The Gaussian kernel size (i.e., the size of the neighborhood) determines the range of smoothing. By experimenting with different kernel sizes and evaluating the results, the most suitable kernel size is determined to balance the noise reduction effect and the need to preserve details. The standard deviation σ controls the degree of blurring. A larger σ will lead to a stronger blurring effect. Through these steps, noise in the image can be effectively reduced and high-contrast areas can be weakened, making the image smoother and more uniform.

[0052] (3) After dividing the grayscale image into multiple non-overlapping sub-image blocks, calculate the histogram H(i) for each sub-image block. x,y δ(I gray (x,y)-i), obtain the frequency H(i) of the gray level, and then calculate the cumulative distribution function. Finally, the equalized pixel values ​​can be calculated. In the formula, M and N are the width and height of the sub-image patch, and L is the number of gray levels, which is set to 256 here. min It is the minimum non-zero value of CDF. Before applying histogram equalization, the histogram must be cropped to limit the contrast. This is achieved by setting the maximum frequency threshold TTT for each gray level in the histogram. For gray levels that exceed the threshold TTT, their frequencies are distributed to other gray levels. Finally, bilinear interpolation is performed on each sub-image block to eliminate block artifacts. Then, all the enhanced sub-image blocks are recombined into a complete image.

[0053] Step 5: Perform deep learning-based alignment image denoising and blind image super-resolution reconstruction. After performing adaptive histogram equalization of the lithographic alignment visual image in Step 4, construct an AlignGAN generative adversarial network for lithographic alignment image super-resolution based on 3D DenseNet fused SE blocks (Squeeze-and-Excitation) to perform image super-resolution reconstruction.

[0054] (1) The generator network uses Dense-SENet, which is a 3D DenseNet fused with SE blocks, as the basic unit. It is fused with the SE block by adding a transition layer after the 3D convolutional layer in the dense block. In the 3D dense block, due to the use of dense connections, any layer All can access the feature maps of the previous layers 0, 1, ..., l-1 within their dense blocks, the first... Number of input feature maps of the layer It can be represented as Where k0 is the number of channels in the network input layer, and k is the growth rate of the input feature map. The growth rate k significantly affects the number of input feature maps. Each layer in the dense block passes the feature map generated by its k convolutional kernels to the next layer. The number of feature maps will increase as the number of layers in the dense block increases.

[0055] To improve network training efficiency, the growth rate k needs to be reasonably limited to control the number of feature maps passed to the next layer. Therefore, a transition layer is added to reduce the dimensionality of the feature maps in the dense block, thereby reducing the training difficulty of the network. The transition layer in the dense block mainly reduces the feature dimension through pooling operations. In addition, to alleviate the computational cost problem, a bottleneck layer is added before the 3D feature map is passed to the dense block to further reduce the feature map dimensionality, significantly reducing the feature map dimension and improving training efficiency. The improved 3D DenseNet structure is as follows. Figure 3 As shown;

[0056] To enable the network to fully learn image features, a compressed excitation SE block is introduced in 3D DenseNet. The structure of the SE layer is as follows: Figure 4 As shown, SENet works as follows: First, the spatial information of the feature map is compressed into channel information descriptors through global average pooling. Then, a gating mechanism consisting of two fully connected layers is used to obtain channel relationships. Finally, the information in the channels is weighted to recalibrate the original features in the channel dimension. The convolutional output u is compressed through the spatial dimension H×W to generate the channel descriptor Z, where the C-th element Z is... C It is represented as:

[0057]

[0058] Where u C The feature map output by the c-convolutional kernel, in order to obtain channel dependencies, the gating mechanism formed by two fully connected layers can be represented as:

[0059] s = F ex (z,W)=σ(g(z,W))=σ(W2δ(W1z))

[0060] Where δ is the ReLU function. r is the reduction rate of the feature dimension, which is used to weight the output of SENet onto the original features and is expressed as:

[0061]

[0062] Based on the above analysis, SENet first compresses the input, then stimulates it, maps the feature map to a global real number in the channel unit, and finally multiplies the real number with the corresponding input to complete the adaptive learning of each channel of the feature map.

[0063] In this embodiment of the invention, since dense blocks and transition layers in the network generate a large number of feature maps, the SE block after the dense block is fused with the nonlinear transformation function of 3D DenseNet. Considering the characteristics of the SE block, inserting the SE block into the transport layer of 3D-DenseNet can filter out some unimportant features, squeeze and activate the effective features of the transport layer. Therefore, introducing the SE block into 3D-DenseNet can not only improve the utilization rate of effective features, but also reduce the impact of network redundancy. Global average pooling is used as the squeezing operation, and then two fully connected layers are applied to establish the correlation between channels and output the same number of weights as the input features. First, the feature dimension is reduced to 1 / 16 of the input, then it is activated by ReLU, and then returned to the original dimension by another FC layer. Using two FC layers can not only have more nonlinearity to better fit the complex correlation between channels, but also greatly reduce the parameters and computation. Then, an S-shaped gate is used to obtain... The normalized weights between the features are then applied to the features of each channel using a scaling factor.

[0064] (2) The discriminator network structure is as follows: Figure 2 As shown, the discriminator network is mainly composed of stacked basic units consisting of 3D convolutions, Dense layers, normalization (NB), and LeakyReLU. The project constructs a relative discriminator to predict the probability that a real image is relatively more real than a fake image. The relative discriminator can be described as... Where x r and x f Ex represents the real image and the pseudo image in the training data, respectively. r [·] Take the average of all pseudo images in the minimum batch, C(x) is the discriminator output, and σ() is the Sigmoid function.

[0065] The discriminator loss function is:

[0066] The adversarial loss function is:

[0067] Where, x f =G(x) i ), x i Given a low-resolution (LR) image as input, the generator adversarial loss function can be seen to include x r and x f The data can provide more detailed features of the generated alignment marker image and the actual alignment module image;

[0068] Step Six: Following the denoising and enhancement based on deep adversarial networks described earlier, template feature matching begins. First, a region of interest is identified in the target image, and template matching is performed only within this region to improve matching efficiency and accuracy. The normalized correlation coefficient between the template and the target image is calculated to find the position with the highest matching score. By extracting the matching feature point pairs from the template and target images, the geometric transformation matrix is ​​estimated using the RANSAC algorithm to obtain the center location of the marker and correct the position of the target image, ensuring accurate alignment of the alignment markers on the mask and wafer. The effectiveness of template matching and correction can be verified by visualizing the binary alignment marker contours, thereby achieving high-precision lithography alignment. The motion control system calculates the alignment motion data based on the center location data of the markers. The system sends motion signals to the motion system based on the alignment motion data to make the outer contour markers and inner contour markers coincide, thus achieving precise overlay.

[0069] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0070] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for enhancing lithographic alignment images based on super-resolution generative adversarial networks, characterized in that, Includes the following steps: S1: Establishment of the AlignNet-SR dataset, which consists of high-resolution overlay mark images covering various overlay mark designs and wafer process conditions, as well as corresponding low-resolution simulation images; S2: Preprocessing of the dataset, including cropping and standardization, data cleaning and enhancement, and dataset partitioning; S3: Acquisition of template images and overlay images, with the image acquisition system acquiring template images before exposure and overlay images after each exposure; S4: Establishment of an adaptive histogram equalization method to process the acquired overlay graphic regions. This technique can effectively eliminate square overlay marks on the wafer after exposure by adjusting the brightness and contrast of local regions; S5: Following S4, constructing a lithography alignment image super-resolution generative adversarial network (AlignGAN) based on a 3D DenseNet fused SE block (Squeeze-and-Excitation). This network can perform deep learning-based alignment image denoising and blind image super-resolution reconstruction. The introduced compressed excitation SE block can complete adaptive learning of each channel of the feature map; S6: Template feature matching. After noise reduction and enhancement of the overprinted marks based on deep adversarial networks in S5, the cv.matchTemplate() function of OpenCV is used for template matching. After obtaining the best matching position, its center positioning data is calculated. Finally, the motion control system achieves accurate overprinting alignment through the center positioning data of the marks. In S4, adaptive histogram equalization technology is applied to eliminate the square overprinted marks printed on the wafer image by the mask after exposure. Specifically, this includes: first converting the color image to a grayscale image. Gaussian blur is then applied to denoise the grayscale image. After dividing the grayscale image into multiple non-overlapping and equally sized sub-image blocks, the histogram of each sub-image block is calculated to obtain the frequency of grayscale levels. Then, the cumulative distribution function is calculated to obtain the equalized pixel values. Finally, bilinear interpolation is performed on each sub-image block to eliminate block artifacts, and all the enhanced sub-image blocks are then recombined into a complete image.In S5, an AlignGAN super-resolution generative adversarial network for lithographic alignment images is introduced, based on a 3D DenseNet fusion SE block (Squeeze-and-Excitation). Its training data originates from the AlignNet-SR dataset, including processed high-resolution overprinted marker images and their corresponding low-resolution simulated images. After initializing the parameters of the generator G and discriminator D, and defining the loss function and optimizer, the generator G and discriminator D are optimized. Alternating training is used to obtain the reconstructed super-resolution image. The specific training steps are as follows: The generator network takes the generated low-resolution simulated image as input, performs initial feature extraction through an initial 3D convolutional layer, and enhances its non-linear expressive power with the Leaky ReLU activation function. When the image is input into a Dense-SENet unit (set to 16 units) consisting of a 3D DenseNet and SE blocks (Squeeze-and-Excitation), the dimensionality and number of channels of the feature map are first reduced through a bottleneck layer 1x1 convolution in the dense block, then spatial features are extracted using 3D convolution, and an SE block is added after the dense block for channel calibration. Important features are enhanced by weighting, while unimportant features are suppressed. The transition layer generates global features through pooling operations. Each Dense-SENet unit contains multiple 3D convolutional layers and can access the feature maps of all previous layers through dense connections. After multiple units are stacked, dimensionality reduction is further achieved through a bottleneck layer. Finally, the super-resolution image is generated through two upsampling layers and one 3D convolutional layer. The discriminator network consists of stacked 3D convolutional layers, Leaky ReLU activation function, and normalization layer (Normalization layer). The system consists of basic units composed of blocks and Dense layers. First, it takes a real high-resolution template image and a generated high-resolution image as input, converting the images to grayscale to simplify computation. Then, it extracts preliminary and high-level features of the image through four 3D convolutional layers, and uses the Leaky ReLU activation function to increase non-linear expressive power and avoid neuron death problems. After the convolutional layers, a batch normalization layer is applied to standardize the features, reduce covariance shift, stabilize and accelerate the training process. Next, a fully connected Dense (1024) layer flattens the extracted features and maps them to a fixed-dimensional feature space. The final fully connected layer outputs a scalar, which is converted into a probability value between 0 and 1 using the Sigmoid activation function, representing the probability that the image is a real image. Finally, the discriminant loss is calculated using the cross-entropy loss function, and the backpropagation algorithm is used to update the discriminator's parameters, improving its ability to distinguish between real and generated images.

2. The lithographic alignment image enhancement method based on super-resolution generative adversarial networks according to claim 1, characterized in that: The overlay mark image in S1 includes an outer contour mark and an inner contour mark. The outer contour mark is set on a mask, and the inner contour mark is set on a wafer. A low-resolution simulated image is generated by applying downsampling, Gaussian blur and adding noise to the high-resolution image.

3. The lithographic alignment image enhancement method based on super-resolution generative adversarial networks according to claim 1, characterized in that: The preprocessing operations of the dataset in S2 specifically include: first, cropping the region of interest (ROI) containing the overlay mark from the original image pair and unifying the size of the image patch and the pixel values ​​of the normalized image; then, after data cleaning, expanding the dataset by using data augmentation techniques such as geometric transformation, noise addition, brightness and contrast adjustment on the obtained image pair; and finally, dividing the preprocessed dataset into training set, validation set and test set according to the proportion.

4. The lithographic alignment image enhancement method based on super-resolution generative adversarial networks according to claim 1, characterized in that: In step S6, the OpenCV template matching algorithm (cv.matchTemplate()) is used to match the super-resolution image obtained in step S5 with the average template generated by the marker image obtained by mask overlay. Matching is performed within the region of interest (ROI) of the target image. The normalized correlation coefficient between the template and the target image is calculated to find the position with the highest matching score. By extracting the matching points, the geometric transformation matrix is ​​estimated using the RANSAC algorithm to obtain the center location of the marker and correct the position of the target image, ensuring that the alignment marks on the mask and the wafer are accurately aligned. The effect of template matching and correction can be verified by visualizing the binary alignment mark contour, thereby achieving high-precision lithography alignment. The system sends motion signals to the motion system according to the alignment motion data to make the outer contour mark and the inner contour mark coincide, thereby achieving precise overlay.

Images