Distortion correction matching method and system for unsupervised scanning electron microscope images

Abstract

The present invention discloses a distortion correction matching method and system for unsupervised scanning electron microscope images, and belongs to the field of image processing. The SEM image and design layout file of the wafer are obtained, and the design layout file contains the corresponding area in the SEM image; the SEM image is converted into a reference layout style image using an unsupervised SEM pattern position extraction model to obtain a false reference layout carrying the pattern position information in the SEM image, the false reference layout is matched with the design layout file, and a matching layout is generated according to the real layout area matching the SEM image in the design layout file; the deformation map between the false reference layout and the matching layout is calculated using the optical flow method, and the distortion in the SEM image is corrected using the deformation map to obtain a corrected SEM image; the corrected SEM image and the matching layout are used for hot spot detection and contour analysis of the wafer. The present invention solves the field of view distortion problem existing in the wafer SEM image and reduces the matching error.

Description

Technical Field

[0001] The present invention belongs to the field of image processing, and particularly relates to an unsupervised distortion correction matching method and system for scanning electron microscope images. Background Art

[0002] The semiconductor chip manufacturing process is evolving towards smaller dimensions, and lithography machines are also evolving continuously. Nevertheless, the difference between the light source wavelength of such lithography machines and the manufacturing node size may still lead to manufacturing defects, also known as lithography hotspots. These hotspots may affect circuit performance and yield, so all hotspots must be comprehensively detected and identified by dedicated equipment. Die-to-Database (D2DB) technology compares the chip image with the design file to detect defects. Compared with the Die-to-Die (D2D) hotspot detection method that compares the chip image with a reference image, D2DB can effectively shorten the detection time and improve the sensitivity. At the same time, D2DB is also widely used in mask detection, lithography proximity effect correction and optimization processes, etc. D2DB detection relies on the precise alignment of the chip image and the design layout file, and the electron beam detection system, with its high-precision alignment performance, can theoretically check the sites with the same (x, y) coordinates in each chip. Unfortunately, in practice, the coordinates checked can deviate by up to 1um at most, which is crucial for high-density and small-size chips. Worse still, when collecting a large number of scanning electron microscope (SEM) images, the position offset will increase continuously over time. Therefore, on the basis of the alignment of the mechanical positioning system, it is also necessary to further correct the distortion of the SEM image.

[0003] Existing solutions mainly rely on extracting the contours of SEM images and design files, and achieving high-precision alignment by minimizing contour errors or the centroid position errors of the closed figures after contour extraction. However, contour-based methods are affected by factors such as photoresist changes, image distortion, and local pattern changes caused by multiple measurements. At the same time, with the miniaturization of nodes, the diversity and complexity of patterns are increasing, resulting in the rounding of pattern edges and the increase of line edge roughness, making contour extraction more difficult. Although some scholars have proposed using the method of average contour, this greatly increases the detection time and computational complexity, and it is difficult to meet the current rapid wafer detection requirements. With the rapid development of deep learning technology, large neural network models have been widely used in the field of computer vision. In the D2DB task, many scholars have proposed deep learning-based methods. For example, by establishing a CNN model, paired layout and SEM images are used to convert the input layout into an acceptable deformed SEM image to compare with the real SEM image for D2DB inspection, or an improved pix2pix model is used to convert the SEM image into an image in a layout-like style for matching. Although these methods have good effects, they rely on a large number of paired images, and due to the confidentiality of the semiconductor industry, it is very difficult to obtain these data. In addition, data annotation requires the professional knowledge of engineers and a large amount of verification work. To solve this problem, some scholars have proposed using CycleGAN for unpaired SEM image style transfer. This method uses a generator to learn the mapping from SEM images to layout image styles with the help of adversarial loss, making the generated layout images look more real. Another generator maps the synthesized layout images back to the SEM domain, and the cycle consistency loss prompts the reconstructed images to match the input images. However, due to the lack of direct constraints between the synthesized images and the input images, CycleGAN cannot guarantee their consistency in structure and position, nor can it solve the distortion problem in SEM images. Summary of the Invention

[0004] In order to solve the problems that in D2DB on high-density chips, there are inaccurate mechanical positioning, the need for complex manual parameter settings, and field-of-view distortion in SEM images, resulting in low detection efficiency and large errors, the present invention proposes an unsupervised distortion correction and matching method and system for scanning electron microscope images.

[0005] The technical solution adopted by the present invention is as follows:

[0006] In a first aspect, the present invention proposes an unsupervised distortion correction and matching method for scanning electron microscope images, including the following steps:

[0007] Obtain the SEM image of the wafer and the design layout file, where the design layout file contains the corresponding area in the SEM image;

[0008] Convert the SEM image into a reference layout-style image using an unsupervised SEM pattern position extraction model to obtain a fake reference layout carrying the pattern position information in the SEM image, match the fake reference layout with the design layout file, and generate a matching layout according to the real layout area in the design layout file that matches the SEM image; the unsupervised SEM pattern position extraction model is a dual-generator and dual-discriminator network structure based on the CycleGAN model, and an edge contrast learning loss and an HV flip invariance loss are introduced during the training process of the unsupervised SEM pattern position extraction model;

[0009] Calculate the deformation map between the fake reference layout and the matching layout using the optical flow method, and correct the distortion in the SEM image using the deformation map to obtain the corrected SEM image;

[0010] The corrected SEM image and the matching layout are used for hot spot detection and contour analysis of the wafer.

[0011] Further, the generator of the unsupervised SEM pattern position extraction model includes a downsampling layer, 2m residual blocks, and an upsampling layer. The several residual blocks are connected in series between the downsampling layer and the upsampling layer. The downsampling layer and the first m residual blocks serve as the encoder, and the upsampling layer and the last m residual blocks serve as the decoder; the downsampling layer and the upsampling layer are composed of the same number of convolutional layers.

[0012] Further, a global attention module is inserted into the last layer of each residual block.

[0013] Further, the dataset for training the unsupervised SEM pattern position extraction model contains SEM images and pseudo-reference layouts, and the pseudo-reference layouts participate in the training process as weak labels of the SEM images.

[0014] Further, the calculation process of the edge contrast learning loss is as follows:

[0015] Obtain the output result of the SEM image passing through the k-th encoder in the first generator, project the output result into a linear space to obtain an encoded feature vector , where, represents the i-th dimension feature in the encoded feature vector of the SEM image, and N is the dimension of the encoded feature vector;

[0016] Obtain the output result of the pseudo-reference layout passing through the k-th encoder in the second generator, project the output result into a linear space to obtain an encoded feature vector , where, represents the i-th dimension feature in the encoded feature vector of the pseudo-reference layout;

[0017] With ([[]] )Construct positive and negative sample pairs. When i = j, they are positive sample pairs; when i ≠ j, they are negative sample pairs, and calculate the margin contrastive learning loss :

[0018]

[0019]

[0020] Among them, represents the included angle between represents taking the modulus, represents the angular interval penalty factor.

[0021] Furthermore, take the mean of the margin contrastive learning losses corresponding to different k values as the final margin contrastive learning loss.

[0022] Furthermore, the calculation process of the HV flip invariance loss is as follows:

[0023] Horizontally and vertically flip the SEM image, convert the flipped and original SEM images into fake reference layouts through the first generator, then flip a pair of fake reference layouts to the same direction, and calculate the loss of the pair of fake reference layouts flipped to the same direction as the HV flip invariance loss of the SEM image;

[0024] Horizontally and vertically flip the pseudo-reference layout, convert the flipped and original pseudo-reference layouts into fake SEM images through the second generator, then flip a pair of fake SEM images to the same direction, and calculate the loss of the pair of fake SEM images flipped to the same direction as the HV flip invariance loss of the pseudo-reference layout;

[0025] Take the sum of the HV flip invariance loss of the SEM image and the HV flip invariance loss of the pseudo-reference layout as the final HV flip invariance loss.

[0026] Furthermore, adversarial loss, cycle consistency loss, and identity mapping loss are also introduced during the training process of the unsupervised SEM pattern position extraction model.

[0027] Furthermore, before training the unsupervised SEM pattern position extraction model, it also includes a preprocessing step of denoising and contour enhancement for the images in the training dataset.

[0028] In the second aspect, the present invention proposes an unsupervised distortion correction matching system for scanning electron microscope images to implement the above-mentioned unsupervised distortion correction matching method for scanning electron microscope images.

[0029] The beneficial effects of the present invention are:

[0030] (1) The present invention proposes an unsupervised SEM pattern position extraction model (SPPE-GAN), which introduces a new edge contrast loss, HV flip invariance loss, and global context attention mechanism to respectively realize the constraints on comprehensive local, global, and key point information. This model can accurately extract the pattern positions in SEM images and transfer them into reference layout style pictures.

[0031] (2) Based on the SPPE-GAN model obtaining the reference layout style picture, the present invention uses an image matching algorithm to perform image matching on the generated reference layout and the design layout file, and then combines the optical flow method calculation to correct the distortion of individual pixel points in the SEM image, which can effectively handle the distortion problem in the SEM image.

[0032] (3) Compared with the experience matching of senior engineers, the distortion correction matching method proposed by the present invention has achieved a more than 10% improvement in the contour intersection over union ratio. And under the condition of the same matching algorithm, the SPPE-GAN model proposed by the present invention outperforms the general full-supervised methods such as pix2pix and the current advanced unsupervised style transfer technologies in the industry in various indicators. Description of the Drawings

[0033] Figure 1 is a flowchart of the distortion correction matching method for unsupervised scanning electron microscope images;

[0034] Figure 2 is a training schematic diagram of the SPPE-GAN model;

[0035] Figure 3 is a schematic diagram of the definition of contour IOU;

[0036] Figure 4 is a schematic diagram of the generator structure in the SPPE-GAN model;

[0037] Figure 5 is a visualization comparison diagram of the results obtained by the present invention and the comparative method. Detailed Embodiments

[0038] The following description is used to disclose the present invention so that those skilled in the art can implement the present invention.

[0039] The drawings are only schematic diagrams of the present invention and are not necessarily drawn to scale. Some of the block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0040] The flowchart shown in the accompanying drawings is only an exemplary illustration and does not necessarily include all steps. For example, some steps can be further decomposed, while some steps can be combined or partially combined. Therefore, the actual execution order may change according to the actual situation.

[0041] The present invention proposes an unsupervised distortion correction matching method for scanning electron microscope images to accurately align SEM images with reference layout diagrams. First, an unsupervised SEM pattern position extraction model (SPPE-GAN) is introduced. Since the traditional CycleGAN lacks direct constraints on the output and generated images, the SPPE-GAN proposed in the present invention can accurately extract the pattern positions in the SEM through comprehensive constraints of local, global, and key information, and migrate them into reference layout style images. Specifically, the SPPE-GAN model constrains the local information consistency by calculating the contrastive learning loss of the feature blocks of the input SEM image and the generated reference layout. In addition, to transfer the global position information, the model performs horizontal and vertical flips on the input image to enhance the generalization performance. At the same time, a global context attention mechanism GcNet is introduced into the generator, enabling the model to focus on the pattern regions in the input SEM image. Based on the SPPE-GAN model, an image matching algorithm is combined to find the corresponding real layout region of the SEM image. Due to the inevitable distortion in the SEM image, the optical flow method is used to correct the distortion of each pixel point in the SEM image, thereby reducing the inherent distortion. The matched layout and the corrected SEM image pair are used for subsequent hot spot detection and contour analysis.

[0042] As Figure 1 shown, the unsupervised distortion correction matching method for scanning electron microscope images mainly includes the following steps:

[0043] S1, collect a data set, including SEM images, design layout files, and pseudo-reference layouts.

[0044] Specifically, after the GDSII file of the active area (AA) layer is corrected by optical proximity effect, it is prepared into a mask template and lithographed on a 55nm wafer production line. Subsequently, SEM images are manually collected using a Review-SEM machine. On the other hand, the design layout file and the pseudo-reference layout are obtained by cropping the original GDSII file of the AA layer. In this embodiment, the size of the SEM image is 480×480×3. The pseudo-reference layout is manually aligned by engineers, with a size of 480×480×3, closely matching the SEM image, but the SEM distortion cannot be completely eliminated. Therefore, these images are used as weak labels to facilitate the analysis of the matching and distortion correction processes. The design layout file is used to verify the matching accuracy between the SEM image and the pseudo-reference layout, and its size is 960×960×3 to ensure that it contains the corresponding region in the SEM image.

[0045] S2. Dataset preprocessing.

[0046] In this embodiment, to meet the input requirements of the model, the SEM images and pseudo-reference layout sizes are adjusted to 256×256×3. A dataset consisting of 800 pairs of SEM images and pseudo-reference layouts is selected as the training set, and the remaining 200 pairs of images form the test set. Since SEM images and pseudo-reference layouts may contain various forms of complex noise, illumination changes, and interference, traditional contour extraction and pattern matching methods are often difficult to effectively handle. In this embodiment, the BM3D algorithm is used to denoise and enhance the contours of the images in the training set. Those skilled in the art can also use other existing denoising and contour enhancement algorithms.

[0047] S3. Construct an unsupervised SEM pattern position extraction model (SPPE-GAN) and train it.

[0048] The SPPE-GAN model is characterized by its dual generator network and discriminator, aiming to promote the conversion between the SEM image domain and the pseudo-reference layout domain. The training strategy of the SPPE-GAN model involves learning two bidirectional mapping generator networks: : X -> Y for forward conversion, converting SEM images into pseudo-reference layouts. For distinction, the pseudo-reference layouts generated by the generator are called false reference layouts; : Y -> X for reverse adaptation, converting pseudo-reference layouts or false reference layouts into SEM images. Similarly, the SEM images generated by the generator are called false SEM images. Each generator network includes an encoder and a decoder. The discriminator is used to ensure that the target domain images (false reference layouts and false SEM images) generated by the generator maintain their inherent domain characteristics.

[0049] In a specific implementation of the present invention, as Figure 4 shown, the structure of generator comprises downsampling, residual blocks, and upsampling. The structure of generator is the same as that of generator First, downsampling is performed through three convolutional layers with convolutional kernels of 7×7, 3×3, and 3×3 respectively, gradually reducing the spatial dimension and increasing the number of feature channels. Then, six residual blocks are used to enhance the feature learning ability through residual connections. Finally, upsampling is performed through three transposed convolutional layers to gradually restore the spatial dimension and generate the target domain image. The present invention also inserts a global attention (GcNet) module after each residual block to enhance the aggregation of global context information.

[0050] The calculation process of the GcNet module is as follows: First, the input feature map has a size of H×W×C. A 1x1 convolutional layer is used to reduce the number of channels from C to 1, resulting in a feature map of size H×W. Then, the H×W feature map is reshaped into HW×1×1. Next, the HW×1×1 feature map undergoes a softmax operation to generate a normalized weight matrix. The weight matrix is multiplied by the matrix reshaped from the original feature map to C×HW×1, obtaining global context features of size 1×1×C. These global context features then pass through two consecutive 1x1 convolutional layers. The first 1x1 convolutional layer reduces the number of channels from C to C / r, and the second 1x1 convolutional layer increases the number of channels from C / r back to C. Finally, the processed global context features are added to the original input feature map H×W×C to form an enhanced output feature map with the same size of H×W×C. In this way, GcNet can more effectively capture long-range dependencies and improve the richness and discriminability of feature representation.

[0051] As Figure 2 shown, the training of the SPPE-GAN model proposed in the present invention incorporates adversarial loss, cycle consistency loss, identity mapping loss, edge contrast learning loss, and HV flip invariance loss. The overall training objective is as follows:

[0052]

[0053] Among them, is the total loss, are the adversarial loss, cycle consistency loss, identity mapping loss, edge contrast learning loss, and HV flip invariance loss respectively, are the weight hyperparameters of the adversarial loss, cycle consistency loss, identity mapping loss, edge contrast learning loss, and HV flip invariance loss respectively. In this embodiment, the five hyperparameters are set to 1, 2, 1, 3, and 4 respectively.

[0054] The following introduces the detailed information of each loss adopted by the SPPE-GAN model.

[0055] (1) Adversarial loss

[0056] The adversarial loss is applied to two mapping directions. For the generator and its discriminator , the generator and its discriminator , the adversarial losses are respectively expressed as:

[0057]

[0058]

[0059] Among them, represents the expectation of the SEM image, represents the expectation of the pseudo-reference layout. For the generator , its source domain is the SEM image and the target domain is the reference layout; for the generator , its source domain is the reference layout and the target domain is the SEM image; the two generators are designed to generate fake images that are indistinguishable from the images in the target domain, while the discriminator tries to distinguish between the source domain image and the target domain image. The generator tries to minimize the loss between the source domain image and the target domain image, while the discriminator tries to maximize the loss between the source domain image and the target domain image. Therefore, the adversarial loss can be expressed as:

[0060]

[0061] (2) Cycle consistency loss

[0062] is used to enforce bidirectional mapping between unpaired images, thus preserving the image semantic information during the translation process. The cycle consistency loss can be expressed as:

[0063]

[0064] Among them, represents the L1 norm.

[0065] (3) Identity mapping loss

[0066] When providing the source domain image from the target domain as input, in order to constrain the generator to be close to the identity mapping, pixel-level consistency is introduced between the input image and the generated image. The identity mapping loss can be expressed as:

[0067]

[0068] (4) Edge contrast learning loss

[0069] To enhance feature extraction, the present invention extracts feature vectors from the encoder layer of the generator through two projectors. In this embodiment, the projector adopts a multi-layer perceptron (MLP); the feature vectors are regarded as anchor points, the feature vectors at the same spatial position are regarded as positive samples, and the remaining feature vectors are regarded as negative samples. Contrast learning becomes a powerful tool for unsupervised representation learning by pulling positive samples closer and pushing negative samples away.

[0070] In order to construct positive and negative pairs for contrast learning, the present invention encodes the SEM image through the encoder in the generator and selects the output of the nth convolutional layer of the encoder to extract features through the projector to obtain the encoded feature vector , the generator The generated fake reference layout passes through the generator in the encoder, encodes, and also selects the nth convolutional layer to extract features through the projector to obtain the encoded feature vector , and constructs sample pairs for contrastive learning with ( ). When i = j, it is a positive sample pair, and when i ≠ j, it is a negative sample pair. The contrastive learning loss can be expressed as:

[0071]

[0072] where N is the dimension of the encoded feature vector, τ is the temperature parameter, and n is the layer number from which the encoded feature vector is taken from the encoder. In this embodiment, n takes 6, 8, 12, and 16 respectively, and the loss values are calculated 4 times and then the average value is obtained. Contrastive learning can encode domain-invariant features. Therefore, to establish an accurate correspondence between two sets of features, high distinguishability between features is required.

[0073] However, since the original contrastive learning tends to produce smooth transitions between different feature clusters, which may lead to smooth and inaccurate correspondences, the margin contrast loss is introduced. Since is a vector, so , is the angle between. The margin contrast loss adds an additional angular interval penalty m (m = 0.3) to the positive samples to expand the separability of the features, thereby generating more distinct and accurate correspondences. Thus, the above formula can be rewritten as:

[0074]

[0075] (5) HV flip invariance loss

[0076] For , use to represent the horizontal-vertical flip transformation. The image x in the X domain and its horizontally and vertically flipped image are respectively transformed through the generator , and the L1 loss between the generator output and is calculated. At the same time, the L1 loss between and is calculated. The same applies to the mapping . The HV flip invariance loss can be expressed as:

[0077]

[0078]

[0079]

[0080] S4. Input the SEM image preprocessed in step S2 into the generator of the trained SPPE-GAN model , generate a fake reference layout carrying the pattern position information in the SEM image, precisely match the generated fake reference layout with the design layout file to obtain the real layout area corresponding to the SEM image, and obtain the matching layout.

[0081] In a specific implementation of the present invention, after using the SPPE-GAN model to generate a fake reference layout carrying the pattern information in the SEM image, the SIFT algorithm is used to extract the features of the generated fake reference layout and the design layout file, and the FLANN algorithm is combined to achieve the matching, so as to accurately locate the real layout area corresponding to the SEM image in the design layout file and obtain the matching layout.

[0082] S5. Based on the results of S3 and S4, use the optical flow method to correct the SEM image, reduce the inherent distortion of the SEM image, and use the matching layout and the corrected SEM image pair for subsequent hot spot detection and contour analysis.

[0083] After obtaining the matching layout in step S4, due to the non-negligible distortion in the original SEM image, the matching layout and the SEM image cannot be directly used for subsequent hot spot detection and contour analysis, and the SEM image needs to be corrected for distortion. However, for the SEM images collected on the production line, the undistorted reference truth value cannot be obtained. Fortunately, the fake reference layout generated by the SPPE-GAN model carries the pattern position and distortion information in the SEM image. Therefore, the present invention corrects the distortion in the SEM image by calculating the deformation map between the generated fake reference layout and the matching layout.

[0084] Specifically, the process of registering these two images is regarded as an optimization problem, the goal of which is to search for the transformation T that maximizes the similarity between the fake reference layout and the matching layout by optimizing some similarity criteria between the fake reference layout and the matching layout. This optimization can be calculated by gradient descent and ends when the maximum similarity is reached or the maximum number of iterations is reached. The optical flow method is a computer vision technique used to estimate the motion of pixels in an image sequence. The optical flow method is based on the assumption that the brightness of the image remains constant in a short period of time, and the movement of objects will cause changes in the pixel positions in the image. It analyzes the brightness changes between images to infer the motion vector field of pixels, that is, the moving direction and speed of each pixel at different time points.

[0085] In a specific implementation of the present invention, the optical flow information from the generated fake reference layout to the matching layout obtained by using the Farneback dense optical flow method is utilized to obtain a deformation map, and the deformation map is applied to the original SEM image, thereby realizing the distortion correction of the SEM image.

[0086] To quantitatively evaluate the performance of SPPE-GAN, the present invention uses three metrics: FID, area IOU, and contour IOU. When evaluating the quality of the generated fake reference layout, the FID metric is used to evaluate the difference between the fake reference layout and the pseudo-reference layout, the area IOU metric is used to evaluate the difference between the matching layout and the pseudo-reference layout, and the contour IOU metric is used to evaluate the alignment between the matching layout and the corrected SEM image, as well as the alignment between the pseudo-reference layout and the corrected SEM image.

[0087] Among them, FID: This metric is used to calculate the distance between two multivariate Gaussian functions. The Inception network is used to extract the mean and covariance from the translated data and the real data, and its performance is consistent with human judgment. If the translation is correct, the FID value will be lower. Therefore, by calculating the FID between the generated fake reference layout and the pseudo-reference layout, since both the image matching and distortion correction processes are based on the generated reference layout, the FID metric not only reflects the success of image translation but also ensures the authenticity and reliability of the experimental results.

[0088] Area IOU: This metric is used to measure the degree of overlap between two regions or images. Specifically, the area IOU is defined as the ratio of the intersection area to the union area.

[0089] Contour IOU: As Figure 3 , the matching layout or the pseudo-reference layout is edge-enhanced, and at the same time, the SEM image is binarized to obtain its contour, and the intersection over union of the contour of the matching layout or the pseudo-reference layout and the contour of the SEM image is calculated, that is . The larger the contour IOU, the higher the matching degree of the two images. At the same time, to ensure the practical significance of the contour IOU metric, in this embodiment, the results with the FID scores in the stable interval are selected, that is, the average value of the training times from 175 to 200 times.

[0090] Table 1 Experimental Results

[0091]

[0092] As shown in Table 1, this experiment was compared with the commonly used Pix2Pix and TSM models in the field. The results show that the FID of the SPPE-GAN of the present invention is the lowest, indicating that its translation performance is better. This may be because the fully supervised model lacks special attention to the right-angled edges of the patterns in the pixel-level loss calculation, resulting in a lower area IOU. Since the fully supervised model is trained using SEM images and pseudo-reference layout maps matched by senior engineers, the corresponding contour IOU indicators all exceed 40% and perform well. However, the contour IOU of the matched layout maps obtained using the SIFT and FLANN algorithms is relatively low. This may be due to the insufficient quality of the generated pseudo-reference layout maps, resulting in large scaling and position deviations in the matched layout maps.

[0093] In addition, it was also compared with the current state-of-the-art unsupervised style transfer models. It should be noted that CUT is a unilateral GAN model that abandons the dual structure of CycleGAN, and DistanceGAN uses the version with the optimal performance based on CycleGAN as the backbone network. Figure 5 The four columns of data in the middle are the comparison results of the present invention with Pix2pix, TSM, CycleGAN, CUT, DistanceGAN, and DCLGAN. The first row is the generated pseudo-reference layout map, the second row is the rendering of the overlap of the SEM image and the pseudo-reference layout map with both set at 50% transparency, and the third and fourth rows are the enlarged views of the two framed areas in the second row. It can be seen that the registration degree between the pseudo-reference layout map generated by the model of the present invention and the contour of the original SEM image is very high. Larger offsets occur when using the CycleGAN and CUT models. The offsets of DistanceGAN and DCL are mainly around the contour width of the SEM image, tending to align with the inner or outer contour of the SEM image, and the position information cannot be accurately extracted. The model of the present invention also obtained the highest score in terms of area IOU. The contour IOU was calculated using the pseudo-reference layout map and the matched layout map respectively, and the model of the present invention also achieved the best results. In addition, it can be seen that when using the pseudo-reference layout map dataset matched by engineers, compared with the SEM image before correction, the contour IOU between the corrected SEM images generated by CycleGAN, CUT, and DistanceGAN and the pseudo-reference layout map all decreased. This may be because the model failed to extract sufficient position information from the patterns on the SEM image, resulting in the deterioration of the results.

[0094] In this embodiment, an unsupervised distortion correction and matching system for scanning electron microscope images is also provided, and this system is used to implement the above embodiment. Terms such as "module" and "unit" used hereinafter can be a combination of software and / or hardware that can achieve a predetermined function. Although the system described in the following embodiments is preferably implemented in software, implementation in hardware, or a combination of software and hardware is also possible.

[0095] An unsupervised distortion correction and matching system for scanning electron microscope images provided in this embodiment includes:

[0096] A wafer data acquisition module, which is used to acquire the SEM image of the wafer and the design layout file, and the design layout file contains the corresponding area in the SEM image;

[0097] A reference layout matching module, which is used to convert the SEM image into a reference layout style image by using an unsupervised SEM pattern position extraction model, obtain a fake reference layout carrying the pattern position information in the SEM image, match the fake reference layout with the design layout file, and generate a matching layout according to the real layout area in the design layout file that matches the SEM image; the unsupervised SEM pattern position extraction model is a dual-generator and dual-discriminator network structure based on the CycleGAN model, and an edge contrast learning loss and an HV flip invariance loss are introduced during the training process of the unsupervised SEM pattern position extraction model;

[0098] An SEM image correction module, which is used to calculate the deformation map between the fake reference layout and the matching layout by using the optical flow method, and correct the distortion in the SEM image by using the deformation map to obtain the corrected SEM image;

[0099] An application module, which is used to realize hotspot detection and contour analysis of the wafer by using the corrected SEM image and the matching layout.

[0100] For the system embodiment, since it basically corresponds to the method embodiment, the relevant parts can be referred to the partial description of the method embodiment, and the implementation methods of the remaining modules will not be elaborated here. The system embodiments described above are merely illustrative, where the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of the present invention. Those of ordinary skill in the art can understand and implement it without creative work.

[0101] The embodiments of the system of the present invention can be applied to any device with data processing capabilities, and the any device with data processing capabilities can be a device or apparatus such as a computer. The system embodiment can be implemented by software, or by hardware or a combination of software and hardware. Taking software implementation as an example, as a logically meaningful device, it is formed by the processor of any device with data processing capabilities reading the corresponding computer program instructions in the non-volatile memory into the memory and running.

[0102] Obviously, the above-described embodiments and drawings are only some examples of the present application. For those of ordinary skill in the art, the present application can also be applied to other similar situations based on these drawings without creative efforts. Additionally, it can be understood that although the work done during this development process may be complex and time-consuming, for those of ordinary skill in the art, certain design, manufacturing, or production changes based on the technical content disclosed in the present application are only conventional technical means and should not be regarded as insufficient disclosure of the present application. Without departing from the concept of the present application, several modifications and improvements can still be made, and these all fall within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.

Claims

1. A method for distortion correction and matching of unsupervised scanning electron microscope images, characterized in that: The following steps are involved: Acquire a SEM image and a design layout file of the wafer, wherein the design layout file includes a corresponding area in the SEM image; The SEM image is converted into a reference layout style image using an unsupervised SEM pattern position extraction model to obtain a fake reference layout that carries the pattern position information in the SEM image. The fake reference layout is matched with the design layout file, and a matching layout is generated based on the real layout area in the design layout file that matches the SEM image. The unsupervised SEM pattern position extraction model is a dual generator dual discriminator network structure based on the CycleGAN model, and edge contrast learning loss and HV flip invariance loss are introduced in the training process of the unsupervised SEM pattern position extraction model; The calculation process of the edge contrast learning loss is: Get the output result of the SEM image after passing through the k-th layer encoder in the first generator, project the output result into the linear space, and obtain the encoded feature vector S = [s1, s2, ..., s N ], where s i represents the i-th dimension feature in the coded feature vector S of the SEM image, and N is the dimension of the coded feature vector; Get the output result of the pseudo reference layout after the k-th layer encoder in the second generator, project the output result into the linear space, and obtain the encoded feature vector T = [t1, t2, ..., t N ], where t i The i-th dimension feature in the encoding feature vector T representing the pseudo reference layout; (s i ,t j ) Construct positive and negative sample pairs, i = j is a positive sample pair, i ≠ j is a negative sample pair, and calculate the edge contrast learning loss L yx : t i ·s i =|t i ||s i |cos(θ ii ) Among them, θ ii Indicates t i and i The angle between them, |.| represents the modulus, and m represents the angle interval penalty factor; The calculation process of the HV flip invariance loss is: The SEM image is flipped horizontally and vertically, and the SEM images before and after the flipping are converted into a pseudo reference layout through the first generator, and then a pair of pseudo reference layouts are flipped to have the same direction, and the loss of the pair of pseudo reference layouts flipped to have the same direction is calculated as the HV flip invariance loss of the SEM image; The pseudo reference layout is flipped horizontally and vertically, and the pseudo reference layout before and after flipping is converted into a pseudo SEM image through the second generator, and then a pair of pseudo SEM images are flipped to have the same direction, and the loss of the pair of pseudo SEM images flipped to have the same direction is calculated as the HV flip invariance loss of the pseudo reference layout; The sum of the HV flip invariance loss of the SEM image and the HV flip invariance loss of the pseudo reference pattern is taken as the final HV flip invariance loss; The optical flow method is used to calculate the deformation map between the pseudo reference layout and the matching layout, and the deformation map is used to correct the distortion in the SEM image to obtain a corrected SEM image; The corrected SEM image and matching layout are used for hot spot detection and profile analysis of the wafer.

2. The method for distortion correction and matching of unsupervised scanning electron microscope images according to claim 1, characterized in that: The generator of the unsupervised SEM pattern position extraction model includes a downsampling layer, 2m residual blocks, and an upsampling layer. The residual blocks are connected in series between the downsampling layer and the upsampling layer. The downsampling layer and the first m residual blocks serve as encoders, and the upsampling layer and the last m residual blocks serve as decoders. The downsampling layer and the upsampling layer are composed of the same number of convolutional layers.

3. The method for distortion correction and matching of unsupervised scanning electron microscope images according to claim 2, characterized in that: The last layer of each residual block is inserted with a global attention module.

4. The method for distortion correction and matching of unsupervised scanning electron microscope images according to claim 2, characterized in that: The data set used for training the unsupervised SEM pattern position extraction model includes SEM images and pseudo-reference layouts, and the pseudo-reference layouts participate in the training process as weak labels of the SEM images.

5. The method for distortion correction and matching of unsupervised scanning electron microscope images according to claim 1, characterized in that: The edge contrast learning loss corresponding to different k values ​​is averaged as the final edge contrast learning loss.

6. The method for distortion correction and matching of unsupervised scanning electron microscope images according to claim 1, characterized in that: Adversarial loss, cycle consistency loss and identity mapping loss are also introduced in the training process of the unsupervised SEM pattern location extraction model.

7. The method for distortion correction and matching of unsupervised scanning electron microscope images according to claim 1, characterized in that: Before training the unsupervised SEM pattern position extraction model, a preprocessing step of denoising and contour enhancement is also included for the images in the training dataset.

8. An unsupervised scanning electron microscope image distortion correction matching system, characterized in that: include: A wafer data acquisition module, which is used to acquire a SEM image and a design layout file of a wafer, wherein the design layout file includes a corresponding area in the SEM image; A reference layout matching module is used to convert the SEM image into a reference layout style image using an unsupervised SEM pattern position extraction model, obtain a fake reference layout carrying the pattern position information in the SEM image, match the fake reference layout with the design layout file, and generate a matching layout based on the real layout area in the design layout file that matches the SEM image; The unsupervised SEM pattern position extraction model is a dual generator dual discriminator network structure based on the CycleGAN model, and edge contrast learning loss and HV flip invariance loss are introduced in the training process of the unsupervised SEM pattern position extraction model; The calculation process of the edge contrast learning loss is: Get the output result of the SEM image after passing through the k-th layer encoder in the first generator, project the output result into the linear space, and obtain the encoded feature vector S = [s1, s2, ..., s N ], where s i represents the i-th dimension feature in the coded feature vector S of the SEM image, and N is the dimension of the coded feature vector; Get the output result of the pseudo reference layout after the k-th layer encoder in the second generator, project the output result into the linear space, and obtain the encoded feature vector T = [t1, t2, ..., t N ], where t i The i-th dimension feature in the encoding feature vector T representing the pseudo reference layout; (s i ,t j ) Construct positive and negative sample pairs, i = j is a positive sample pair, i ≠ j is a negative sample pair, and calculate the edge contrast learning loss L yx : t i ·s i =|t i ||s i |cos(θ ii ) Among them, θ ii Indicates t i and i The angle between them, |.| represents the modulus, and m represents the angle interval penalty factor; The calculation process of the HV flip invariance loss is: The SEM image is flipped horizontally and vertically, and the SEM images before and after the flipping are converted into a pseudo reference layout through the first generator, and then a pair of pseudo reference layouts are flipped to have the same direction, and the loss of the pair of pseudo reference layouts flipped to have the same direction is calculated as the HV flip invariance loss of the SEM image; The pseudo reference layout is flipped horizontally and vertically, and the pseudo reference layout before and after flipping is converted into a pseudo SEM image through the second generator, and then a pair of pseudo SEM images are flipped to have the same direction, and the loss of the pair of pseudo SEM images flipped to have the same direction is calculated as the HV flip invariance loss of the pseudo reference layout; The sum of the HV flip invariance loss of the SEM image and the HV flip invariance loss of the pseudo reference pattern is taken as the final HV flip invariance loss; A SEM image correction module, which is used to calculate a deformation map between the pseudo reference layout and the matching layout using an optical flow method, and use the deformation map to correct the distortion in the SEM image to obtain a corrected SEM image; An application module is used to realize hot spot detection and contour analysis of the wafer by using the corrected SEM image and the matching layout.

Images