Unsupervised scanning electron microscope image distortion correction matching method and system

Through the unsupervised SEM pattern position extraction model (SPPE-GAN) and optical flow method, the problems of mechanical positioning inaccurate and image distortion in semiconductor chip manufacturing are solved, efficient image correction and detection are achieved, and detection efficiency and accuracy are improved.

CN119091441AActive Publication Date: 2024-12-06ZHEJIANG UNIV +1
View PDF 9 Cites 0 Cited by

Patent Information

Application Number
CN202411562064.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-11-05
Publication Date
2024-12-06
Estimated Expiration
2044-11-05

AI Technical Summary

Technical Problem

In semiconductor chip manufacturing, there are inaccurate mechanical positioning, complex manual parameter settings and field distortion of scanning electron microscope images on high-density chips, resulting in low detection efficiency and large errors. The existing methods rely on paired images and are difficult to ensure structure and Location consistency.

Method used

Using the unsupervised SEM pattern position extraction model (SPPE-GAN), the dual generator dual discriminator network structure of the CycleGAN model is introduced to edge comparison learning loss and HV flip invariance loss to generate a false reference layout carrying pattern position information, and The optical flow method is used to correct the distortion in SEM images.

Benefits of technology

It realizes the precise extraction of pattern positions in SEM images and corrects them without relying on paired images, improving the contour cross-section ratio by more than 10%, surpassing the full supervision method and the industry's advanced unsupervised style in various indicators. Migration technology.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119091441A_ABST
    Figure CN119091441A_ABST
Patent Text Reader

Abstract

The invention discloses a distortion correction matching method and system for an unsupervised scanning electron microscope image, and belongs to the field of image processing. Obtaining an SEM image and a design layout file of the wafer, wherein the design layout file comprises a corresponding region in the SEM image; converting the SEM image into a reference layout style image by using an unsupervised SEM pattern position extraction model to obtain a false reference layout carrying pattern position information in the SEM image, matching the false reference layout with a design layout file, and generating a matched layout according to a real layout area matched with the SEM image in the design layout file; calculating a deformation graph between the false reference layout and the matching layout by adopting an optical flow method, and correcting distortion in the SEM image by utilizing the deformation graph to obtain a corrected SEM image; the corrected SEM image and the matching layout are used for hot spot detection and contour analysis of the wafer. According to the invention, the problem of field-of-view distortion of the wafer SEM image is solved, and the matching error is reduced.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the field of image processing, and in particular relates to a distortion correction matching method and system for unsupervised scanning electron microscope images. Background Art

[0002] Semiconductor chip manufacturing processes continue to evolve towards smaller sizes, and lithography machines are also constantly evolving. Despite this, the difference between the wavelength of the lithography machine's light source and the size of the manufacturing node can still lead to manufacturing defects, also known as lithography hotspots. These hotspots can affect circuit performance and yield, so all hotspots must be fully detected and identified by dedicated equipment. Die-to-Database (D2DB) technology compares chip images with design files to detect defects. Compared with Die-to-Die (D2D), a hotspot detection method that compares chip images with reference images, D2DB can effectively shorten detection time and improve sensitivity. At the same time, D2DB is also widely used in mask detection, lithography proximity effect correction optimization process, etc. D2DB detection relies on the precise alignment of chip images and design layout files, and the electron beam detection system, with its high-precision alignment performance, can theoretically check sites with the same (x, y) coordinates in each chip. But unfortunately, in practice, the coordinates can deviate by up to 1um, which is critical for high-density, small-size chips. Worse still, when collecting a large number of scanning electron microscope (SEM) images, the position offset will continue to increase over time. Therefore, based on the alignment of the mechanical positioning system, further distortion correction of the SEM images is required.

[0003] Existing solutions mainly rely on extracting the contours of SEM images and design files, and achieve high-precision alignment by minimizing contour errors or the center of gravity position errors of closed figures after contour extraction. However, contour-based methods are affected by factors such as photoresist changes, image distortion, and local pattern changes caused by multiple measurements. At the same time, with the miniaturization of nodes, the diversity and complexity of patterns are also increasing, resulting in rounded edges of patterns, increased roughness of line edges, and more difficult contour extraction. Although some scholars have proposed the use of average contour methods, this greatly increases the time and amount of calculation for detection, and it is difficult to adapt to the current needs of fast wafer detection. With the rapid development of deep learning technology, large neural network models have been widely used in the field of computer vision. In the D2DB task, many scholars have proposed methods based on deep learning. For example, by establishing a CNN model, the input layout is converted into an SEM image with acceptable deformation by pairing layouts and SEM images, and compared with the real SEM image for D2DB inspection, or the SEM image is converted into an image with a similar layout style for matching using an improved pix2pix model. Although these methods work well, they rely on a large number of paired images, which are difficult to obtain due to the confidentiality of the semiconductor industry. In addition, data annotation requires the expertise of engineers and a lot of verification work. To address this problem, some scholars have proposed using CycleGAN for unpaired SEM image style transfer. This method uses a generator to learn the mapping of SEM images to layout image styles with the help of adversarial losses, making the generated layout images look more realistic. Another generator maps the synthesized layout image back to the SEM domain, and the cycle consistency loss prompts the reconstructed image to match the input image. However, due to the lack of direct constraints between the synthesized image and the input image, CycleGAN cannot guarantee their consistency in structure and position, nor can it solve the distortion problem in the SEM image. Summary of the invention

[0004] In order to solve the problems faced by D2DB on high-density chips, such as inaccurate mechanical positioning, complex manual parameter settings, and field distortion of SEM images, which lead to low detection efficiency and large errors, the present invention proposes a distortion correction matching method and system for unsupervised scanning electron microscope images.

[0005] The technical solution adopted by the present invention is as follows:

[0006] In a first aspect, the present invention provides an unsupervised scanning electron microscope image distortion correction matching method, comprising the following steps:

[0007] Acquire a SEM image and a design layout file of the wafer, wherein the design layout file includes a corresponding area in the SEM image;

[0008] An unsupervised SEM pattern position extraction model is used to convert a SEM image into a reference layout style image to obtain a fake reference layout carrying pattern position information in the SEM image, the fake reference layout is matched with a design layout file, and a matching layout is generated according to a real layout area in the design layout file that matches the SEM image; the unsupervised SEM pattern position extraction model is a dual generator and dual discriminator network structure based on a CycleGAN model, and edge contrast learning loss and HV flip invariance loss are introduced in the training process of the unsupervised SEM pattern position extraction model;

[0009] The optical flow method is used to calculate the deformation map between the pseudo reference layout and the matching layout, and the deformation map is used to correct the distortion in the SEM image to obtain a corrected SEM image;

[0010] The corrected SEM image and matching layout are used for hot spot detection and profile analysis of the wafer.

[0011] Furthermore, the generator of the unsupervised SEM pattern position extraction model includes a downsampling layer, 2m residual blocks, and an upsampling layer. The several residual blocks are connected in series between the downsampling layer and the upsampling layer. The downsampling layer and the first m residual blocks serve as encoders, and the upsampling layer and the last m residual blocks serve as decoders; the downsampling layer and the upsampling layer are composed of the same number of convolutional layers.

[0012] Furthermore, a global attention module is inserted into the last layer of each residual block.

[0013] Furthermore, the data set used for training the unsupervised SEM pattern position extraction model includes SEM images and pseudo-reference layouts, and the pseudo-reference layouts participate in the training process as weak labels of the SEM images.

[0014] Furthermore, the calculation process of the edge contrast learning loss is:

[0015] Get the output result of the SEM image after passing through the k-th layer encoder in the first generator, project the output result into the linear space, and obtain the encoded feature vector ,in, The encoded feature vector representing the SEM image The i-th dimension feature in , N is the dimension of the encoded feature vector;

[0016] Get the output result of the pseudo reference layout after the k-th layer encoder in the second generator, project the output result into the linear space, and obtain the encoded feature vector ,in, Encoded feature vector representing the pseudo reference layout The i-th dimension feature in ;

[0017] by( ) Construct positive and negative sample pairs, i=j is a positive sample pair, i≠j is a negative sample pair, and calculate the edge contrast learning loss :

[0018]

[0019]

[0020] in, express The angle between Indicates modulus, Represents the angle separation penalty factor.

[0021] Furthermore, the edge contrast learning loss corresponding to different k values ​​is averaged as the final edge contrast learning loss.

[0022] Furthermore, the calculation process of the HV flip invariance loss is:

[0023] The SEM image is flipped horizontally and vertically, and the SEM images before and after the flipping are converted into a pseudo reference layout through the first generator, and then a pair of pseudo reference layouts are flipped to have the same direction, and the loss of the pair of pseudo reference layouts flipped to have the same direction is calculated as the HV flip invariance loss of the SEM image;

[0024] The pseudo reference layout is flipped horizontally and vertically, and the pseudo reference layout before and after flipping is converted into a pseudo SEM image through the second generator, and then a pair of pseudo SEM images are flipped to have the same direction, and the loss of the pair of pseudo SEM images flipped to have the same direction is calculated as the HV flip invariance loss of the pseudo reference layout;

[0025] The sum of the HV flip invariance loss of the SEM image and the HV flip invariance loss of the pseudo-reference layout is taken as the final HV flip invariance loss.

[0026] Furthermore, adversarial loss, cycle consistency loss and identity mapping loss are also introduced in the training process of the unsupervised SEM pattern position extraction model.

[0027] Furthermore, before training the unsupervised SEM pattern position extraction model, a preprocessing step of denoising and contour enhancing the images in the training data set is also included.

[0028] In a second aspect, the present invention proposes an unsupervised scanning electron microscope image distortion correction matching system for implementing the above-mentioned unsupervised scanning electron microscope image distortion correction matching method.

[0029] The beneficial effects of the present invention are:

[0030] (1) This paper proposes an unsupervised SEM pattern position extraction model (SPPE-GAN), which introduces a new edge contrast loss, HV flip invariance loss and global context attention mechanism to respectively realize the constraints of comprehensive local, global and key point information. The model can accurately extract the pattern position in the SEM image and transfer it to the reference layout style image.

[0031] (2) Based on the reference layout style picture obtained by the SPPE-GAN model, the present invention uses an image matching algorithm to perform image matching on the generated reference layout and the design layout file, and then combines the optical flow method to calculate the distortion correction of single pixel points in the SEM image, which can effectively deal with the distortion problem in the SEM image.

[0032] (3) Compared with the experience matching of senior engineers, the distortion correction matching method proposed in the present invention achieved an improvement of more than 10% in contour intersection and union ratio. Under the same matching algorithm conditions, the SPPE-GAN model proposed in the present invention surpassed the commonly used fully supervised methods such as pix2pix in the field and the current advanced unsupervised style transfer technology in the industry in all indicators. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 is a flow chart of the distortion correction matching method for unsupervised scanning electron microscope images;

[0034] Figure 2 This is a training diagram of the SPPE-GAN model;

[0035] Figure 3 It is a schematic diagram of the definition of contour IOU;

[0036] Figure 4 This is a schematic diagram of the generator structure in the SPPE-GAN model;

[0037] Figure 5 It is a visual comparison diagram of the results obtained by the present invention and the comparative method. DETAILED DESCRIPTION

[0038] The following description is used to disclose the present invention so that those skilled in the art can implement the present invention.

[0039] The accompanying drawings are only schematic illustrations of the present invention and are not necessarily drawn to scale. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0040] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the steps. For example, some steps may be decomposed, while some steps may be combined or partially combined, so the actual execution order may change according to the actual situation.

[0041] The present invention proposes an unsupervised scanning electron microscope image distortion correction matching method to accurately align the SEM image with the reference layout. First, an unsupervised SEM pattern position extraction model (SPPE-GAN) is introduced. Since the traditional CycleGAN lacks direct constraints on the output and generated images, the SPPE-GAN proposed in the present invention can accurately extract the pattern position in the SEM and transfer it to the reference layout style image through the comprehensive constraints of local, global and key information. Specifically, the SPPE-GAN model constrains the consistency of local information by calculating the contrastive learning loss of the feature blocks of the input SEM image and the generated reference layout. In addition, in order to transfer the global position information, the model flips the input image horizontally and vertically to enhance the generalization performance. At the same time, the global context attention mechanism GcNet is introduced in the generator, so that the model can focus on the pattern area in the input SEM image. On the basis of the SPPE-GAN model, the image matching algorithm is combined to find the real layout area corresponding to the SEM image. Since SEM images have non-negligible distortion, the optical flow method is used to correct the distortion of a single pixel in the SEM image to reduce the inherent distortion. The matching layout and the corrected SEM image pair are used for subsequent hotspot detection and contour analysis.

[0042] like Figure 1 As shown in FIG. 1 , the distortion correction matching method of the unsupervised scanning electron microscope image mainly includes the following steps:

[0043] S1, collect data sets, including SEM images, design layout files, and pseudo reference layouts.

[0044] Specifically, the GDSII file of the active area (AA) layer is corrected for optical proximity effects, prepared into a mask, and photolithography is performed on a 55nm wafer production line, and then the SEM image is manually collected using a Review-SEM machine. On the other hand, the design layout file and the pseudo-reference layout are obtained by cropping the GDSII file of the original AA layer. In this embodiment, the SEM image size is 480×480×3. The pseudo-reference layout is manually aligned by an engineer and has a size of 480×480×3, which closely matches the SEM image, but the SEM distortion cannot be completely eliminated. Therefore, these images are used as weak labels to facilitate the analysis of the matching and distortion correction process. The design layout file is used to verify the matching accuracy between the SEM image and the pseudo-reference layout. Its size is 960×960×3 to ensure that it contains the corresponding area in the SEM image.

[0045] S2. Dataset preprocessing.

[0046] In this embodiment, in order to meet the input requirements of the model, the size of the SEM image and the pseudo-reference layout is adjusted to 256×256×3. A data set consisting of 800 pairs of SEM images and pseudo-reference layouts is selected as the training set, and the remaining 200 pairs of images constitute the test set. Since SEM images and pseudo-reference layouts may contain various forms of complex noise, illumination changes and interference, traditional contour extraction and pattern matching methods are often difficult to process effectively. This embodiment uses the BM3D algorithm to denoise and contour enhance the images of the training set. Those skilled in the art can also use other existing denoising and contour enhancement algorithms.

[0047] S3. Build and train an unsupervised SEM pattern position extraction model (SPPE-GAN).

[0048] The SPPE-GAN model features a dual generator network and a discriminator, designed to facilitate the translation between the domain of SEM images and the domain of pseudo-reference layouts. The training strategy of the SPPE-GAN model involves learning two bidirectional mapping generator networks: : X -> Y is used for forward conversion, converting the SEM image into a pseudo reference layout. In order to distinguish, the pseudo reference layout generated by the generator is called a false reference layout; : Y -> X is used for reverse adaptation to convert the pseudo reference layout or fake reference layout into SEM images. Similarly, the SEM images generated by the generator are called fake SEM images. Each generator network consists of an encoder and a decoder. The discriminator is used to ensure that the target domain images (fake reference layouts and fake SEM images) generated by the generator maintain their inherent domain characteristics.

[0049] In a specific implementation of the present invention, Figure 4 As shown, the generator The structure consists of downsampling, residual blocks and upsampling, generator Structure and generator The structure is the same. First, downsampling is performed through three convolutional layers with convolution kernels of 7×7, 3×3 and 3×3, which gradually reduce the spatial dimension and increase the feature channel. Then, six residual blocks are used to enhance the feature learning ability through residual connection. Finally, upsampling is performed through three deconvolution layers to gradually restore the spatial dimension and generate the target domain image. The present invention also inserts a global attention (GcNet) module after each residual block to enhance the aggregation of global context information.

[0050] The calculation process of the GcNet module is as follows: First, the input feature map size is H×W×C, and the number of channels is reduced from C to 1 through a 1x1 convolution layer to obtain a feature map of size H×W. Then, the H×W feature map is reshaped to HW×1×1. Next, the HW×1×1 feature map is subjected to a softmax operation to generate a standardized weight matrix, which is multiplied by the matrix reshaped from the original feature map to C×HW×1 to obtain a global context feature of size 1×1×C. These global context features are then passed through two consecutive 1x1 convolution layers. The first 1x1 convolution layer reduces the number of channels from C to C / r, and the second 1x1 convolution layer increases the number of channels from C / r back to C. Finally, the processed global context features are added to the original input feature map H×W×C to form an enhanced output feature map of size H×W×C. In this way, GcNet can more effectively capture long-distance dependencies and improve the richness and discriminability of feature expression.

[0051] like Figure 2 As shown in the figure, the training of the SPPE-GAN model proposed in the present invention integrates adversarial loss, cycle consistency loss, identity mapping loss, edge contrast learning loss and HV flip invariance loss. The overall training objectives are as follows:

[0052]

[0053] in, is the total loss, They are adversarial loss, cycle consistency loss, identity mapping loss, edge contrastive learning loss, and HV flip invariance loss. They are weight hyperparameters of adversarial loss, cycle consistency loss, identity mapping loss, edge contrastive learning loss and HV flip invariance loss. In this embodiment, the five hyperparameters are set to 1, 2, 1, 3, and 4, respectively.

[0054] The details of each loss adopted by the described SPPE-GAN model are presented below.

[0055] (1) Adversarial Loss

[0056] The adversarial loss is applied to both mapping directions, for the generator and its discriminator , generator and its discriminator , the adversarial loss is expressed as:

[0057]

[0058]

[0059] in, represents the expectation of the SEM image, represents the expectation of finding a pseudo reference layout, for the generator , the source domain is the SEM image, and the target domain is the reference layout; for the generator , where the source domain is the reference layout and the target domain is the SEM image; the two generators aim to generate fake images that are completely indistinguishable from the images in the target domain, while the discriminator tries to distinguish between the source domain images and the target domain images. The generator tries to minimize the loss between the source domain images and the target domain images, while the discriminator tries to maximize the loss between the source domain images and the target domain images. Therefore, the adversarial loss can be expressed as:

[0060]

[0061] (2) Cycle consistency loss

[0062] It is used to enforce a bidirectional mapping between unpaired images, thereby preserving image semantic information during translation. The cycle consistency loss can be expressed as:

[0063]

[0064] in, represents the L1 norm.

[0065] (3) Identity Mapping Loss

[0066] When a source domain image from the target domain is provided as input, in order to constrain the generator to be close to the identity mapping, a pixel-level consistency is introduced between the input image and the generated image. The identity mapping loss can be expressed as:

[0067]

[0068] (4) Marginal Contrastive Learning Loss

[0069] In order to enhance feature extraction, the present invention extracts feature vectors from the encoder layer of the generator through two projectors. In this embodiment, the projector uses a multi-layer perceptron (MLP); the feature vectors are regarded as anchor points, feature vectors at the same spatial position are regarded as positive samples, and the remaining feature vectors are regarded as negative samples. Contrastive learning becomes a powerful tool for unsupervised representation learning by pulling positive samples closer and pushing negative samples away.

[0070] In order to construct positive and negative pairs for contrastive learning, the present invention passes the SEM images through the generator Encoder in Encode and select the output of the nth convolutional layer of the encoder through the projector Extract features and obtain encoded feature vectors , the generator The generated fake reference layout is passed through the generator Encoder in Encode and also select the nth convolutional layer through the projector Extract features and obtain encoded feature vectors ,by( ) Construct sample pairs for contrastive learning. When i=j, it is a positive sample pair, and when i≠j, it is a negative sample pair. The contrastive learning loss can be expressed as:

[0071]

[0072] Where N is the dimension of the encoded feature vector, τ is the temperature parameter, and n is the number of layers from which the encoded feature vector is taken. In this embodiment, n is 6, 8, 12, and 16, respectively, and the loss value is calculated four times. Contrastive learning can encode domain-invariant features, so establishing an accurate correspondence between two sets of features requires a high degree of distinguishability between the features.

[0073] However, since the original contrastive learning tends to produce smooth transitions between different feature clusters, which may lead to smooth and inaccurate correspondences, the edge contrast loss is introduced. is a vector, so , for The edge contrast loss adds an additional angle interval penalty m (m = 0.3) to the positive sample to expand the separability of the features, thereby producing a clearer and more accurate correspondence, so that the above formula can be rewritten as:

[0074]

[0075] (5) HV flip invariance loss

[0076] for ,use Represents a horizontal-vertical flip transformation, which flips the image x in the X domain horizontally and vertically. Through the generator Perform the conversion and calculate the generator output and The L1 loss between and The L1 loss between them. For the mapping Similarly, the HV flip invariance loss can be expressed as:

[0077]

[0078]

[0079]

[0080] S4: Input the SEM image preprocessed in step S2 into the generator of the trained SPPE-GAN model , generate a fake reference layout carrying the pattern position information in the SEM image, accurately match the generated fake reference layout with the design layout file, obtain the real layout area corresponding to the SEM image, and obtain the matching layout.

[0081] In a specific implementation of the present invention, after using the SPPE-GAN model to generate a pseudo reference layout carrying the pattern information in the SEM image, the SIFT algorithm is used to extract features from the generated pseudo reference layout and the design layout file, and the FLANN algorithm is combined to achieve matching, so as to accurately locate the real layout area corresponding to the SEM image in the design layout file and obtain a matching layout.

[0082] S5. Based on the results of S3 and S4, the optical flow method is used to correct the SEM image to reduce the inherent distortion of the SEM image, and the matching layout and the corrected SEM image are used for subsequent hotspot detection and contour analysis.

[0083] After the matching layout is obtained in step S4, the matching layout and SEM image cannot be directly used for subsequent hotspot detection and contour analysis due to the non-negligible distortion in the original SEM image, and the SEM image needs to be distorted. However, for the SEM images collected on the production line, it is impossible to obtain their distortion-free benchmark truth. Fortunately, the fake reference layout generated by the SPPE-GAN model carries the pattern position and distortion information in the SEM image. Therefore, the present invention corrects the distortion in the SEM image by calculating the deformation map between the generated fake reference layout and the matching layout.

[0084] Specifically, the process of registering the two images is considered as an optimization problem, whose goal is to search for the pseudo reference layout after transformation T to maximize the similarity with the matching layout by optimizing some similarity criteria between the pseudo reference layout and the matching layout. This optimization can be calculated by gradient descent and ends when the maximum similarity is reached or the maximum number of iterations is reached. Optical flow is a computer vision technique used to estimate the movement of pixels in an image sequence. The optical flow method is based on the assumption that the brightness of the image remains constant over a short period of time, while the movement of the object will cause the position of the pixel in the image to change. It infers the motion vector field of the pixel by analyzing the brightness changes between the images, that is, the direction and speed of movement of each pixel at different time points.

[0085] In a specific implementation of the present invention, the optical flow information from the generated pseudo reference layout to the matching layout is obtained based on the Farneback dense optical flow method to obtain a deformation map, and the deformation map is applied to the original SEM image, thereby achieving distortion correction of the SEM image.

[0086] In order to quantitatively evaluate the performance of SPPE-GAN, the present invention uses three indicators: FID, area IOU and contour IOU. When evaluating the quality of the generated pseudo reference layout, the FID metric is used to evaluate the difference between the pseudo reference layout and the pseudo reference layout, the area IOU metric is used to evaluate the difference between the matching layout and the pseudo reference layout, and the contour IOU metric is used to evaluate the alignment between the matching layout and the corrected SEM image, as well as the alignment between the pseudo reference layout and the corrected SEM image.

[0087] Among them, FID: This indicator is used to calculate the distance between two multivariate Gaussian functions. The Inception network is used to extract the mean and covariance from the translated data and the real data, and its performance is consistent with human judgment. If the translation is correct, the FID value will be lower. Therefore, by calculating the FID between the generated fake reference layout and the pseudo reference layout, since the image matching and distortion correction processes are based on the generated reference layout, the FID metric not only reflects the success of the image translation, but also ensures the realism and reliability of the experimental results.

[0088] Area IOU: This metric is used to measure the degree of overlap between two regions or images. Specifically, area IOU is defined as the ratio of the intersection area to the union area.

[0089] Contour IOU: Figure 3 , enhance the edge of the matching layout or pseudo-reference layout, binarize the SEM image to obtain its contour, and calculate the matching layout or pseudo-reference layout contour Contour with SEM images The intersection-and-union ratio of The larger the contour IOU is, the higher the matching degree of the two images is. At the same time, in order to ensure the practical significance of the contour IOU indicator, this embodiment selects the result where the FID score is in a stable range, that is, the average value of the training times is 175 to 200.

[0090] Table 1 Experimental results

[0091]

[0092] As shown in Table 1, this experiment was compared with the commonly used Pix2Pix and TSM models in the field. The results showed that the FID of the SPPE-GAN of the present invention was the lowest, indicating that its translation performance was better. This may be because the fully supervised model lacks special attention to the right-angle edges of the pattern in the pixel-level loss calculation, resulting in a low area IOU. Since the fully supervised model uses SEM images and pseudo-reference layouts matched by senior engineers for training, the corresponding contour IOU indicators are all over 40%, which performs well, but the contour IOU of the matching layouts obtained using the SIFT and FLANN algorithms is low. This may be due to the insufficient quality of the generated pseudo-reference layout, resulting in large scaling and position deviations in the matching layout.

[0093] In addition, it is also compared with the current industry-leading unsupervised style transfer model. It should be noted that CUT is a unilateral GAN ​​model that abandons the dual structure of CycleGAN, while DistanceGAN uses the best-performing version with CycleGAN as the backbone network. Figure 5 The four columns of data are the comparison results between the present invention and Pix2pix, TSM, CycleGAN, CUT, DistanceGAN, and DCLGAN, respectively. The first row is the generated fake reference layout, the second row is the effect diagram of the SEM image and the fake reference layout overlapping with 50% transparency setting, and the third and fourth rows are respectively the enlarged images of the two boxed areas in the second row. It can be seen that the registration between the fake reference layout generated by the model of the present invention and the original SEM image contour is very high. The use of CycleGAN and CUT models has a large offset. The offset of DistanceGAN and DCL is mainly around the contour width of the SEM image, tending to align with the inner or outer contour of the SEM image, and the position information cannot be accurately extracted. The model of the present invention also obtained the highest score in area IOU. The contour IOU was calculated using the pseudo reference layout and the matching layout respectively, and the model of the present invention also achieved the best results. In addition, it can be seen that when using the pseudo-reference layout dataset matched by engineers, the contour IOU of the corrected SEM images generated by CycleGAN, CUT, and DistanceGAN with the pseudo-reference layout is reduced compared to the SEM images before correction. This may be because the model fails to extract enough position information from the pattern on the SEM image, which deteriorates the results.

[0094] In this embodiment, a distortion correction matching system for unsupervised scanning electron microscope images is also provided, which is used to implement the above embodiment. The terms "module", "unit", etc. used below can implement a combination of software and / or hardware for predetermined functions. Although the system described in the following embodiments is preferably implemented in software, it is also possible to implement hardware, or a combination of software and hardware.

[0095] This embodiment provides an unsupervised scanning electron microscope image distortion correction matching system, including:

[0096] A wafer data acquisition module, which is used to acquire a SEM image and a design layout file of a wafer, wherein the design layout file includes a corresponding area in the SEM image;

[0097] A reference layout matching module is used to convert a SEM image into a reference layout style image using an unsupervised SEM pattern position extraction model, obtain a fake reference layout carrying pattern position information in the SEM image, match the fake reference layout with a design layout file, and generate a matching layout based on a real layout area in the design layout file that matches the SEM image; the unsupervised SEM pattern position extraction model is a dual generator and dual discriminator network structure based on a CycleGAN model, and introduces edge contrast learning loss and HV flip invariance loss in the training process of the unsupervised SEM pattern position extraction model;

[0098] A SEM image correction module, which is used to calculate a deformation map between the pseudo reference layout and the matching layout using an optical flow method, and use the deformation map to correct the distortion in the SEM image to obtain a corrected SEM image;

[0099] An application module is used to realize hot spot detection and contour analysis of the wafer by using the corrected SEM image and the matching layout.

[0100] For the system embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment, and the implementation methods of the remaining modules will not be repeated here. The system embodiment described above is only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of the present invention. Ordinary technicians in this field can understand and implement it without paying creative work.

[0101] The embodiments of the system of the present invention can be applied to any device with data processing capabilities, and the device with data processing capabilities can be a device or apparatus such as a computer. The system embodiments can be implemented by software, or by hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, the corresponding computer program instructions in the non-volatile memory are read into the memory by the processor of any device with data processing capabilities and run.

[0102] Obviously, the above-described embodiments and drawings are only some examples of the present application. For those of ordinary skill in the art, the present application can also be applied to other similar situations based on these drawings without creative work. In addition, it is understandable that, although the work done in this development process may be complicated and lengthy, for those of ordinary skill in the art, certain changes in design, manufacturing or production based on the technical content disclosed in the present application are only conventional technical means and should not be regarded as insufficient content disclosed in the present application. Without departing from the concept of the present application, several variations and improvements can also be made, which all belong to the scope of protection of the present application. Therefore, the scope of protection of the present application shall be subject to the attached claims.

Claims

1. An unsupervised matching method for distortion correction of scanning electron microscope images, It is characterized in that The following steps are involved: Acquire a SEM image and a design layout file of the wafer, wherein the design layout file includes a corresponding area in the SEM image; The SEM image is converted into a reference layout style image using an unsupervised SEM pattern position extraction model to obtain a fake reference layout that carries the pattern position information in the SEM image. The fake reference layout is matched with the design layout file, and a matching layout is generated based on the real layout area in the design layout file that matches the SEM image. The unsupervised SEM pattern position extraction model is a dual generator dual discriminator network structure based on the CycleGAN model, and edge contrast learning loss and HV flip invariance loss are introduced in the training process of the unsupervised SEM pattern position extraction model; The optical flow method is used to calculate the deformation map between the pseudo reference layout and the matching layout, and the deformation map is used to correct the distortion in the SEM image to obtain a corrected SEM image; The corrected SEM image and matching layout are used for hot spot detection and profile analysis of the wafer.

2. The method for distortion correction and matching of unsupervised scanning electron microscope images according to claim 1, It is characterized in that The generator of the unsupervised SEM pattern position extraction model includes a downsampling layer, 2m residual blocks, and an upsampling layer. The plurality of residual blocks are connected in series between the downsampling layer and the upsampling layer. The downsampling layer and the first m residual blocks serve as encoders, and the upsampling layer and the last m residual blocks serve as decoders. The downsampling layer and the upsampling layer are composed of the same number of convolutional layers.

3. The method for distortion correction and matching of unsupervised scanning electron microscope images according to claim 2, It is characterized in that The last layer of each residual block is inserted with a global attention module.

4. The method for distortion correction and matching of unsupervised scanning electron microscope images according to claim 2, It is characterized in that The data set used for training the unsupervised SEM pattern position extraction model includes SEM images and pseudo-reference layouts, and the pseudo-reference layouts participate in the training process as weak labels of the SEM images.

5. The method for distortion correction and matching of unsupervised scanning electron microscope images according to claim 4, It is characterized in that The calculation process of the edge contrast learning loss is: Get the output result of the SEM image after passing through the k-th layer encoder in the first generator, project the output result into the linear space, and obtain the encoded feature vector ,in, The encoded feature vector representing the SEM image The i-th dimension feature in , N is the dimension of the encoded feature vector; Get the output result of the pseudo reference layout after the k-th layer encoder in the second generator, project the output result into the linear space, and obtain the encoded feature vector ,in, Encoded feature vector representing the pseudo reference layout The i-th dimension feature in ; by( ) Construct positive and negative sample pairs, i=j is a positive sample pair, i≠j is a negative sample pair, and calculate the edge contrast learning loss : ; ; in, express The angle between Indicates modulus, Represents the angle separation penalty factor.

6. The method for distortion correction and matching of unsupervised scanning electron microscope images according to claim 5, It is characterized in that The edge contrast learning loss corresponding to different k values ​​is averaged as the final edge contrast learning loss.

7. The method for distortion correction and matching of unsupervised scanning electron microscope images according to claim 4, It is characterized in that The calculation process of the HV flip invariance loss is: The SEM image is flipped horizontally and vertically, and the SEM images before and after the flipping are converted into a pseudo reference layout through the first generator, and then a pair of pseudo reference layouts are flipped to have the same direction, and the loss of the pair of pseudo reference layouts flipped to have the same direction is calculated as the HV flip invariance loss of the SEM image; The pseudo reference layout is flipped horizontally and vertically, and the pseudo reference layout before and after flipping is converted into a pseudo SEM image through the second generator, and then a pair of pseudo SEM images are flipped to have the same direction, and the loss of the pair of pseudo SEM images flipped to have the same direction is calculated as the HV flip invariance loss of the pseudo reference layout; The sum of the HV flip invariance loss of the SEM image and the HV flip invariance loss of the pseudo-reference layout is taken as the final HV flip invariance loss.

8. The method for distortion correction and matching of unsupervised scanning electron microscope images according to claim 1, It is characterized in that Adversarial loss, cycle consistency loss and identity mapping loss are also introduced in the training process of the unsupervised SEM pattern location extraction model.

9. The method for distortion correction and matching of unsupervised scanning electron microscope images according to claim 1, It is characterized in that Before training the unsupervised SEM pattern position extraction model, a preprocessing step of denoising and contour enhancement is also included for the images in the training dataset.

10. An unsupervised matching system for distortion correction of scanning electron microscope images. It is characterized in that include: A wafer data acquisition module, which is used to acquire a SEM image and a design layout file of a wafer, wherein the design layout file includes a corresponding area in the SEM image; A reference layout matching module is used to convert the SEM image into a reference layout style image using an unsupervised SEM pattern position extraction model, obtain a fake reference layout carrying the pattern position information in the SEM image, match the fake reference layout with the design layout file, and generate a matching layout based on the real layout area in the design layout file that matches the SEM image; The unsupervised SEM pattern position extraction model is a dual generator dual discriminator network structure based on the CycleGAN model, and edge contrast learning loss and HV flip invariance loss are introduced in the training process of the unsupervised SEM pattern position extraction model; A SEM image correction module, which is used to calculate a deformation map between the pseudo reference layout and the matching layout using an optical flow method, and use the deformation map to correct the distortion in the SEM image to obtain a corrected SEM image; An application module is used to realize hot spot detection and contour analysis of the wafer by using the corrected SEM image and the matching layout.

Citation Information

Patent Citations

  • A failure analysis method for a bad point on a wafer

    CN106033171A

  • Cross-modal unsupervised domain adaptive medical image segmentation method

    CN115731178A

  • Network video recovery method for color distortion of social networking platform

    CN115760592A

  • Unsupervised stereo matching method

    CN115830094A

  • Mixed weak supervision wafer SEM defect segmentation method and system

    CN118096799A