Alignment of distorted images

Abstract

A method for determining optimized weights for an encoder and a decoder network, the method comprising: for each test weight of a plurality of test weights, performing the following steps using the encoder and the decoder operating using the test weight: (a) encoding, using the encoder, a reference image and a distorted image into a latent space to form an encoding; (b) decoding, using the decoder, the encoding to form a distortion map indicative of a difference between the reference image and the distorted image; (c) spatially transforming, by the distortion map, the distorted image to obtain an aligned image; (d) comparing the aligned image to the reference image to obtain a similarity measure; and (e) determining a loss function defined at least in part by the similarity measure; wherein the optimized weights are determined as the test weight having the optimized loss function.

Description

[0001] Cross-reference to related applications

[0002] This application claims priority to EP application 20169199.5, filed on April 10, 2020, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This disclosure relates to methods for determining the operation of an encoder-decoder network and for using an encoder-decoder network to align distorted images, particularly for fabrication using a photolithography apparatus. Background Technology

[0004] A photolithography apparatus is a machine that applies a desired pattern onto a substrate, typically onto a target portion of the substrate. Photolithography apparatus can be used, for example, in the manufacture of integrated circuits (ICs). In this case, a patterning apparatus (or mask or photomask) can be used to generate a circuit pattern to be formed on a single layer of the IC. This pattern can be transferred onto a target portion (e.g., a portion of a die, a die, or several dies) on a substrate (e.g., a silicon wafer). The transfer of the pattern is typically achieved by imaging onto a radiation-sensitive material (resist) layer provided on the substrate. Typically, a single substrate will contain a network of continuously patterned adjacent target portions.

[0005] Most semiconductor devices require the formation of multiple patterned layers and their transfer onto a substrate. For the device to function properly, there are typically limitations on the tolerable errors in edge positioning, edge placement errors, or EPE (edge ​​overlap). EPE can arise from errors in the relative positioning of successive layers (known as overlap) or from errors in the size of features (especially critical dimensions or CDs). As the desire to reduce the size of features that can be formed in photolithography (shrinkage) continues, the limitations on EPE become even more stringent.

[0006] Overlap can be caused by various factors in the photolithography process, such as errors in substrate positioning during exposure and aberrations in the projected image. Overlap can also occur during process steps (such as etching) used to transfer patterns onto the substrate. Some of these process steps generate stress within the substrate, leading to localized or global distortion. Forming three-dimensional structures on the substrate (such as those required by recently developed memory types and MEMS) can also cause significant substrate distortion. CD variations can also originate from various causes, including dose or focusing errors. Summary of the Invention

[0007] This disclosure aims to achieve, for example, more precise measurement for use in the manufacturing process of photolithography devices.

[0008] According to an embodiment, a method for determining optimized weights for encoder and decoder networks is provided; the method includes:

[0009] For each of the multiple test weights, the following steps are performed using the encoder and decoder with the test weights:

[0010] (a) Using an encoder, the reference image and the distorted image are encoded into the latent space to form a code;

[0011] (b) The code is decoded using a decoder to form a distortion map that indicates the difference between the reference image and the distorted image;

[0012] (c) By performing spatial transformation on the distorted image through distortion mapping, an aligned image is obtained;

[0013] (d) Compare the aligned image with the reference image to obtain a similarity measure; and

[0014] (e) Determine a loss function that is at least partially defined by a similarity metric;

[0015] The optimized weights are determined to be test weights with an optimized loss function.

[0016] According to an embodiment, a method for aligning distorted images is provided, the method comprising:

[0017] Use an encoder-decoder network with optimized weights determined according to the above method and perform the following operations:

[0018] An encoder is used to encode the reference image and the distorted image into the latent space to form a code;

[0019] A decoder is used to decode the encoding to form an optimized distortion map; and

[0020] Distortion mapping is used to spatially transform the distorted image to obtain an aligned image.

[0021] According to an embodiment, a method for aligning distorted images is provided, the method comprising:

[0022] Using a pre-trained encoder-decoder network trained to decode from the latent space to produce a distortion map, and for each of several different test latent vectors, the following is performed:

[0023] Based on the test latent vector, a spatial transformation is performed on the distorted image to provide an aligned test image;

[0024] The aligned test image is compared with a reference image;

[0025] A similarity metric is obtained by comparing the aligned image with a reference image;

[0026] Determine the optimized latent vector corresponding to the test latent vector that provides the optimized similarity metric;

[0027] The optimized distortion mapping is determined by decoding the optimized latent vectors using a pre-trained decoder; and

[0028] The distorted image is spatially transformed using an optimized distortion map to output an aligned image.

[0029] According to an embodiment, an inspection tool is provided, the inspection tool including:

[0030] An imaging system is configured as part of an imaging semiconductor substrate; and

[0031] An image analysis system is configured to perform a method for aligning distorted images according to any of the methods described above.

[0032] According to an embodiment, a method for manufacturing a semiconductor substrate is provided, the method comprising the following steps:

[0033] Manufacturing at least a portion of a semiconductor substrate;

[0034] Perform the method described above for aligning distorted images;

[0035] Analyze the aligned image to obtain image metrics; and

[0036] The metric is compared to the target metric, and remedial actions are taken when the metric fails to meet the target metric. Attached Figure Description

[0037] Embodiments will now be described by way of example with reference to the accompanying drawings.

[0038] Figure 1 It describes photolithography equipment and other equipment used to form production facilities for semiconductor devices.

[0039] Figure 2 A flowchart illustrating a method for identifying defects in an image is provided.

[0040] Figure 3 A flowchart illustrating the method for setting the weights of an encoder-decoder network is shown.

[0041] Figure 4 In an exemplary case involving a reference image and a distorted image of a semiconductor substrate, Figure 3 A schematic diagram of the method shown.

[0042] Figure 5 A flowchart illustrating a method for obtaining an aligned image is provided.

[0043] Figure 6 The diagram is depicted for training. Figure 5 The flowchart shows the types of encoder-decoder networks used in the method.

[0044] Figure 7 Depicting according to Figure 6 A diagram of the encoder-decoder network trained using the method shown. Detailed Implementation

[0045] Electronic devices consist of circuits typically formed on a silicon wafer called a substrate, which can be referred to as a semiconductor substrate. Of course, any other suitable material can be used for the substrate. Many circuits can be formed together on the same silicon wafer and are called integrated circuits or ICs. The size of these circuits has been reduced so significantly that many of them can be fitted onto a substrate. For example, the IC chip in a smartphone can be as small as a thumbnail and can include more than 2 billion transistors, each less than 1 / 1000th the size of a human hair.

[0046] Manufacturing these extremely small ICs is a complex, time-consuming, and expensive process, typically involving hundreds of individual steps. Even an error in one step can lead to defects in the finished IC, rendering it useless. Therefore, one goal of the manufacturing process is to avoid such defects in order to maximize the number of functional ICs manufactured in the process; that is, to improve the overall yield of the method.

[0047] One component of improving yield is monitoring the chip manufacturing process to ensure it produces a sufficient number of functional integrated circuits. One way to monitor this process is to inspect the chip circuit structure at various stages of its formation. Inspection can be performed using scanning electron microscopy (SEM), optical inspection systems, and the like. Such systems can be used to image these structures—essentially taking "photographs" of the wafer structure. SEMs are capable of imaging even the smallest components within these structures. The images can be used to determine if the structure is correctly formed in the correct location. If a defect is found, the process can be adjusted, making it less likely for the defect to recur.

[0048] To control errors in photolithography manufacturing processes, such as errors in the relative positions of features in different layers (referred to as overlap) and errors in feature size (referred to as CD variation), these errors need to be measured, for example, using scanning electron microscopy (SEM), optical inspection systems, etc., before corrections can be applied. When using SEM or other inspection systems, an image of the substrate is typically obtained, and the dimensions of features on the substrate are measured from the image. This allows, for example, the determination of CD variation or EPE. However, images obtained by inspection systems (e.g., SEM) are often distorted. Such distortions can include, for example, field-of-view (FOV) distortion due to limitations in electron optical design (similar to, for example, pincushion and barrel distortions in optical systems), and effects due to charging (electron beam-sample interaction, causing, for example, beam bending). As a result of distortion, the measurement of features on the substrate may not be perfectly accurate, and thus the distortion can introduce errors into the measurement. Given small tolerances in the measurement of features on such substrates, this type of error is not desirable and may lead to changes in the manufacturing process that are neither necessary nor too extreme. One current solution for aligning distorted images involves subpixel alignment of the distorted image with a reference image. However, this process is computationally very expensive and therefore unsuitable for handling large numbers of distorted images.

[0049] Some of the methods disclosed in this paper target the use of encoder-decoder networks configured to generate distortion maps, which can be used to transform distorted images into aligned images, wherein the distortion is at least partially removed from the aligned images. Measurements can then be performed using the aligned images, and measurement accuracy can be increased. This can therefore improve the monitoring of chip manufacturing processes.

[0050] The method disclosed in this paper involves determining the weights, such as operating parameters, of an encoder-decoder network such that the network can take a reference image and a distorted image as input and output a distortion map representing the distortion between the distorted and reference images. This method involves iterating over a range of test weights until a distortion map is found that, when applied to the distorted image, returns an aligned image similar to the reference image. This method works on the basis that the aligned image should be similar to the reference image. Another method disclosed in this paper utilizes a pre-trained encoder-decoder network, which can be trained by encoding multiple different input distortion maps into a latent space and decoding the encoding to obtain a decoded distortion map. Again, to ensure that the decoded distortion map is as similar as possible to the input distortion map, the weights of the encoder and decoder can be determined. Once trained, given a reference image and a distorted image, the difference between the reference image and the distorted image transformed by multiple different latent vectors can be found. Once the maximum similarity is determined, the optimal latent vector can be decoded by the trained decoder to return the distortion map. This distortion map can then be applied to the distorted image to return an aligned image.

[0051] Before describing the various embodiments in detail, it is helpful to present example environments in which the techniques disclosed herein can be implemented.

[0052] Figure 1 The illustration depicts a typical layout of a semiconductor manufacturing facility. A photolithography apparatus 100 applies a desired pattern onto a substrate. Photolithography equipment is used, for example, in the manufacture of integrated circuits (ICs). In this case, a patterning apparatus MA (alternatively referred to as a mask or photomask) includes a circuit pattern of features (typically referred to as "product features") formed on a separate layer of the IC. This pattern is transferred to a target portion (e.g., a portion of one or more dies) on the substrate "W" (e.g., a radiation-sensitive material (resist) layer provided on the patterning apparatus 104 by exposing the patterning apparatus 104 onto the substrate "W" (e.g., a silicon wafer). Typically, a single substrate will contain a network of adjacent target portions patterned sequentially.

[0053] Known photolithography equipment illuminates each target portion by irradiating a patterning apparatus, simultaneously positioning the target portion of the substrate at the image position of the patterning apparatus. The irradiated target portion of the substrate is called the "exposure field" or simply the "field". The layout of the field on the substrate is typically a network of adjacent rectangles or other shapes aligned according to a Cartesian two-dimensional coordinate system (e.g., aligned along the X and Y axes, with the two axes orthogonal to each other).

[0054] The requirement of photolithography equipment is to accurately reproduce the desired pattern onto the substrate. The position and size of the applied product features need to be within certain tolerances. Positional errors can cause overlap errors (often referred to as "overlap"). Overlap is the error in placing a first product feature in a first layer relative to a second product feature in a second layer. Photolithography equipment reduces overlap errors by precisely aligning each wafer with a reference before patterning. This is done by measuring the position of alignment marks applied to the substrate. Based on the alignment measurements, the substrate position is controlled during the patterning process to prevent overlap errors exceeding the tolerances. Alignment marks are typically created as part of the product image, forming a reference for measuring overlap. Alternatively, alignment marks from previously formed layers can be used.

[0055] When the applied dose associated with exposure 104 is outside the specifications, errors in the critical dimension (CD) of the product feature may occur. Therefore, the lithography apparatus 100 must be able to precisely control the dose of radiation applied to the substrate. Exposure 104 is controlled by a measuring tool 102 integrated into the lithography apparatus 100. CD errors may also occur when the substrate is not correctly positioned relative to the focal plane associated with the patterned image. Focusing position errors are typically associated with the non-planarity of the substrate surface. The lithography apparatus reduces these focusing position errors by measuring the substrate surface topography using a level sensor before patterning. Substrate height correction is applied during subsequent patterning to ensure proper imaging (focusing) of the patterning apparatus on the substrate.

[0056] To verify overlap and CD errors associated with the photolithography process, the patterned substrate is inspected using metrology equipment 140. Common examples of metrology equipment are scatterometers and scanning electron microscopes. Scatterometers typically measure the characteristics of dedicated metrology targets. These metrology targets represent product features, differing in that they are typically large enough to allow for precise measurement. Scatterometers measure overlap by detecting the asymmetry of the diffraction pattern associated with the overlap metrology target. Critical dimensions are measured by analyzing the diffraction pattern associated with the CD metrology target. The CD metrology target is used to measure the results of the most recently exposed layer. The overlap target is used to measure the difference between the positions of the previous and most recent layers. Electron-beam-based inspection tools such as scanning electron microscopes (SEMs) are well-suited for measuring small overlap and CD values.

[0057] In semiconductor manufacturing, lithography equipment 100 and metrology equipment 140 form part of a "lithography unit" or "lithography cluster." The lithography cluster also includes a coating unit 108 for applying photoresist to a substrate W, a baking unit 110, a developing unit 112 for developing an exposed pattern into a physical resist pattern, an etching station 122, a unit 124 for performing a post-etch annealing step, and possible further processing equipment 126, etc. Metering equipment 140 is configured to inspect the substrate after development at developing unit 112 or after further processing (e.g., etching at etching station 122). The various devices within the lithography unit are controlled by a supervisory control system (SCS), which issues control signals 166 to control the lithography equipment to perform process R via lithography equipment control unit (LACU) 106. The SCS allows operation of different devices, providing maximum throughput and product yield. An important control mechanism is the feedback 146 from metrology equipment 140 to various devices (via the SCS), particularly to lithography equipment 100. Based on the characteristics of metrological feedback, correction actions are determined to improve the processing quality of subsequent substrates. The SCS can be one or more computers that can communicate or not. Fabrication method R can be implemented as one method or multiple independent methods. For example, a method for a process step such as etching can be completely independent of a method used to check the results of that process step (e.g., etching). For example, two or more methods for various steps can be correlated such that one method is adjusted to take into account the performance results of another method on the same or different substrates.

[0058] The performance of lithography equipment is conventionally controlled and calibrated using methods such as Advanced Process Control (APC), as described, for example, in US 2012008127A1. Advanced process control techniques utilize measurements applied to a metrological target on the substrate. The Manufacturing Execution System (MES) schedules the APC measurements and communicates the measurement results to a data processing unit. The data processing unit translates the characteristics of the measurement data into instructions for the lithography equipment. This method is highly effective in suppressing drift phenomena associated with lithography equipment.

[0059] In semiconductor manufacturing, it is crucial to process metrology data into calibration actions performed by processing equipment. Beyond metrology data, the characteristics of individual patterning apparatuses, substrates, processing equipment, and other contextual data may be needed to further optimize the manufacturing process. The framework in which available metrology and contextual data are used to holistically optimize the lithography process is often referred to as part of holistic lithography. For example, contextual data associated with CD errors on a mask can be used to control various devices (lithography equipment, etching stations) so that these CD errors do not affect the manufacturing process yield. Subsequent metrology data can then be used to validate the effectiveness of the control strategy, and further calibration actions can be determined.

[0060] To ensure process window compliance, separate CD and overlap measurements are performed using one or more existing tools and combined into an edge placement error (EPE) budget. Typically, one metrology step can be performed after development (ADI) and another after etching (AEI), and there are inherent difficulties in calibrating two such different measurements to give equivalent results.

[0061] EPE (Extended Physical Surface) is crucial for ensuring the proper functioning of semiconductor devices, as it can affect the availability of sufficient electrical contacts in back-end modules. This makes EPE measurement invaluable for ensuring that the process window accommodates a sufficient EPE budget and for controlling the process to remain within the window.

[0062] The metrology device 140 can acquire an image of the semiconductor substrate to inspect and obtain measurements of the substrate. However, as previously mentioned, the image acquired by the metrology device may be distorted. Figure 2 A flowchart illustrating typical processing of an image for determining defects is shown, including methods for addressing image distortion problems. An image is acquired using a suitable metrology device 140, which may include a SEM, and an image preprocessing step S1 is performed. Image preprocessing can, for example, include any number of suitable preprocessing steps. Preprocessing may include image noise estimation for determining the substrate containing defects, image focus index calculation, and image quality enhancement (EQE). Once this preprocessing is performed, patterning is performed in step S2. Patterning may, for example, include global alignment of the image. After global alignment, local alignment is performed in step S3. This local alignment may include die-to-die (D2D) or die-to-database (D2DB) alignment. This local alignment can correct problems such as distortion or charging. After alignment in step S3, a final step of defect detection is performed in step S4. In step S4, defect detection and classification can be performed. For example, this may include measuring features of the image to derive CD or EPE. These CDs and EPEs can then be compared with predetermined metrics to determine the presence and extent of defects on the substrate.

[0063] Field of view (FOV) distortion and charged artifacts affect the direct measurement and comparison of structures in different parts of the field, or, if the field of view changes, affect the direct measurement and comparison of structures between different images. Existing techniques for performing local alignment involve subpixel alignment of small patches of the image after a global alignment step S2 has been performed. Global alignment can be, for example, on the order of approximately 10 nm. However, this method for local alignment is computationally intensive. Therefore, expensive computing systems are required to perform such processing. Furthermore, in the case of metrology equipment using SEM, the computational requirements are proportional to the number of beams used and quickly become impractical.

[0064] Some of the methods disclosed in this paper aim to address the aforementioned problems associated with local alignment and provide alternative mechanisms for achieving local alignment with lower computational intensity.

[0065] Alignment of distorted images can be performed using an encoder-decoder network, which is configured to encode into and decode from the latent space. This paper discloses a method for determining the optimized weights of the encoder and decoder networks; this method can perform this purpose. Such a method... Figure 3 The flowchart is shown in the figure. Figure 4 The text also schematically depicts... Figure 3 The flowchart outlines some steps. See below for reference. Figure 3 and Figure 4 The method is initiated at step S101. Steps S102-S110 are iterated to find optimized test weights for the encoder-decoder network. In the first iteration of step S102, the encoder and decoder of the network are set to operate with initial test weights. The initial test weights can be selected using various methods. For example, all values ​​can be set to the maximum, minimum, or intermediate range, random values, or values ​​obtained from previous uses of the method. Step S103 involves selecting a reference image and a distorted image for analysis. In step S104, the reference image and the distorted image are encoded into the latent space using the encoder to form an encoding. Following this step, step S105 is performed, where the encoding is decoded by the decoder to form a distortion map representing the difference between the reference image and the distorted image. Once the distortion map is formed, in step S106, the distorted image is spatially transformed using the distortion map to obtain an aligned image.

[0066] Using the obtained aligned image, the loss function is determined in step S107. The loss function is defined at least in part by a similarity metric, which is obtained by comparing the aligned image with a reference image. The loss metric can be obtained by inputting the reference image and the aligned image into a discriminator network, which outputs a value based on the similarity between the images. For example, for similar inputs, the network can output a value close to 0, while for significantly different inputs, the network can output a value close to 1. Of course, any metric can be used.

[0067] The loss function can also be defined, at least in part, by a smoothness metric, which is defined by the smoothness of the distortion map. Therefore, the step of determining the loss function in step S107 can further include determining a smoothness metric for the distortion map. This smoothness metric... Figure 4 The diagram is shown schematically as follows A smoothness metric can be defined by any appropriate measure of the distortion map representing smoothness. In one example, the smoothness metric is defined at least in part by the spatial gradient of the distortion map. Images of semiconductor substrates obtained using SEM are known to exhibit first-order, second-order, and sometimes third-order distortions. Therefore, by optimizing the smoothness of the distortion map, i.e., maximizing its smoothness, the weights of the encoder-decoder network can be set such that an appropriate distortion map can be generated. When compared with a reference image, if the distorted image and the reference image are obtained from different locations on the substrate, higher frequency distortions may be due to actual differences in the measured geometry of the distorted image or due to noise, and therefore it may not be desirable to form a distortion map that corrects for these differences. This ensures that the distortion map indicates the distortion in the image, rather than other differences between the reference image and the distorted image. In some cases, such as if the aligned image and the reference image are obtained from different locations on the substrate, or derived from different modalities (e.g., comparing an SEM image with a mask image, GDSII, or a simulated image), it can be expected that the aligned image may have some differences from the reference image.

[0068] This method can be performed using only a single reference image and a single distorted image. However, to obtain a more robust encoder-decoder network that can more accurately align the distorted image, the process can be repeated for multiple different combinations of the reference image and the distorted image. Therefore, the method may include step S108, which involves determining whether all appropriate image combinations have been analyzed. If not, steps S103 to S107 are repeated for each test weight for multiple different combinations of the reference image and the distorted image. Any combination of the reference image and the distorted image can be used. For example, multiple combinations may include a combination of at least one reference image with multiple different distorted images. Multiple different reference images can be used. For example, multiple combinations may include a combination of multiple different reference images with multiple different distorted images. Increasing the number of combinations of reference images and distorted images can lead to better optimization of the test weights.

[0069] Once these steps have been performed for all appropriate combinations, the method can proceed to step S109, in which the loss function for a given test weight is a combination of the loss functions determined for each of the different combinations of the reference image and the distorted image. The loss functions for each combination can be combined in any suitable manner. For example, the loss functions for each combination can be summed to provide a total loss function for a particular test weight.

[0070] After performing the above method for specific test weights, in step S110, it is determined whether a termination condition is met. The termination condition can be one or more of the following: a predetermined value for the loss function has been reached; the improvement in the loss function compared to previous iterations is less than a predetermined value; a local minimum of the loss function has been found; and a predetermined number of iterations have been performed. If the termination condition is not met, the test weights are adjusted in step S113, and the method returns to step S102, repeating the above process except using different test weights. In step S113, the values ​​of the test weights are adjusted in a manner predicted to minimize the loss function. In one embodiment, a random component may also be added to prevent the optimization routine from getting trapped in a local minimum.

[0071] The loss function L can therefore be determined according to the following equation:

[0072]

[0073] Where w is a specific weight, L sim It is a similarity metric, L smooth Here, f is the smoothness metric, m is the distorted image, and φ is the distortion mapping. The loss function for each weight is the sum of the similarity metric and the smoothness metric for each image combination i.

[0074] Once all necessary test weights have been iterated, the method proceeds to step S111, where optimized weights are determined as test weights with an optimized loss function. The weights of the encoder-decoder network are then set to the optimized weights, and the method terminates at step S112.

[0075] If, for a specific test weight, or for a set of test weights, the loss function indicates that an optimal weight has been found, the above method may not need to iterate over all test weights. For example, the loss function could be pre-set to indicate an optimal weight. Similarly, the loss function for multiple test weights could indicate the existence of an optimal weight without performing further iterations. For example, the loss function could be minimized for a specific test weight and subsequently increased for other test weights. Based solely on this information, it can be determined that the test weight providing the minimized loss function is the optimal weight without requiring further iterations over other test weights.

[0076] The optimized loss function can be correlated with the reference image and the aligned image, particularly the type of alignment being performed. The optimized loss function can correspond to the maximum similarity between the aligned image and the reference image.

[0077] The method described above can be unsupervised and does not require providing a real-world distortion map for each pair of reference and distorted images. This is advantageous because it simplifies the process of training such a model, as only paired SEM images are needed without the distortion map.

[0078] The method described above can utilize reference images obtained from a database. Images from the database can be enlarged, pixelated, and transformed into simulated images, such as simulated SEM images.

[0079] The method described above for determining the weights of the encoder-decoder network effectively provides a global optimization of the encoder-decoder network weights. While this determination of the optimized weights may be computationally relatively expensive, it can be performed offline before analyzing the distorted image of interest. Furthermore, once performed, it provides the encoder-decoder network, which is extremely fast in determining the distortion mapping for a given pair of reference and distorted images. This process can be several orders of magnitude faster than existing techniques. Its evaluation is also extremely fast. The use of shared weights w ensures that the generated distortion mapping is consistent across different patterns under the same true distortion.

[0080] Another advantage of the method described above is that determining the optimization weights over a larger field of view (relative to a small patch) has the added benefit of improved robustness and accuracy. This is because all relevant data are considered, thus greatly reducing the impact of factors such as noise and discretization errors. Compared to noise / discretization errors and even device characteristics, the actual distortion is relatively smooth in space. Therefore, when "fitting" such distortion over a large area and using a large number of data points, the resulting "fitting error" is averaged out and significantly reduced compared to determining the distortion locally over a small area.

[0081] The above method involves weight settings for an encoder-decoder network. A method using an encoder-decoder network is also disclosed for aligning a distorted image with weights set according to the above method. In a first step, the method includes using an encoder to encode a reference image and the distorted image into a latent space to form an encoding. Thereafter, a step is performed to decode the encoding using a decoder to form an optimized distortion map. Next, a step is performed to perform a spatial transformation on the distorted image using the distortion map to obtain the aligned image. This method essentially corresponds to the above reference... Figure 3 and Figure 4 The method described differs in that it does not optimize weights or repeat for different image combinations. These steps correspond to steps S104, S105, and S106 above, where the encoder-decoder network operates with optimized test weights.

[0082] The above methods involve training an encoder-decoder network and its use in encoding a reference image and a distorted image into a latent space and outputting a distortion map for transforming the distorted image. This paper also discloses another method for aligning distorted images. This other method... Figure 5 The method uses a pre-trained encoder-decoder network, which is trained to decode from the latent space to produce the distortion map.

[0083] The method is initiated at step S201. After initiation, in step S202, the distorted image is spatially transformed based on the test latent vectors to provide an aligned test image. Once the aligned test image is determined, in step S203, the aligned test image is compared with a reference image. After the comparison, in step S204, a similarity metric is obtained. The similarity metric is based on the comparison between the aligned image and the reference image. The specific form of the similarity metric may be related to the type of comparison performed. At step S205, it is determined whether multiple test latent vectors have been tested; if not, the process returns to step S202, and steps S202 to S204 are repeated. If the optimal similarity metric is determined before testing all test latent vectors, it is not necessary to analyze all test latent vectors.

[0084] Once sufficient test latent vectors have been processed, the method proceeds to step S206, which involves determining an optimized latent vector corresponding to the test latent vector that results in the optimal value of the similarity metric. The optimized similarity metric can be preset before processing begins; for example, it can be a similarity metric below a certain level, or it can be a similarity metric corresponding to the aligned image most similar to the reference image. With the optimized latent vectors determined, in step S207, an optimized distortion map is determined. This is achieved by decoding the optimized latent vectors using a pre-trained decoder. In step S208, the distorted image is spatially transformed using the optimized distortion map to output the aligned image. The process then ends at step S209. This method effectively utilizes the distribution of distortion maps encoded into the latent space to determine the appropriate distortion map for a given pair of reference and distorted images. By performing optimization in the latent space, the dimensionality of the optimization problem is reduced, making the process computationally cheaper. Performing optimization in the latent space allows gradient-based optimization to efficiently guide the search for the optimized latent vectors.

[0085] The similarity metric in this method can be any suitable metric indicating the similarity between the reference image and the aligned test image. For example, the similarity metric obtained in step S204 above can be determined by squaring the difference between the reference image and the aligned test image. In this case, the more similar the aligned test image is to the reference image, the smaller the similarity metric will be. Therefore, in this case, the optimized latent vector can correspond to the test latent vector where the similarity metric is minimized. The process described above for finding the optimized latent vector z*, when squaring the difference between the reference image and the aligned image, is mathematically described in the following equation:

[0086]

[0087] Where R k Let f be the k-dimensional real-valued space of the latent vectors, f be the reference image, m be the distorted image, and D(z) be the distortion map obtained by decoding z using a pre-trained encoder-decoder network. As described above, once the optimized latent vector z* is found (e.g., using gradient descent or a similar algorithm), the estimated distortion map is computed by forward-passing the solution through the decoder.

[0088] The pre-trained encoder-decoder network used in the above method can be trained with any suitable device, enabling it to encode and decode images in the desired manner. Figure 6 This is a flowchart describing an exemplary method for training a pre-trained encoder-decoder. Training is initiated in step S301. After initiation, the weights of the encoder-decoder network are set in step S302. These weights determine how the encoder encodes into the latent space and how the decoder decodes out of the latent space. In step S303, using the weight set, training images are encoded into the latent space to form training codes.

[0089] Then, in step S304, the training encoding is decoded to form a decoded image. The decoded image is then compared with the training image initially encoded by the encoder to obtain a training similarity metric. In step S306, it is determined whether the weights result in an optimized similarity metric. If not, the process returns to step S302 with different test weights, and steps S302 to S306 are repeated for the desired different weights until an optimized similarity metric is achieved. Once the optimized similarity metric is obtained, the weights that achieve this optimized similarity metric are used to set the weights of the encoder-decoder network. These steps thus form part of the pre-training of the encoder-decoder network. Ideally, the autoencoder is variable, in which case it is able to predict multiple outputs from a single input. These multiple outputs can be viewed as samples from a distribution. If the network is deterministic about the outputs, all outputs will be very similar (a distribution with low variance). If the network is uncertain about the outputs, the outputs will be less similar to each other (a distribution with high variance). Therefore, the determinism of the predictions generated by the network can be determined.

[0090] The similarity metric can be trained based on a classic loss function or a metric learned directly from the data using a discriminator network. The discriminator network learns to distinguish between real and fake distortion mappings, thereby generating the learned similarity metric; the more realistic the image predicted by the discriminator, the more similar it is to reality (and vice versa).

[0091] Training images used to train the network may include distortion maps. Figure 7 A schematic diagram illustrating steps S303 and S304 described above is shown. Figure 7 In the diagram shown, the distortion map is used to train the encoder-decoder network. As shown, in step S303, the encoder encodes the distortion map into the latent space, and in step S304, the decoder decodes the latent space. The encoder E(x) encodes the distortion map x∈C. n Transform into a low-dimensional real-valued continuous latent vector z∈R k (k << n), where n is the dimension of the distortion map and k is the dimension of the latent space. The encoder also models the uncertainty of this latent vector. The decoder D(z) transforms the continuous latent vector back into a copy of the input distortion map x'. The goal of training the encoder-decoder network is to set the weights of the encoder and decoder so that the decoder can decode the latent space to obtain an image as similar as possible to the image originally encoded by the encoder. The distortion map used in the above training process can be a measured or simulated distortion map. High uncertainty means that the network is not confident in its predictions for a particular distortion map. By guiding the training with more examples of high-uncertainty distortion maps, the network can be made more confident. This interactive training process is called active learning.

[0092] Therefore, the encoder-decoder network is taught how to encode the distortion map into a low-dimensional latent space, and given a low-dimensional input vector z, the decoder is able to generate a new distortion map D(z).

[0093] In any of the methods described above, the undistorted image can be obtained by synthesizing a function that computes the distorted image and the optimized distortion map.

[0094] In any of the examples discussed above, the reference images(s) and distorted images(s) may be of the semiconductor substrate. In any of the examples discussed, the reference images may include composite images. For example, a reference image may include an image reproduced from a database, rather than an actual image of the substrate. For example, a composite image may be an image from a database used to manufacture the substrate. Thus, a reference image may be a composite image of features on the substrate.

[0095] In any of the examples discussed above, at least one of the reference image and the distorted image can be obtained using a scanning electron microscope (e.g., voltage contrast SEM or cross-sectional SEM) or a transmission electron microscope (TEM), a scatterometer, etc.

[0096] In any of the methods described above for generating distortion maps based on distorted and reference images, the distortion map generated by the method can be used as a performance indicator. It can serve as an indicator of the performance of metrology equipment (e.g., SEM). For example, when a distortion map indicating an abnormally large degree of distortion is generated, this may indicate that the metrology equipment is not functioning properly. Following such an indication, the metrology equipment can be adjusted accordingly to perform more accurately.

[0097] The techniques disclosed in this article can reduce the complexity of the SEM process.

[0098] The techniques disclosed in this article can be used for in-line measurements of control loops and wafer deployment.

[0099] While specific techniques have been described above, it should be understood that this disclosure may be implemented in ways other than those described.

[0100] One embodiment may include a computer program that contains one or more machine-readable instruction sequences configured to instruct, as follows: Figure 1 The various devices shown perform measurement and optimization steps and control the subsequent exposure process as described above. This computer program can, for example, in... Figure 1 It is executed in a control unit (LACU) or a supervisory control system (SCS), or a combination of both. A data storage medium (e.g., semiconductor memory, magnetic disk, or optical disk) in which such a computer program is stored may also be provided.

[0101] While the above specifically references photolithography, it should be understood that the techniques disclosed herein can be used in other applications, such as imprint lithography. In imprint lithography, the morphology within a patterning apparatus defines the pattern created on a substrate. The morphology of the patterning apparatus can be pressed into a resist layer provided to the substrate, which is then cured by applying electromagnetic radiation, heat, pressure, or a combination thereof. The patterning apparatus is removed from the resist, leaving a pattern therein after the resist has cured.

[0102] As used herein, the terms “radiation” and “beam” encompass all types of electromagnetic radiation, including ultraviolet (UV) radiation (e.g., wavelengths of about 365 nm, 355 nm, 248 nm, 193 nm, 157 nm, or 126 nm) and extreme ultraviolet (EUV) radiation (e.g., wavelengths in the range of 1–100 nm), as well as particle beams (such as ion beams or electron beams). Implementations of scatterometers and other inspection equipment can be performed using suitable sources at UV and EUV wavelengths, and this disclosure is by no means limited to systems using IR and visible radiation.

[0103] Where the context permits, the term "lens" can refer to any one or a combination of various types of optical components, including refractive, reflective, magnetic, electromagnetic, and electrostatic optical components. Reflective components can be used in devices operating in the UV and / or EUV range.

[0104] As used herein, unless otherwise specifically stated or impractical, the term "or" covers all possible combinations. For example, if it is indicated that a component may include A or B, then unless otherwise specifically stated or impractical, the component may include A or B, or A and B. As a second example, if it is indicated that a component may include A, B, or C, then unless otherwise specifically stated or impractical, the component may include A, or B, or C, or A and B, or A and C, or B and C, or A and B and C.

[0105] The various aspects of this disclosure are set forth in the following numbered clauses:

[0106] 1. A method for determining optimized weights for encoder and decoder networks, the method comprising:

[0107] For each of the multiple test weights, the following steps are performed using the encoder and decoder with the test weights:

[0108] (a) Using an encoder, the reference image and the distorted image are encoded into the latent space to form a code;

[0109] (b) Using a decoder, the encoding is decoded to form a distortion map that indicates the difference between the reference image and the distorted image;

[0110] (c) By performing spatial transformation on the distorted image through distortion mapping, an aligned image is obtained;

[0111] (d) Compare the aligned image with the reference image to obtain a similarity measure; and

[0112] (e) Determine a loss function that is at least partially defined by a similarity metric;

[0113] The optimized weights are determined to be test weights with an optimized loss function.

[0114] 2. The method according to Clause 1, wherein the optimized loss function corresponds to the maximum similarity between the aligned image and the reference image.

[0115] 3. The method according to Clause 1 or 2, wherein the loss function is further defined at least in part by a smoothness metric, which is defined by the smoothness of the distortion map.

[0116] 4. The method according to Clause 3, wherein the smoothness metric is defined at least in part by the spatial gradient of the distortion map.

[0117] 5. The method according to any one of clauses 1 to 4, wherein for each test weight, steps (a)-(e) are repeated for multiple different combinations of the reference image and the distorted image, and wherein the loss function for a given test weight is a combination of loss functions determined based on each of the different combinations of the reference image and the distorted image.

[0118] 6. The method according to Clause 5, wherein the multiple combinations include a combination of at least one reference image and multiple different distorted images.

[0119] 7. The method according to Clause 5 or 6, wherein the multiple combinations include a combination of multiple different reference images and multiple different distorted images.

[0120] 8. A method for aligning distorted images, the method comprising:

[0121] An encoder-decoder network is used, which operates with the optimized weights determined according to the method described in Clause 1 and performs the following:

[0122] An encoder is used to encode the reference image and the distorted image into the latent space to form a code;

[0123] A decoder is used to decode the encoding to form an optimized distortion map; and

[0124] Distortion mapping is used to spatially transform the distorted image to obtain an aligned image.

[0125] 9. A method for aligning distorted images, the method comprising:

[0126] Using a pre-trained encoder-decoder network trained to decode from the latent space to produce a distortion map, the following is performed for each of a plurality of different test latent vectors:

[0127] (a) Based on the corresponding test latent vector, spatial transformation is performed on the distorted image to provide an aligned test image;

[0128] (b) Compare the aligned test image with the reference image;

[0129] (c) Obtain a similarity measure based on the comparison between the aligned image and the reference image;

[0130] (d) Determine the optimized latent vector corresponding to the test latent vector that provides the optimized similarity metric;

[0131] (e) Determine the optimized distortion mapping by decoding the optimized latent vectors using a pre-trained decoder; and

[0132] (f) Spatial transformation of the distorted image is performed using an optimized distortion map to output an aligned image.

[0133] 10. The method according to Clause 9, wherein the similarity metric is determined by squaring the difference between the reference image and the aligned test image, and wherein the optimized latent vector corresponds to the test latent vector in which the similarity metric is minimized.

[0134] 11. The method according to clause 9 or 10, wherein the pre-trained encoder-decoder network is trained by:

[0135] (a) Encode the training images into the latent space to form the training encoding;

[0136] (b) Decode the training code to form a decoded image;

[0137] (c) Compare the decoded image with the training image to obtain the training similarity metric;

[0138] And for different weights in the encoder-decoder network, repeat steps (a)-(c) to find the optimized weights in which the similarity metric is optimized for training.

[0139] 12. The method according to any one of clauses 1 to 11, wherein spatial transformation of the distorted image by means of an optimized distortion map comprises: determining the dot product of the distorted image and the distortion map.

[0140] 13. The method according to any one of clauses 1 to 12, wherein the reference image and the distorted image are on a semiconductor substrate.

[0141] 14. The method according to any one of clauses 1 to 13, wherein at least one of the reference image and the distorted image is obtained using a scanning electron microscope.

[0142] 15. The method according to any one of clauses 1 to 14, wherein the reference image includes a synthetic image (e.g., obtained from a database).

[0143] 16. The method according to any one of clauses 1 to 15, wherein the training image includes a distortion map.

[0144] 17. An inspection tool, comprising:

[0145] An imaging system is configured to image a portion of a semiconductor substrate; and

[0146] An image analysis system is configured to perform a method for aligning a distorted image according to any one of clauses 1 to 16.

[0147] 18. A method for manufacturing a semiconductor substrate, the method comprising the following steps:

[0148] Manufacturing at least a portion of a semiconductor substrate;

[0149] Perform the method pursuant to any one of Articles 1 to 16;

[0150] Analyze aligned images to obtain image metrics; and

[0151] The metric is compared with a target metric, and remedial actions are taken when the metric does not meet the target metric.

[0152] 19. An inspection tool, comprising:

[0153] An imaging system is configured to capture an image of a portion of a semiconductor substrate; and

[0154] An image analysis system is configured to reduce distortion in images captured by an imaging system by determining optimized weights for a network comprising an encoder and a decoder, wherein the image analysis system is configured to:

[0155] For each of the multiple test weights, the following steps are performed using the encoder and decoder with the test weights:

[0156] An encoder is used to encode the reference image and the distorted image into the latent space to form a code;

[0157] The decoder is used to decode the encoding to form a distortion map that indicates the difference between the reference image and the distorted image;

[0158] A spatial transformation is performed on the distorted image using distortion mapping to obtain an aligned image; the aligned image is then compared with a reference image to obtain a similarity metric; and...

[0159] Determine a loss function that is at least partially defined by a similarity metric;

[0160] The optimized weights are determined to be test weights with an optimized loss function.

[0161] 20. The inspection tool as described in Clause 19, wherein the optimized loss function corresponds to the maximum similarity between the aligned image and the reference image.

[0162] 21. The inspection tool according to Clause 19 or 20, wherein the loss function is further defined at least in part by a smoothness metric, which is defined by the smoothness of the distortion map.

[0163] 22. The inspection tool as described in Clause 21, wherein the smoothness metric is defined at least in part by the spatial gradient of the distortion map.

[0164] 23. The inspection tool according to any one of clauses 19 to 22, wherein steps (a)-(e) are repeated for each test weight for multiple different combinations of the reference image and the distorted image, and wherein the loss function for a given test weight is a combination of loss functions determined based on each of the different combinations of the reference image and the distorted image.

[0165] 24. The inspection tool as described in Clause 23, wherein the multiple combinations include a combination of at least one reference image and multiple different distorted images.

[0166] 25. The inspection tool as described in Clause 23, wherein the multiple combinations include a combination of multiple different reference images and multiple different distorted images.

[0167] 26. The inspection tool according to any one of clauses 19 to 25, wherein the image analysis system is further configured to:

[0168] Using a network with optimized weights, to:

[0169] An encoder is used to encode the reference image and the distorted image into the latent space to form a code;

[0170] A decoder is used to decode the encoding to form an optimized distortion map; and

[0171] The distorted image is spatially transformed using distortion mapping to obtain an aligned image.

[0172] 27. An inspection tool, comprising:

[0173] An imaging system is configured to capture an image of a portion of a semiconductor substrate; and

[0174] An image analysis system is configured to align with a distorted image captured by an imaging system. The image analysis system includes a pre-trained encoder-decoder network trained to decode from the latent space to produce a distortion map, and is configured to perform the following for each of a plurality of different test latent vectors:

[0175] Based on the test latent vector, a spatial transformation is performed on the distorted image to provide an aligned test image;

[0176] The aligned test image is compared with a reference image;

[0177] A similarity metric is obtained by comparing the aligned image with a reference image;

[0178] Determine the optimized latent vector corresponding to the test latent vector that provides the optimized similarity metric;

[0179] The optimized distortion mapping is determined by decoding the optimized latent vectors using a pre-trained decoder; and

[0180] The distorted image is spatially transformed using an optimized distortion map to output an aligned image.

[0181] 28. The inspection tool according to Clause 27, wherein the similarity metric is determined by squaring the difference between the reference image and the aligned test image, and wherein the optimized latent vector corresponds to the test latent vector in which the similarity metric is minimized.

[0182] 29. The inspection tool as described in Clause 27, wherein the pre-trained encoder-decoder network is trained by:

[0183] (a) Encode the training images into the latent space to form the training encoding;

[0184] (b) Decode the training code to form a decoded image;

[0185] (c) Compare the decoded image with the training image to obtain a training similarity metric;

[0186] And repeat steps (a)-(c) for different weights of the encoder-decoder network to find the optimized weights in which the similarity metric is trained.

[0187] 30. The inspection tool according to any one of clauses 19 to 29, wherein spatial transformation of the distorted image by means of an optimized distortion map comprises: determining the dot product of the distorted image and the distortion map.

[0188] 31. The inspection tool according to any one of Clauses 19 to 30, wherein the reference image and the distorted image are of a semiconductor substrate.

[0189] 32. The inspection tool according to any one of clauses 19 to 31, wherein at least one of the reference image and the distorted image is obtained using a scanning electron microscope.

[0190] 33. An inspection tool according to any one of Clauses 19 to 32, wherein the reference image includes a composite image (e.g., obtained from a database).

[0191] 34. A computer program comprising code (ideally embodied in a non-transitory computer-readable medium), which, when executed by a computer system, instructs the computer system to perform a method for determining optimized weights for encoder and decoder networks, the method comprising:

[0192] For each of the multiple test weights, the following steps are performed using the encoder and decoder with the test weights:

[0193] (a) Using an encoder, the reference image and the distorted image are encoded into the latent space to form a code;

[0194] (b) Using a decoder, the encoding is decoded to form a distortion map that indicates the difference between the reference image and the distorted image;

[0195] (c) By performing spatial transformation on the distorted image through distortion mapping, an aligned image is obtained;

[0196] (d) Compare the aligned image with the reference image to obtain a similarity measure; and

[0197] (e) Determine a loss function that is at least partially defined by a similarity metric;

[0198] The optimized weights are determined to be test weights with an optimized loss function.

[0199] 35. A computer program comprising code (ideally embodied in a non-transitory computer-readable medium), which, when executed by a computer system, instructs the computer system to perform a method for aligning a distorted image, the method comprising:

[0200] Using a pre-trained encoder-decoder network trained to decode from the latent space to produce a distortion map, the following is performed for each of several different test latent vectors:

[0201] (a) Based on the test latent vector, spatial transformation is performed on the distorted image to provide an aligned test image;

[0202] (b) Compare the aligned test image with the reference image;

[0203] (c) Obtain a similarity measure based on the comparison between the aligned image and the reference image;

[0204] (d) Determine the optimized latent vector corresponding to the test latent vector that provides the optimized similarity metric;

[0205] (e) Determine the optimized distortion mapping by decoding the optimized latent vectors using a pre-trained decoder; and

[0206] (f) Spatial transformation of the distorted image is performed using an optimized distortion map to output an aligned image.

[0207] 36. A method for aligning distorted images, the method comprising:

[0208] The following can be performed using an encoder-decoder network that operates by optimizing weights:

[0209] An encoder is used to encode the reference image and the distorted image into the latent space to form a code;

[0210] A decoder is used to decode the encoding to form an optimized distortion map; and

[0211] Using distortion mapping, a spatial transformation is performed on the distorted image to obtain an aligned image.

[0212] Embodiments of the invention have been described, and it should be understood that modifications may be made within the spirit and scope of this disclosure and the appended claims and their equivalents.

Claims

1. A method for determining optimized weights for encoder and decoder networks, the method comprising: For each of the multiple test weights, the encoder and decoder are used to perform the following steps using the test weights: (a) Using the encoder, the reference image and the distorted image are encoded into the latent space to form an encoding; (b) Using the decoder, the encoding is decoded to form a distortion map indicating the difference between the reference image and the distorted image; (c) Perform spatial transformation on the distorted image using the distortion mapping. To obtain an aligned image; (d) Compare the aligned image with the reference image to obtain a similarity measure; as well as (e) Determine a loss function that is at least partially defined by the similarity metric; The optimized weights are determined to be the test weights with an optimized loss function.

2. The method of claim 1, wherein the optimized loss function corresponds to the maximum similarity between the aligned image and the reference image.

3. The method of claim 1 or 2, wherein the loss function is further defined at least in part by a smoothness metric, the smoothness metric being defined by the smoothness of the distortion map.

4. The method of claim 3, wherein the smoothness metric is defined at least in part by the spatial gradient of the distortion map.

5. The method according to claim 1 or 2, wherein steps (a)-(e) are repeated for multiple different combinations of the reference image and the distorted image for each test weight, and wherein the loss function for a given test weight is a combination of the loss functions determined based on each of the different combinations of the reference image and the distorted image.

6. The method of claim 5, wherein the plurality of combinations comprises a combination of at least one reference image and a plurality of different distorted images.

7. The method of claim 5, wherein the plurality of combinations comprises a plurality of different reference images and a plurality of different distorted images.

8. A method for aligning a distorted image, the method comprising: The encoder-decoder network is operated using the optimized weights determined according to claim 1, and the following steps are performed: Using the encoder, the reference image and the distorted image are encoded into the latent space to form an encoding; The decoder is used to decode the encoding to form an optimized distortion map; as well as The distortion map is used to perform a spatial transformation on the distorted image to obtain an aligned image.

9. A method for aligning a distorted image, the method comprising: Using a pre-trained encoder-decoder network trained to decode from the latent space to produce a distortion map, the following steps are performed for each of a plurality of different test latent vectors: (a) Based on the corresponding test latent vector, the distorted image is spatially transformed to provide an aligned test image; (b) Compare the aligned test image with the reference image; (c) Based on the comparison between the aligned image and the reference image, Obtain a similarity metric; (d) Determine the optimized latent vector corresponding to the test latent vector that provides the optimized similarity metric; (e) Determine the optimized distortion mapping by decoding the optimized latent vector using the pre-trained decoder; as well as (f) The distorted image is spatially transformed using the optimized distortion mapping to output an aligned image.

10. The method of claim 9, wherein the similarity metric is determined by squaring the difference between the reference image and the aligned test image, and wherein the optimized latent vector corresponds to the test latent vector to which the similarity metric is minimized.

11. The method of claim 9 or 10, wherein the pre-trained encoder-decoder network is trained by: (a) Encode the training images into the latent space to form the training encoding; (b) Decode the training code to form a decoded image; (c) Compare the decoded image with the training image to obtain a training similarity metric; And for different weights of the encoder-decoder network, repeat steps (a)-(c) to find the optimized weights for which the training similarity metric is optimized.

12. The method according to any one of claims 1, 2, 8, 9, and 10, wherein spatial transformation of the distorted image via the optimized distortion map comprises: Determine the dot product between the distorted image and the distorted mapping.

13. The method according to any one of claims 1, 2, 8, 9 and 10, wherein the reference image and the distorted image are of a semiconductor substrate.

14. The method according to any one of claims 1, 2, 8, 9 and 10, wherein at least one of the reference image and the distorted image is obtained using a scanning electron microscope.

15. The method according to any one of claims 1, 2, 8, 9 and 10, wherein the reference image comprises a synthetic image.

16. The method of claim 11, wherein the training image includes a distortion map.

17. An inspection tool, comprising: An imaging system configured to image a portion of a semiconductor substrate; as well as An image analysis system is configured to perform a method for aligning a distorted image according to any one of claims 1 to 16.

18. A method for manufacturing a semiconductor substrate, the method comprising the following steps: To manufacture at least a portion of the semiconductor substrate; Perform the method according to any one of claims 1 to 16; Analyze the aligned image to obtain a metric of the image; as well as The metric is compared with a target metric, and remedial actions are taken when the metric does not meet the target metric.

19. An inspection tool, comprising: An imaging system is configured to capture an image of a portion of a semiconductor substrate; as well as An image analysis system is configured to reduce distortion in images captured by the imaging system by determining optimized weights for a network comprising an encoder and a decoder, wherein the image analysis system is configured to: For each of the multiple test weights, the encoder and decoder are used to perform the following steps using the test weights: (a) Using the encoder, the reference image and the distorted image are encoded into the latent space to form an encoding; (b) Using the decoder, the encoding is decoded to form a distortion map indicating the difference between the reference image and the distorted image; (c) Perform spatial transformation on the distorted image using the distortion mapping. To obtain an aligned image; (d) Compare the aligned image with the reference image to obtain a similarity measure; as well as (e) Determine a loss function that is at least partially defined by the similarity metric; The optimized weights are determined to be the test weights with an optimized loss function.

20. The inspection tool of claim 19, wherein the optimized loss function corresponds to the maximum similarity between the aligned image and the reference image.

21. The inspection tool of claim 19 or 20, wherein the loss function is further defined at least in part by a smoothness metric, the smoothness metric being defined by the smoothness of the distortion map.

22. The inspection tool of claim 21, wherein the smoothness metric is defined at least in part by the spatial gradient of the distortion map.

23. The inspection tool according to claim 19 or 20, wherein steps (a)-(e) are repeated for multiple different combinations of the reference image and the distorted image for each test weight, and wherein the loss function for a given test weight is a combination of the loss functions determined based on each of the different combinations of the reference image and the distorted image.

24. The inspection tool of claim 23, wherein the plurality of combinations comprises a combination of at least one reference image and a plurality of different distorted images.

25. The inspection tool of claim 23, wherein the plurality of combinations comprises a plurality of different reference images and a plurality of different distorted images.

26. The inspection tool according to claim 19 or 20, wherein the image analysis system is further configured to: Using a network with the aforementioned optimization weights, to: Using the encoder, the reference image and the distorted image are encoded into the latent space to form an encoding; The decoder is used to decode the encoding to form an optimized distortion map; as well as The distortion map is used to spatially transform the distorted image to obtain an aligned image.

27. An inspection tool, comprising: An imaging system is configured to capture an image of a portion of a semiconductor substrate; as well as An image analysis system is configured to align with a distorted image captured by the imaging system. The image analysis system includes a pre-trained encoder-decoder network trained to decode from the latent space to produce a distortion map, and the pre-trained encoder-decoder network is configured to perform the following for each of a plurality of different test latent vectors: Based on the corresponding test latent vectors, the distorted image is spatially transformed to provide an aligned test image; The aligned test image is compared with a reference image; A similarity metric is obtained based on the comparison between the aligned image and the reference image; Determine the optimized latent vector corresponding to the test latent vector that provides the optimized similarity metric; The optimized distortion mapping is determined by decoding the optimized latent vector using the pre-trained decoder. as well as The distorted image is spatially transformed using the optimized distortion mapping to output an aligned image.

28. The inspection tool of claim 27, wherein the similarity metric is determined by squaring the difference between the reference image and the aligned test image, and wherein the optimized latent vector corresponds to the test latent vector for which the similarity metric is minimized.

29. The inspection tool of claim 27, wherein the pre-trained encoder-decoder network is trained by: (a) Encode the training images into the latent space to form the training encoding; (b) Decode the training code to form a decoded image; (c) Compare the decoded image with the training image to obtain a training similarity metric; And for different weights of the encoder-decoder network, steps (a)-(c) are repeated to find optimized weights in which the training similarity metric is optimized.

30. The inspection tool according to any one of claims 19, 20, 27, 28, and 29, wherein spatial transformation of the distorted image via the optimized distortion mapping comprises: Determine the dot product between the distorted image and the distorted mapping.

31. The inspection tool according to any one of claims 19, 20, 27, 28 and 29, wherein the reference image and the distorted image are of a semiconductor substrate.

32. The inspection tool according to any one of claims 19, 20, 27, 28 and 29, wherein at least one of the reference image and the distorted image is obtained using a scanning electron microscope.

33. The inspection tool according to any one of claims 19, 20, 27, 28 and 29, wherein the reference image comprises a composite image.

34. A computer program including code, ideally implemented in a non-transitory computer-readable medium, wherein the code, when executed by a computer system, instructs the computer system to perform a method for determining optimized weights of an encoder and decoder network, the method comprising: For each of the multiple test weights, the encoder and decoder are used to perform the following steps using the test weights: (a) Using the encoder, the reference image and the distorted image are encoded into the latent space to form an encoding; (b) Using the decoder, the encoding is decoded to form a distortion map indicating the difference between the reference image and the distorted image; (c) Perform spatial transformation on the distorted image using the distortion mapping. To obtain an aligned image; (d) Compare the aligned image with the reference image to obtain a similarity measure; as well as (e) Determine a loss function that is at least partially defined by the similarity metric; The optimized weights are determined to be the test weights with an optimized loss function.

35. A computer program including code, ideally implemented in a non-transitory computer-readable medium, the code, when executed by a computer system, instructing the computer system to perform a method for aligning a distorted image, the method comprising: Using a pre-trained encoder-decoder network trained to decode from the latent space to produce a distortion map, the following steps are performed for each of a plurality of different test latent vectors: (a) Based on the test latent vectors, perform spatial transformation on the distorted image. To provide aligned test images; (b) Compare the aligned test image with the reference image; (c) Based on the comparison between the aligned image and the reference image, Obtain a similarity metric; (d) Determine the optimized latent vector corresponding to the test latent vector that provides the optimized similarity metric; (e) Determine the optimized distortion mapping by decoding the optimized latent vector using the pre-trained decoder; as well as (f) The distorted image is spatially transformed using the optimized distortion mapping to output an aligned image.

36. A method for aligning a distorted image, the method comprising: The following steps are performed using an encoder-decoder network that operates by optimizing weights: Using the encoder, the reference image and the distorted image are encoded into the latent space to form an encoding; The decoder is used to decode the encoding to form an optimized distortion map; as well as The distortion map is used to perform a spatial transformation on the distorted image to obtain an aligned image.

Images