Obtaining high resolution information from low resolution images

Abstract

The invention proposes a method for determining values of a plurality of numerical parameters, which collectively define a product model of at least one product, using low-resolution images of at least one product produced by one or more imaging processes and an imaging model characterizing the imaging processes. The determination of the value is performed by forming a loss function based on the acquired image, the imaging model, and a numerical parameter of the model, and executing a minimization algorithm to minimize the loss function relative to the numerical parameter. Due to the priori knowledge of the product encoded in the loss function, the product model may include a reconstructed image with a higher resolution than the low-resolution image.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Application No. 63 / 470,582 filed on June 2, 2023 and European Patent Application No. 22185297.3 filed on July 15, 2022, which are incorporated herein by reference in their entirety. Technical Field

[0003] The present application relates to methods and systems for using acquired images of at least one product of a manufacturing process to form a product model of the product(s), such as a product model containing information at a higher spatial resolution than the acquired images. Background Art

[0004] A lithographic apparatus is a machine constructed to apply a desired material pattern to a substrate. For example, a lithographic apparatus can be used in the manufacture of integrated circuits (ICs). For example, a lithographic apparatus can project a patterned electromagnetic radiation beam generated by a patterning device onto a radiation-sensitive material (resist) layer provided on a substrate. The term "patterning device" as used herein should be broadly interpreted as referring to a device that can be used to impart a patterned cross-section to an incident electron radiation beam, the patterned cross-section corresponding to a pattern to be created in a target portion of the substrate. For example, a patterning device can be a mask (or a graticule) that causes selective transmission (in the case of a transmission mask) or reflection (in the case of a reflective mask) of a radiation beam impinging on the mask, depending on the pattern on the mask. The patterned beam changes the physical properties of the portion of the resist that receives the radiation.

[0005] A patterned structure having multiple layers can be formed on a substrate by a repetitive process of depositing layers onto a substrate (including a resist layer and other layers), patterning the resist layer using a radiation beam having a patterned cross-section, and selectively removing portions of the resist layer based on different physical properties of the resist layer. The wavelength of the radiation determines the minimum size of features that can be formed on the substrate.

[0006] Many processes are known for imaging the products of a manufacturing process to check for manufacturing defects. Such tools include brightfield imaging tools, darkfield imaging tools, and scanning electron microscope (SEM) tools. These tools have different advantages and disadvantages. In brightfield imaging, light having a certain range of wavelengths (such as "white" light) is used to illuminate the sample, and an image is formed using the light transmitted or reflected by the sample. Brightfield imaging tools are generally used to detect coarse imaging defects. These tools are fast, and large areas of the equipment can be easily measured as millions of images, but the resolution of the resulting images is low, making them not easily suitable for making metrological measurements, such as the size of any defects or the size of any structures formed in the product, so it is difficult to use them to detect fine imaging defects. Summary of the invention

[0007] The present invention relates to a method and system for obtaining information describing a manufacturing process of at least one product based on a plurality of acquired images of the at least one product. The images may optionally be acquired by the same imaging process, but more generally, a plurality of imaging processes may be used to capture different ones of the images.

[0008] In general, the present invention proposes using images representing (multiple) corresponding imaging processes and (multiple) corresponding imaging models to determine values ​​of multiple numerical parameters that together define a product model of (multiple) portions of at least one product. The determination of the numerical values ​​is performed by forming a loss function based on the acquired images, the (multiple) imaging models and the numerical parameters of the models, and performing a minimization algorithm to minimize the loss function with respect to the numerical parameters. In an embodiment, the product model describes the (multiple) products with a higher spatial resolution than the captured images.

[0009] The determination of the value may employ a loss value identifying the difference between the acquired image and the result of applying the imaging model(s) to the product model. The loss value may be included as a term in the loss function, or an upper limit of the loss function may be used as a constraint for the minimization algorithm.

[0010] Based on the product model after the numerical parameters are determined, a defect inspection process can be performed. For example, areas of the product(s) where suspected defects exist may be identified, and the identified areas may then be further inspected (e.g., by different imaging processes). Alternatively or additionally, the product model may be measured to obtain the dimensions of the product(s).

[0011] The acquired images may include multiple images of a single portion of the product, such as captured with different corresponding imaging processes. For example, the imaging process performed by the imaging device may be characterized by at least one imaging parameter, and each imaging process may correspond to a different implementation of the at least one imaging parameter. For example, the at least one imaging parameter may be selected from the group consisting of:

[0012] a distance between a sensor of the imaging device and at least one product;

[0013] an orientation of at least one product relative to an imaging direction in which images are captured during the imaging process;

[0014] a translational position of at least one product transverse to an imaging direction in which images are captured during the imaging process;

[0015] The frequency (e.g., frequency range) of electromagnetic radiation used in the imaging process;

[0016] The focal position of the imaging device relative to the product (“Focus”); and

[0017] The exposure time ("dose") used to capture the image. All of these can be considered control singulars of the imaging process because the imaging system allows them to be controlled (e.g., by an operator).

[0018] Alternatively or additionally, the acquired images may be images of different corresponding imaging areas of at least one product in which the products are expected to have similar structures, for example because the imaging areas of the products are created based on corresponding design data (e.g., graphic design system (GDS) data), the data being the same for different imaging areas, or at least meeting similarity criteria (e.g., being the same for at least a certain proportion of the imaging areas). For example, if the product has a repeating structure with a certain periodicity, the acquired images may include images of different corresponding imaging areas spaced apart on the product according to the periodicity. Furthermore, the acquired images may include images of corresponding imaging areas located at the same position on different images of the product (i.e., if the product is a semiconductor wafer, the imaging areas may include imaging areas on different wafers that are located at the same position relative to the center of the corresponding wafer).

[0019] In this case, the product model includes a corresponding model portion for each of the imaging regions having similar structures, and the loss function may include a term that penalizes the differences between the model portions. Each model portion may be defined by a subset of the numerical parameters of the product model. In this way, information contained in the image and associated with similar structures is shared between certain portions. The difference term may be a term that penalizes the rank of the concatenation of the subset of numerical parameters that define the corresponding model portion. For example, as follows, the model portion may be in the form of an image of the imaging region having similar structures (a "reconstructed image", indicating what the product would look like if the product were imaged by an imaging process having a better spatial resolution than the actual imaging process), and in this case, the difference term may penalize the rank of the concatenation of the reconstructed images. More generally, the difference term may be the rank of the concatenation of the subset of numerical parameters that define the model portion.

[0020] Many (although not all) imaging processes actually apply a point spread function to the imaged structure in the product. In this case, the imaging model comprises a convolution model, for example based on an array of values ​​indicating the values ​​of corresponding points on the point spread function.

[0021] As above, an example of a product model is that the product model includes one or more reconstructed images (two-dimensional arrays of intensity values) indicating what the product would look like if the product was imaged by an imaging process that provides a higher spatial resolution than the actual imaging process. For example, the product model may include a corresponding reconstructed image for each of the acquired images, and the reconstructed images may include information about the product(s) having a higher spatial resolution than the corresponding acquired image. This is possible because information about each reconstructed image can be collected from multiple of the acquired images, and because the loss function can be selected to encode knowledge about the product (i.e., minimizing the loss function corresponds to obtaining a product model with a higher prior likelihood of being correct). Therefore, this example of the present disclosure makes it possible to use multiple low spatial resolution images (e.g., bright field images) to obtain a higher spatial resolution image of a product.

[0022] In the case where the imaging process applies a point spread function to the imaging structure, features just outside the imaging area can have an impact on the acquired image. To this end, the reconstructed images can correspond to areas of the product that extend outside the corresponding imaging area (for example, they can include the edges of the corresponding imaging area around all sides of the imaging area, that is, around the peripheral edge of the imaging area). The imaging model not only applies the convolution model to the reconstructed image, but also preferably applies a mask to the result, thereby trimming the result of the convolution to only include pixels corresponding to positions in the corresponding imaging area.

[0023] As above, the reconstructed images may contain higher spatial resolution information than the acquired images, and furthermore, the reconstructed images may have a higher pixel resolution (i.e., a greater number of pixels per unit distance on the product(s)) than the acquired images. In this case, the imaging model includes a pixel resolution reduction model that, when applied to the reconstructed images (before or after application of the convolution model and masks, if they exist), reduces the pixel resolution of these reconstructed images to be equal to the pixel resolution of the acquired images.

[0024] In order to promote the reconstructed image to have low background intensity and thus high dynamic range, the loss function may include at least one regularization term based on the norm of the intensity values. In addition, since it is well known that typical products include a structure with well-defined edges, the regularization term may include a structure term that promotes the formation of edges in the two-dimensional array of intensity values. The regularization term may be based on applying a transform (e.g., a wavelet transform and / or a transform that extracts image gradients) to the reconstructed image to obtain transformed values, and the regularization term may include a sum of the transformed values ​​of a function of each transformed value.

[0025] In addition to determining numerical parameters of the product model, the method may also include inferring (multiple) imaging models from the acquired image by inferring values ​​of imaging parameters that define the (multiple) imaging models. In addition to or in lieu of the imaging parameters listed above (which may be considered control parameters), in the case of an imaging model that includes a convolution model, the imaging parameters may include an array of values ​​indicating values ​​of corresponding points on a point spread function. While the point spread function typically depends on the selected focus position (and may depend on other control parameters), it is typically a function of the imaging system itself to some extent.

[0026] The imaging parameters may optionally be inferred as part of the minimization algorithm, i.e. as the imaging parameters that minimize the loss function. Alternatively, some or all of the imaging parameters may be determined experimentally, for example by using a target with known geometry. If the target comprises an object whose diameter in each direction transverse to the imaging direction is smaller than the spatial resolution of the imaging process (e.g. smaller than the half width of the point spread function in these directions), then the array of values ​​indicating the values ​​of corresponding points on the point spread function may be intensity values ​​of an image of the object.

[0027] The loss function comprises a plurality of terms multiplied by corresponding hyperparameters. The value of the hyperparameter may be selected by repeatedly performing the method for different corresponding trial values ​​of the hyperparameter and selecting the value of the hyperparameter from the trial values ​​of the hyperparameter using a selection criterion. For example, the criterion may be to select which trial values ​​of the hyperparameter that minimize the value of the loss function (or at least some terms of the loss function) after performing a minimization algorithm relative to the numerical parameters of the production model. In principle, other selection criteria may also be used, for example based on reducing the time taken for the minimization algorithm to converge to a solution.

[0028] Although as above, the product model may include (a plurality of) reconstructed images, it may alternatively (or additionally) be defined based on numerical parameters that define corresponding geometric parameters of at least one product. These numerical parameters may be any geometric parameters, the selection of which is based on a priori knowledge of the structure that the manufacturing process is expected to form (e.g. based on design data used to configure the manufacturing process).

[0029] For example, the design data may specify that the manufacturing process should produce a product including at least one elongated element. In this case, the numerical parameters may include: the position of the elongated element in the product; the length of the elongated element in the elongation direction; the width of the elongated structure transverse to the elongation direction; the angle between the length direction of the elongated element and a reference direction; or the angle between the corresponding length directions of two elongated elements in the product.

[0030] Alternatively or additionally, the design data may specify that at least one product comprises a plurality of layers spaced apart in a depth direction, and the numerical parameters comprise numerical parameters respectively indicating one of: a translational offset in the depth direction transverse to two structures in at least one product in different respective layers; an angular offset in the depth direction transverse to the corresponding length directions of two elongated structures in at least one product in different respective layers; or a spacing in the depth direction between two structures in the product.

[0031] The present concepts can be applied to acquired images captured by an imaging process that relies on any imaging mode. In one example, the imaging mode is bright field microscopy. However, the imaging mode can alternatively be dark field microscopy or an imaging mode that does not rely on electromagnetic radiation, such as a scanning electron microscope (SEM).

[0032] Once the numerical parameters of the product model are determined, the product model can be used in a variety of ways. As above, the first way is to measure the product model to obtain numerical data describing the product. For example, in the case where the product model includes (multiple) reconstructed images, distance values ​​in the images can be measured to obtain the dimensions of the structures in the product. These dimensions can be compared with the design data to determine the accuracy of the manufacturing process. If the difference between the design data and the measured dimensions of the structure in the product is higher than a threshold value (tolerance), an indication of the presence of an anomaly can be generated. Therefore, the threshold constitutes an anomaly criterion.

[0033] Another possibility is an anomaly criterion based on observed differences between model parts that describe parts of a product that should be similar according to the design data. For example, if the product model is a set of reconstructed images comprising at least one imaged part of the product that, according to the design data, comprises the same structure, the anomaly criterion may be based on whether a difference metric between the reconstructed images is above a threshold, such as whether an elongated element in the structure has a measured length and / or width and / or orientation in one of the reconstructed images that differs from a measured length and / or width and / or orientation in another of the reconstructed images by more than a threshold.

[0034] The anomaly detection process can be repeatedly performed for successive products of the manufacturing process, for example by determining that a product model associated with a newly produced product meets the anomaly criteria. For example, depending on the manufacturing process producing a new product, the method can be performed based on acquired images of the new product, and optionally also based on acquired images captured by a previous execution of the manufacturing process (such as an immediately previous execution).

[0035] After determining that the anomaly criteria are met, additional inspection procedures may be performed, such as using a different imaging modality that provides an image with a higher spatial resolution than the acquired image (eg, using a SEM if the acquired image was captured by brightfield microscopy).

[0036] The fact that an anomaly is detected based on differences between model parts corresponding to parts of at least one product that should be similar (e.g., based on the same design data) does not mean which model part contains the anomaly. Optionally, all model parts used to determine that meet the anomaly criteria can be checked by an additional inspection process. Alternatively, in the case where there are at least three model parts corresponding to parts of at least one product that should be similar, there may be a process for identifying which of these model parts is most different from the other model parts, so that the anomaly is most likely to be in the identified model part, that is, the abnormal model part is likely to correspond to a defective imaging area of ​​the product. For example, one way to identify a defective imaging area is to repeatedly form a loss function for different respective appropriate subsets of the model part and its corresponding acquired image, and perform a step of determining the value of a numerical parameter of each subset. Then, the defective imaging area in the imaging area can be identified as an imaging area, the value of which after the minimization process is lower for the subset omitting the corresponding model part than for the subset including the corresponding model part.

[0037] In certain forms of the invention, the acquired images may be captured with a known positional relationship (e.g., the imaging process may indicate with high accuracy the location of the imaging area of ​​each image on the product). Alternatively, the method may include a process of inferring the imaging area on the product ("registering the images"). The process may be performed as part of a minimization algorithm. For example, each of the images may be associated with a corresponding displacement vector, which indicates the translational position of the corresponding image on at least one product (e.g., in a plane transverse to the imaging direction), and the displacement vector may be determined during the minimization process. This process may be performed iteratively, for example by repeatedly: (i) performing minimization relative to numerical parameters of the product model based on the current value of the (multiple) displacement vectors; and (ii) updating the displacement vectors (registration). Through experiments, it has been found that this process can achieve highly accurate registration, for example, accurate to a tolerance less than the pixel resolution of the acquired image (i.e., the tolerance is less than the distance on the product between the points corresponding to the centers of two adjacent pixels of the acquired image, i.e., the "width" of the pixel).

[0038] Optionally, two or more of the imaging regions of a given one of the products may overlap so that the images together form a continuous region on the product. In this case, the product model may also include numerical parameters describing the continuous region inferred from multiple of the images (effectively by stitching the acquired images together), which may be useful in the case where the images are captured successively by passing through the continuous imaging regions of at least one product under the imaging device that captured the acquired images.

[0039] For example, in the case where the manufacturing process is a wafer manufacturing process such as a photolithography process, the present invention is suitable for use.

[0040] The concept may be represented as a method or a computing system (e.g., part of a lithographic apparatus) programmed to perform the method. It may also be represented as a computer program product (e.g., downloadable software or a program stored in a non-transitory form on a recording medium, such as a CD-ROM) comprising program instructions operable to cause a processor to perform the method.

[0041] As above, optionally, the multiple acquired images may be images of the same imaging area, which are captured using different imaging processes, such as imaging processes characterized by (multiple) imaging parameters that take different values ​​in each imaging process. Possible imaging parameters include the imaging parameters listed above. The concept constitutes a second aspect of the invention, independent of the first aspect. Therefore, the second aspect of the invention provides a method for measuring at least one product of a manufacturing process, comprising: imaging at least one product using an imaging system through an imaging process characterized by at least one imaging parameter, wherein for multiple different corresponding implementations of at least one imaging parameter, an imaging unit of the imaging system captures multiple images of at least one imaging area of ​​the at least one product; and using the multiple images of the at least one imaging area together to obtain a product model of the at least one product. The method can be performed by an imaging system, the imaging system comprising an imaging unit (e.g., comprising a camera) and a control system configured to change the imaging parameters, for example, comprising a drive system for moving at least one product and / or the imaging unit to change the imaging parameters related to the relative translation and / or orientation position of the product and the imaging unit.

[0042] A third aspect of the invention relates to the provision and training of a neural network model configured to generate a reconstructed image of a target object from an input image of the target object. The reconstructed image represents the target object at a higher spatial resolution than the input image. The neural network can be trained using a training data set comprising a plurality of training items. Each training item comprises an acquired image of the training object and a computationally generated high-resolution training image corresponding to the corresponding acquired image. For example, the high-resolution training image can be generated according to the first aspect or the second aspect of the invention. Alternatively, other known computational methods (e.g., other known iterative deconvolution methods) can be used to generate the high-resolution training image. The training object is a product of the same manufacturing process as the target object. Therefore, in the absence of defects in the manufacturing process, the training object is substantially the same as the target object. Therefore, the neural network is typically specific to the manufacturing process, however, ideally, it is generated without knowing the design data defining the manufacturing process. The acquired training image can have substantially the same spatial resolution as the input image.

[0043] The phrase "same manufacturing process" may be defined here to mean that the target object and the training object are products (e.g., semiconductor products) manufactured according to the same design data. In other words, part or all of the target object and the training object may be manufactured using different instances of manufacturing equipment (e.g., lithography equipment), but the manufacturing equipment is controlled based on the same design data. Ideally, the manufacturing equipment used for all instances of the manufacturing process is produced according to the same specification (e.g., it is different instances of the same type of lithography equipment).

[0044] In one example, the target object may include an elongated object having a width substantially equal to the width of the elongated object included in the training object. Additionally or alternatively, the target object may include an object having a diameter (eg, width) substantially equal to the diameter of an object included in the training object.

[0045] For example, the neural network described above (once trained) enables the generation of high resolution images from acquired low resolution images in a computationally lightweight manner, for example compared to the first and second aspects of the invention. This is achieved by training the neural network described above with images (acquired low resolution and generated high resolution) of an object substantially identical to the target object without defects. Training in this manner promotes the neural network to learn computationally inexpensive image-to-image mappings rather than more complex image enhancement tasks, such as the first and second aspects of the invention. Therefore, the neural network invention described above can improve the processing time required to generate high resolution images, and can enable applications that require the generation of high resolution images from acquired images in real time. In addition, the process can be performed by a computer system that is operated by an individual who has only limited access to design data, for example if the manufacturing process runs perfectly, it is unknown how the training object and the target object appear.

[0046] Optionally, the neural network may also receive as input (during training and use) corresponding imaging parameters, which characterize the imaging process used to acquire the input images and the training images, respectively.

[0047] Training the neural network model using the training data set may include adjusting network parameters to reduce the difference between each image in the generated high-resolution training images of the training data set and a corresponding output image, which is generated by inputting the corresponding acquired training image into the neural network model.

[0048] The neural network model may include at least one of an autoencoder, a variational autoencoder, and a U-Net architecture. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Embodiments of the present invention will now be described, by way of example only, with reference to the accompanying drawings, in which:

[0050] Figure 1 A bright field microscopy inspection tool is schematically shown;

[0051] Figure 2 The first inspection task encountered in the semiconductor manufacturing process is illustrated;

[0052] Figure 3 illustrates a second inspection task encountered in a semiconductor manufacturing process;

[0053] Figure 4 is a flow chart of a method as an embodiment of the present invention;

[0054] Figure 5 Schematically illustrates Figure 4 Examples of methods;

[0055] Figure 6 Will Figure 1 The images captured by the system shown are similar to those captured by Figure 5 The images with higher resolution obtained by the method are compared;

[0056] Figure 7 Illustrated is a Graphic Data System (GDS) image of a portion of a semiconductor product and an inset Scanning Electron Microscope (SEM) image of the portion of the location.

[0057] Figure 8 The GDS image is shown in Figure 2. Figure 5 Overlay of high-resolution images recorded by the method.

[0058] Fig. 9 shows the corresponding parts based on one or more semiconductor products through Figure 5 How the differences between the series of reconstructed images produced by the method allow the identification of defects.

[0059] Fig.10 schematically illustrates an example neural network as an embodiment of the present invention;

[0060] Fig.11 is used Fig.10 Flowchart of an example method for a neural network.

[0061] Fig.12 is used Fig.10 Flowchart of an example method for training a neural network.

[0062] Fig.13 A method of generating a high-resolution image from images acquired using different imaging modalities is illustrated.

[0063] Figures 14 to 16 Shown by using Fig.10 Experimental results obtained by training neural networks. DETAILED DESCRIPTION

[0064] First refer to Figure 1 , shows a schematic diagram of a bright field microscopy semiconductor inspection tool 100 suitable for use in embodiments of the present invention. The tool is used to obtain an image of a product, such as a manufactured die 10 on a semiconductor wafer 11. A light source 101 and a reflector 102 generate a light beam, which is focused by lenses 103, 104 and then provided to a beam splitter 105. The beam splitter 105 generates a light beam (in Figure 1 The beam is focused onto the product 10 by the objective lens 106. The reflected light passes through the objective lens 106 back to the beam splitter 105, which redirects it to the lens 107. The lens 107 focuses the beam onto the detector array 108. (For visual simplicity, the Figure 1A single detector array 108 is shown in FIG. 1 , but more typically, the reflected light is split and provided to multiple detector arrays. ) For example, the detector array 108 may be a TDI detector array. The product 10 forms an image 109 on the detector array 108 .

[0065] Semiconductor inspection tool 100 is just one example of a semiconductor inspection tool that may be used with the techniques disclosed herein, and in fact any semiconductor inspection tool that forms image(s) of a product may be used, such as a dark field microscopy inspection tool or a scanning electron microscope imaging tool.

[0066] Figure 2 A first product inspection task encountered when inspecting products, such as wafers manufactured by photolithography processes, including integrated circuits, is schematically illustrated. Two wafers 20, 21 are depicted, which can be, for example, consecutive products of a manufacturing process. Each of the wafers 20, 21 includes a plurality of structures formed by a manufacturing process having a periodicity indicated by the dashed lines. That is, within each of the rectangular regions ("fields") defined between four of the dashed lines, the wafers 20 and 21 are intended to have substantially the same structure, as defined by the design data.

[0067] Furthermore, even within a single domain in a domain, there can be multiple structures that are intended to be the same. Some of these structures are Figure 2 20. Thus, circles 22 and 24 represent "same" structures within a single domain on wafer 20 (according to design data). Circles 22 and 23 represent "same" structures at the same location in different domains within wafer 20. Circles 22 and 25 represent "same" structures at the same location on different wafers 20 and 21.

[0068] Figure 3 A second product inspection task encountered when inspecting products, such as wafers manufactured by photolithography processes, including integrated circuits, is schematically shown. A structure is depicted, which may be, for example, Figure 2 One of structures 22, 23, 24, 25. The structure is defined in a three-dimensional coordinate system xyz.

[0069] Figure 3The structure of the invention comprises multiple layers (two layers in the example) of elements 31, 32, 33, 35, 36, 37, arranged in respective planes 34, 28 separated by a distance z0 in the z direction. For example, the elements 31, 32, 33, 35, 36, 37 may be bodies of conductive material arranged in an insulating or semiconducting matrix (not shown). The elements 31, 32, 33 in the plane 34 are elongated, wherein the elongation direction of the element 33 is indicated by L1. The elements 35, 36, 37 in the plane 38 are elongated, wherein the elongation direction of the element 37 is indicated by L2. A certain corner of the elongated element 31 is marked 353, and this has a displacement x0 in the x direction and a displacement y0 in the y direction relative to the corresponding corner 351 in the structure 35. The inspection task includes determining the values ​​of x0, y0 and z0 and the orientation of each of the directions L2 and L3 in the xy plane and / or the angle between the directions L2 and L3.

[0070] Figure 4 The steps of the method 400 according to the present invention are shown. Figure 1 The application of the inspection process first refers to Figures 5 to 9 Then, we will discuss how to Figure 4 The method 400 is applied to perform Figure 3 inspection task.

[0071] The method 400 may be performed by a computer processor. In a first step 401 of the method 400, N images of at least one product are acquired (e.g., received from an imaging unit that acquires them by capturing images; for example, the imaging unit may be as follows: Figure 1 The imaging process (or processes) for capturing the acquired image is described by an imaging model (or a plurality of corresponding imaging models).

[0072] In steps 402, 403, the N images acquired in step 401 are used in combination with the imaging model(s) of the imaging process(es) that captured the N images to determine the numerical value of the product model. Specifically, in step 402, a loss function is formed based on the numerical parameters of the product model, the acquired images, and the imaging model(s). In step 403, the numerical parameters of the product model are determined by minimizing the loss function with respect to the numerical parameters of the product model (and optionally also with respect to other parameters, as described below).

[0073] In step 404, using the product model, it is determined whether the product model meets anomaly criteria indicating the presence of a defect. Optionally, the location of the defect on at least one product may also be estimated. If a defect is detected, additional metrology and / or defect inspection is performed. Alternatively or additionally, step 404 may include metrology (measurement) of the product model.

[0074] We will now describe Figure 4 The method is Figure 2 A first example of application of an inspection task. Figure 2 The two wafers 20 and 21 are two different products, for example, produced continuously by continuously executing the wafer manufacturing process. In step 401, a low-resolution bright field image can be acquired to show the corresponding imaging area. The image is acquired by {P k}, where k is an integer exponent, k = 1,…N. Figure 5 Three of the acquired bright field images are shown as 51. Image 51 is blurry and has poor spatial resolution.

[0075] Each acquired image 51 is an n×n array of pixels, where n is an integer (for simplicity, a square array of pixels is considered, but in variants the array need not be square), and each pixel is associated with a corresponding intensity value.

[0076] Each acquired image 51 has a corresponding imaging area on one of the wafers 20, 21 containing the same (or similar) structures. For example, each acquired image 51 may have a corresponding imaging area including one of the structures 22, 23, 24, 25.

[0077] In this example of method 400, the product model may be Figure 5 A set of N images, depicted as 53 in FIG. 5 , have a one-to-one correspondence with the acquired image 51. Each image 53 is an array of m×m pixels, where m is an integer, and where each pixel is associated with an intensity value. The intensity values ​​of all images 53 together form the numerical value of the product model. The image 53 is referred to as a “reconstructed” image. The reconstructed image is represented by {X k}. They are not as blurred as the acquired image 51 and therefore have a higher spatial resolution. In addition, m can be greater than n. The reconstructed image 53 can have a higher pixel resolution than the acquired image 51 because a given distance on the product spans a greater number of pixels in the reconstructed image 53 than in the acquired image 51.

[0078] In this case, the imaging model includes a convolution 54, which is assumed to be known, and which applies a blur defined by a point spread function B, which is a two-dimensional array of values ​​(kernel), followed by a pixel resolution reduction process 56 that reduces the pixel size to n×n. In other words, the N reconstructed images 53 are images that, if they are convolved with the kernel B, result in N corresponding arrays 55 of convolution values ​​(which may also have a size of m×m), and if the pixel size of each of the arrays 55 is reduced to n×n, N images 52 are generated that are similar to the corresponding one of the acquired images (i.e., by blurring and having a lower spatial resolution). These N images 52 are referred to herein as "corrupted images". Note that each of the corrupted images 52 corresponds to one of the acquired images 51 and is an image of the same imaging area on one of the products 20, 21.

[0079] In a simple form of the pixel resolution reduction process 56, m is a multiple of n (i.e., m=an, where a is an integer), so that each pixel of the image 52 corresponds to a corresponding a×a patch of pixels of the convolution array 55. Therefore, to perform the pixel resolution reduction process 56, the intensity value of each pixel of each corrupted image 52 can be obtained as the average of the intensities of the corresponding patches of the convolution array 55.

[0080] A more complex form of process 56 is based on the observation that, due to the convolution 54, parts of the product that are slightly outside the imaging area (i.e., the area on the product imaged in the image 51) affect the image 51. For this reason, the reconstructed image 53 is preferably an image of the corresponding area on the product 20, 21, slightly larger than and including the corresponding imaging area. In other words, the area of ​​the product(s) depicted by each corresponding reconstructed image 51 is the imaging area of ​​the corresponding acquired image 51 plus a fringe area around the periphery of the imaging area, the width of which may be approximately half the point spread function defined by the kernel B (e.g., if the amplitude of the kernel is essentially a Gaussian function of the distance from the center position, the width of the fringe may be in the range of one to three standard deviations of the Gaussian function). In this case, process 56 may include removing the fringe from the array 55 of convolution values ​​(which may be thought of as applying a mask to the array 55), and then reducing the dimensionality of the result to n×n.

[0081] The reconstructed images 53 may be considered as intensity images which would be obtained if the imaged area of ​​the product (plus the edge areas) were imaged by an imaging process with a higher resolution than the imaging process which produced the corresponding image 51 .

[0082] In principle, a large number of possible reconstructed images 53 have Figure 5The properties shown (i.e., the problem of forming a reconstructed image 53 using the acquired image 51 is ill-posed). However, this application of method 400 exploits the fact that there is some prior knowledge about the reconstructed image 53. First, it is known that the complexity of the reconstructed image 53 is limited, with a relatively small number of top-down features visible on the patterned stack. In addition, the desired device structure has smooth local variations (e.g., patterned lines are formed by spots of material rather than sharp features). Additionally, there are similarities between the reconstructed images 53 in that their corresponding imaging regions contain the same or similar structures. The prior knowledge is used to define the terms of the loss function as functions of the reconstructed image 53 (as well as the acquired image 51 and the imaging model), so that the loss function is minimized with respect to the reconstructed image 53, ensuring that the reconstructed image 53 conforms to the prior knowledge, and also by Figure 5 The process shown matches the measured data.

[0083] Specifically, the loss function can be in the following form:

[0084]

[0085] Here, the square brackets [...] denote the concatenation of the elements within the brackets, M denotes the process 56 of applying a mask and reducing the pixel size if necessary, F denotes Fourier transform, and F -1 represents the inverse Fourier transform. k represents the kth acquired image 51. k represents the k-th reconstructed image 53. Therefore, F -1 (FX k FB) represents the reconstructed image X by applying convolution 54 k The obtained convolution value is 55, and MF -1 (FX k FB) represents the corrupted image 52. Note that the Fourier transform (i.e., the transform from the spatial domain to the spatial frequency domain) is employed because it is a computationally efficient way to perform the convolution operation represented by the kernel B, for example using a Fast Fourier Transform (FFT) operation; in principle, the convolution operation can be implemented directly in the spatial domain rather than via a Fourier transform. Represents the Frobenius norm of the matrix, which is the squared l2 norm of the matrix in vector format. This quantifies the goodness of fit between the measured and reconstructed data.

[0086] is the regularization term, including the structural term D TV X k W represents wavelet transform (several wavelet transforms are known; the wavelet transform used in this experiment is a wavelet transform based on Haar wavelet). TV represents a well-known operator that converts Xk Convert to image gradient domain. It uses nearby pixel differences (in horizontal, vertical and / or diagonal directions) to encode information. is the l1 norm, i.e. the sum of absolute values. Here, it is applied to [WX k ;D TV X k ;X k ], which means WX k , D TV X K and X k The concatenation.

[0087] D represents the matrix X k Operations converted to vectors. ||·|| * represents the nuclear norm operation. The norm is calculated as the absolute single value sum of the concatenation of all vectorized images, i.e. [DX1, DX2, ..., DX N ].

[0088] α and β are hyperparameters that determine the relative importance of terms in the loss function.

[0089] The minimization algorithm can then be expressed as finding:

[0090] For X k For all k and all points on Satisfy 0≤X k ≤M. (2) Therefore, X k Each point in is constrained to be in the range of 0 to the upper pixel intensity limit M. The image restoration task (i.e., obtaining a reconstructed image) can be described as a deconvolution task with smoothness and low-rank constraints.

[0091] Specifically, item The corrupted image 52 is encouraged to be similar to the acquired image 51, i.e., data consistency is ensured. Note that equation (1) expresses properties in the Fourier domain, since it is easy to express the convolution task using kernel B as a multiplication in the spatial frequency domain.

[0092] Regularization term Encourage X k The wavelet components WX include desirable features of the reconstructed image, such as regions of uniform intensity and well-defined lines. k and based only on X k The amount of encouragement X k Has a low filling ratio. Structural Item D TV X K Encourage the presence of edges. Optionally, to promote the recovery of horizontal and vertical lines (assuming that the x-axis and y-axis in image 51 are strongly related to the elongation direction of the elongated elements in the product), D can be defined TV, so that its weight in the vertical and horizontal directions is higher than that in the diagonal direction. Step d can be introduced between the pixels used in the calculation (for example, D TV Defined as computing the difference between horizontal / vertical pixels rather than nearest neighbor pixels, which are separated by a distance of d pixels, where d is greater than one).

[0093] Item||[DX1, DX2, ..., DX N ]|| * Encourage Image X k is similar because they are images of corresponding regions containing similar structures (e.g., based on the same design data or design data that meets similar criteria). The requirement is encoded in (1) by ensuring that the concatenated reconstructed images produce a low-rank matrix. This property is encoded by a convex relaxation of the low-rank property, namely the nuclear norm || [DX1, DX2, ..., DX N ]|| * .

[0094] The hyperparameters α and β can be chosen by trial and error to produce a reconstructed image with the desired properties53. For example, if high-resolution images are available for some product, such as an SEM tool, the hyperparameters α and β can be chosen to ensure the best match to that image. Note that the setting of the hyperparameters α and β typically only needs to be done once, during the “setup” phase, so it is worth incurring the cost of obtaining high-resolution images so that the obtained hyperparameter values ​​can be used later in Figure 5 is used to inspect a large number of other products in the process.

[0095] The equivalent formula can be expressed as a constrained optimization of the data fidelity part. That is, the loss function can be expressed as:

[0096]

[0097] where minimization is redefined as

[0098]

[0099] where ∈ k is an arbitrarily small value, or can be chosen to match the noise statistics of the measured data. If the noise statistics of the measured data are known, this can make it easier to define the cost function.

[0100] To test the method 400, bright field microscopy images were acquired from a real wafer manufacturing process, each consisting of 32×32 pixels. The acquired bright field microscopy images were from multiple locations on the wafer where the same pattern was intended to be printed (i.e., the structures in these imaged locations were based on the same GDS). Equations (3) and (4) were solved. Figure 6One of the acquired images is shown on the right, including the low-resolution features in region 62. The corresponding reconstructed image is shown on the left, and in region 61 corresponding to region 62 the super-resolution features are clearly visible.

[0101] To check whether the super-resolution features are spurious, Figure 7 A GDS clip (ie, design data) for this location is shown. Figure 7 Measured high resolution SEM data of the same GDS clip is also shown as an inset. Figure 8 The reconstructed image is shown overlaid with the GDS clip, showing clear consistency. This demonstrates the ability of the embodiment to reconstruct high resolution images of the stack, particularly since no information from the GDS is used in the reconstruction process.

[0102] Fig. 9 The pixel-to-pixel differences between several reconstructed images 53 are shown. Fig. 9 There are three sections, each showing the difference between corresponding pairs of reconstructed images as brightness levels. It can be seen that in regions 91 and 92 the difference is high. This indicates that these areas in the structure are affected by the defect. In other words, the anomaly criteria are defined based on the difference between the reconstructed images corresponding to the imaged areas of the product manufactured using the same design data.

[0103] The above technique has several possible applications. As above, the first such application is as a measurement scheme for multiple locations (imaging regions) on one or more wafers, using acquired images of structures that are intended to be similar (e.g., having the same GDS design). These can be different locations (domains) on a single wafer (e.g., Figure 2 positions 22, 23, 24 on the wafer) and / or the same corresponding positions on multiple wafers (e.g. Figure 2 Positions 22 and 25 on the diagram). Fig. 9 As shown, the structures in different acquired images do not need to be exactly the same, i.e., we can tolerate process variations as long as they share the same overall features.

[0104] In a variation, the acquired images may all relate to the same imaging area on the wafer. Specifically, each of the N acquired images may be acquired using different imaging parameters (eg, control parameters) in the imaging unit (sensor) that captured the acquired image. k The image obtained by the collection. Imaging parameters c k Each of these can be explained by having a different imaging model for each k, for example by replacing the convolution kernel B in (1) to (2) or (3) to (4) with a kernel that depends on c k The convolution kernel B kThe imaging parameters may include different sensor-to-target alignment conditions: slight vertical and / or horizontal motion of the sensor relative to the target, rotation of the sensor relative to the target (e.g., rotation about the z-axis if the indicated plane of the wafer is the xy-plane; note that in this case the rotation is accounted for in the imaging model, e.g., by varying the D TV And the convolution kernel B k ), tilt (i.e., rotation about the x-axis and / or y-axis), observation channel configuration (e.g., wavelength, focus, “dose” (associated with the power reaching the sample), and exposure time, where different settings are again obtained by different convolution kernels B k quantify).

[0105] Note that this variant can be combined with the possibility of using multiple images from different imaging positions. That is, there can be images with different corresponding imaging parameters c k The plurality of images are from each of the plurality of locations on each of the one or more products.

[0106] Another variation of equations (1) to (2) or (3) to (4) is in a continuous measurement scheme where the sensor moves relative to the product while recording images. Depending on the acquisition rate and the speed of motion, the same target can be in the field of view of several acquired images. This allows scanning of a continuous (continuous) area of ​​the wafer by defining a single reconstructed image based on multiple of the acquired images. That is, the reconstructed image can have a continuous area of ​​the product (e.g., a wafer) that spans the imaging area of ​​multiple acquired images (i.e., it "stitches the imaging area together").

[0107] In all these applications, Figure 4 In step 404 of method 400 of (hereinafter referred to as "super-resolution"), the reconstructed ("super-resolution") image may be used to detect defects in the observed product(s). There are several options for defining anomaly criteria for the presence of defects, such as: comparing the super-resolution image to the intended design and reporting significant deviations from the intended structure; and comparing multiple images from locations where the same structure is expected to be located and observing any deviations that exist in a limited subset of the images. Once an anomaly is detected, the area that meets the anomaly criteria is optionally further inspected using a specialized low-throughput tool (SEM).

[0108] Furthermore, for any application, step 404 of method 400 may include extracting metrology information, such as critical dimension (CD) or overlay, by measuring the reconstructed image(s), similar to how it is conventionally done from SEM measurements.

[0109] Several variations of the above process will now be considered. The first variation is when the imaging parameter(s) of the imaging process(es) are unknown, for example known only within a certain range. Specifically, in one example, kernel B is identified from the data.

[0110] This can be performed in two ways: (a) as a two-step process, where a known target is imaged (perhaps a target with a single feature that acts as a Dirac delta function in the image domain) and B is estimated from the image; after this estimation, the value of B can be used in (1)-(2) or (3)-(4); or (b) as a one-step non-convex optimization task, where the minimization algorithm solves {X k} and optimize the value B. Specifically, the loss function can be modified as follows:

[0111]

[0112] And the minimization algorithm of equation (2) is reformulated as:

[0113] For X k For all k and all points on Satisfy 0≤X k ≤M. (6)

[0114] In addition to the reconstructed image {X k}, in this formulation, the minimization algorithm is also searching for B that is consistent with all N observations (because it is a property of the imaging system). Typically, optical elements act as low-pass filters, so these properties can be encoded into the optimization task. An example is the property that B has a compact support in the Fourier transform. That is, it has non-zero gain for a finite number of frequencies.

[0115] In the above embodiment, it is assumed that the image alignment of the acquired images 51 is accurate (i.e., the corresponding structures on the product shown in each acquired image are in the same position in the acquired image). However, in some imaging systems, the image alignment between observations will be slightly different (although typically only by a fraction of a pixel size in the acquired image). Therefore, in a second variant of the embodiment, the information content between the two reconstructed images is (re)aligned. One purpose of doing this is to enforce the consistency property between the images, just as the above embodiment aims to use the low rank property defined by the nuclear norm. One way to correct for the alignment offset is to represent the offset of the kth acquired image in the pixel domain as a phase shift matrix {S in the Fourier domain}. k}, and replace the nuclear norm term in equation (1) (or equation (3)) with in

[0116] Matrix {Sk} can be found by an iterative method, where the minimization algorithm of equation (2) (or equation (4)) is used with {S k} are solved multiple times with different implementations. k} can be an identity map and is updated after each execution of the minimization algorithm to achieve a better distribution between the high-resolution images.

[0117] Alternatively, by replacing {S k} as the variable to be optimized in the minimization algorithm, we can obtain the matrix {S k} as a non-convex problem. This can be achieved by changing the variables to and Solve, where Note that the matrix {S k} has a very specific structure, in fact each matrix depends on the known dimensions on the two image axes and two unknown offsets. While in the image domain these offsets are necessarily discrete due to the pixel grid, in the Fourier domain they can be treated as continuous variables. The loss function then becomes:

[0118]

[0119] And the minimization algorithm becomes:

[0120] For X k For all k and all points on the k ≤M(8)

[0121] Many variations are possible within the scope of the invention. For example, although the above embodiments use bright field microscopy images {X k}, but the acquired image can alternatively be a SEM measurement. Here, the raw signal from the measured structure is observed by the SEM detector after passing through the different focusing systems of the SEM tool. This is similar to low-pass filtering using an unknown filter kernel. In view of this, there are similarities with bright field measurements, so the same data processing framework can be applied. Therefore, the acquired images from fast SEMs (optimized for speed rather than accuracy) can be enhanced.

[0122] In another variation, only a subset of the points in the field of view (imaging area) of the SEM tool may be measured. That is, the SEM tool may be configured to measure / excite only a portion of the points from the field of view (given appropriate sampling: e.g. uniform random or pseudo-random), thereby providing faster measurements. In this case, the above techniques may be applied to fill in the missing information. This is possible because each observed signal (e.g. SEM pixel of an acquired image) is a sample of the electron interactions in the device given a mixing, blurring kernel. Therefore, each individual measured pixel contains information from nearby structures (which are blurred).

[0123] Additionally, although the product model in the above embodiment includes the reconstructed image {X k}, but the product model may alternatively be a model in which the structure of the product is geometrically defined based on geometric parameters (such as numerical values ​​of lengths (dimensions) or angles in the product model). For example, the product model may be defined based on a plurality of elements (e.g., all in one plane, or different ones of the elements in a plurality of spaced-apart planes), and the geometric parameters may include the dimensions of the elements and / or directions defined by the elements and / or angles between predetermined directions.

[0124] For example, consider again Figure 3 The product shown comprises R (e.g. 6) elongated elements 33, 32, 31, 35, 36, 37. These elements can be labeled with r=1, ... R. A product model can be defined in which each element has a corresponding geometric parameter g r The geometric properties defined by the set (vector). In the case of elongated elements, the geometric parameters g of the rth element r It may include, for example: the translational position of a point (e.g., the center of mass) of the elongated element (e.g., a two-dimensional position in an xy plane parallel to layers 34, 35 and a z position in a direction separating layers 34, 35); the length of the elongated element; the width of the elongated element; the angle between the length direction of the elongated element in the product and a reference direction; or the angle between the corresponding length direction and the length direction of another elongated element in the product (e.g., an elongated element in another layer).

[0125] N images of (multiple) products {P k Different relationships corresponding to the imaging parameter set {c k} to obtain. For a given implementation of imaging parameters c k , the rth element can be in the image P k The intensity pattern is generated in the r ,c k ). Therefore, ignoring the interaction between components, the image P kThe corresponding damaged image will be represented by ∑ r f(g r , c k ) is given.

[0126] The loss function can be defined as:

[0127]

[0128] Then, by using Relative to the numerical parameter {g r} is minimized, the numerical parameter {g r}.

[0129] Here, as part of the imaging model, it is assumed that each of the R elements will be k In more sophisticated variants of the algorithm, the imaging model can take into account interactions between elements, such as elements in one layer of the product partially occluding elements in the next layer. To this end, numerical simulations can be used to perform the r} The process of generating corrupted images. This information can be included in a more complex version of the loss function in Eq. (9).

[0130] At another level of complexity, the product model may include elements other than the elongated elements 33, 32, 31, 35, 36, 37, such as elements having another shape or elongated elements extending in the z-direction (such as vias, for example).

[0131] In some embodiments, performing the above process (particularly referring to Figure 4 The computational processing of the captured image described in steps 402 and 403 of FIG. 400 may be computationally time-intensive or computationally resource-intensive. For example, as described above, method 400 may be implemented using an iterative method, wherein the minimization algorithm of equation (2) (or equation (4)) requires using {S k} are solved multiple times for different implementations of N. The computational cost of this iterative method can be very high. In addition, for larger values ​​of N (i.e., for larger sets of images), method 400 can generate more accurate values ​​for the parameters describing the product model. Processing larger sets of images can increase the time or computing resources required to perform the method. Therefore, implementing method 400 can require expensive computing hardware to achieve real-time processing, or alternatively take an unnecessarily long time for certain applications.

[0132] Thus, for some applications (e.g., where real-time processing is required), it may be desirable to provide a process that is computationally cheaper than process 400 but that approximates process 400 (or at least portions thereof, such as steps 402 and 403) well enough to generate useful estimates of numerical parameters that define a product model or images of the product that contain higher spatial resolution information than the captured images.

[0133] Deep learning neural networks are well known for performing image enhancement on input images. For example, a neural network model can be trained to map a highly convolved image to a corresponding deconvolved image, i.e. such a model can learn an approximation of the convolution task. The generated image can contain higher spatial resolution information than the (highly convolved) input image. To this end, the neural network model can be trained using reference images representing the same sample with different spatial resolutions (e.g. low-resolution and high-resolution images of the same sample can be provided to the model during training). However, such reference images are often unavailable or very expensive to acquire. For light-based imaging tools (e.g. brightfield microscopy), it may not be possible to obtain images with "high spatial resolution" because the achievable resolution (e.g. limited by the properties of the light employed) may be too low to resolve features of the product of interest. For scanning electron microscopy (SEM) tools, data with high-resolution measurements are often slow to acquire or suffer from high noise levels. In addition, images may not be available due to confidentiality issues. Furthermore, neural network models trained on acquired reference data may not generalize well to other use cases (e.g. products with completely different geometric features) and new, use case-specific reference data may be required. Therefore, it is desirable to provide a process for training a neural network that facilitates the neural network to learn an approximation of the deconvolution task without requiring high-resolution images to be acquired during training. Figures 10 to 15 Describe this process (and the corresponding system).

[0134] In general, a training data set is created by computationally performing image enhancement on a number of acquired images. For example, a high spatial resolution image can be computationally generated from an acquired low resolution image using, for example, the process 400 described above or any other suitable computational method. The generated high resolution image and the acquired low resolution image can then be used as training data to train a neural network so that the neural network generates an output image from a low resolution input image that (closely) matches the generated corresponding high resolution image. Once trained, the neural network can be deployed to perform the learned image enhancement task, i.e., generate a high resolution image from a low resolution input image. Because the high resolution training images are generated by a computational process, there is no need to acquire a high resolution reference image to train the neural network.

[0135] In some cases, where the acquired (low-resolution) training images are a set of images representing the same (or similar) structure (such as process 400 described above), the trained neural network model can perform the image enhancement task several orders of magnitude faster than the computational process used to generate the high-resolution training data. This is because by only providing training images of the same (or similar) structure, the neural network is not encouraged to learn a "general image enhancement task" (i.e., a process that can be successfully applied to images representing a wide range of structures), but rather an image-to-image mapping that is specific to a particular product structure (e.g., a product of a single manufacturing process), so that it can successfully enhance images of the same (or similar) structure used to create the training data set.

[0136] Reference Fig.10 , shows a schematic diagram of a neural network 94. The neural network 94 has a plurality of network parameters and is configured to receive an input image and process the input image according to the network parameters to generate an output image. Specifically, the neural network 94 is configured to perform an image enhancement task, such as generating a high-resolution image from a low-resolution input image. Suitable values ​​of the network parameters can be determined by performing the following reference Fig.12 The neural network 94 comprises a convolutional neural network, for example, having a known “U-Net” structure (Ronneberger O, Fischer P, Brox T (2015). “U-Net: Convolutional Networks for Biomedical Image Segmentation”. arXiv: 1505.04597), such as Fig.10 As shown, etc. It should be understood that other model architectures can be used, such as (variational) autoencoders. In general, any neural network model that can perform image-to-image mapping can be suitable. A system of one or more computers located in one or more locations can be used to implement the neural network 94.

[0137] The neural network 94 is configured to receive as input an image (ie a two-dimensional array of intensity values). Fig.10 Three input images 95 are shown which may be provided (sequentially) to a neural network 94. Fig.10 The input image 95 shown is acquired using a SEM tool. In other embodiments, the input image may be acquired using another tool, such as a bright field imaging tool or a dark field imaging tool.

[0138] The image provided (as input) to the neural network 94 may be an image of a product of a manufacturing process, such as a semiconductor product manufactured on a wafer. Fig.10 The example image 95 shows a pattern of holes. In general, the input image to the neural network 94 can be an image captured by performing an imaging process on an imaging area on one of the wafers. In the same manner as described above, for the acquired image for process 400, the imaging process used to capture the input image to the neural network 94 is also performed by an imaging device and can be characterized by at least one imaging parameter. In an embodiment, the at least one imaging parameter can specify the configuration of the imaging device. Broadly speaking, the image quality of the input image provided to the neural network 94 is generally insufficient, for example, the image does not adequately resolve certain details of the imaged product. In other words, the input image can have a low spatial resolution. For example, the example input image 95 is too noisy and can only poorly resolve the hole pattern.

[0139] The neural network 94 is configured (processing the input image according to the network parameters) to generate a reconstructed image having a higher resolution than the input image. Thus, the generated image may contain higher spatial resolution information than the input image. For example, the neural network may process the example input image 95 to generate a high resolution output image 96. The resolution of the hole pattern in the output image 96 is better than that in the input image 95. For clarity, note that although Fig.10 Three input images and three output images are shown, but the neural network 94 may process each input image 95 individually (ie, independently of the other input images) to generate a corresponding high resolution image 96.

[0140] In an embodiment, the neural network 94 is further configured to receive as input (in addition to the input image) at least one imaging parameter characterizing an imaging process used to capture the input image. In an embodiment, the neural network 94 may be configured to process the input image and the at least one imaging parameter to generate a reconstructed image having a higher resolution image than the input image.

[0141] Fig.11A flow chart of an example process 110 for processing an image with a neural network 94 to generate a reconstructed image having a higher resolution image than an input image is shown. In step 111, an input image (having a low spatial resolution) is received. In an embodiment, at least one imaging parameter is also received in step 111, the imaging parameter characterizing the imaging process used to capture the input image. In step 112, the neural network 94 processes the input image to generate a reconstructed image having a higher resolution image than the input image. In embodiments where the at least one imaging parameter is provided to the neural network 94, the neural network 94 processes the input image and the at least one imaging parameter to generate a reconstructed image having a higher resolution image than the input image.

[0142] Fig.12 A flow chart of an example process 120 for training a neural network 94 is shown. In general, the neural network 94 is trained using a training data set that includes a plurality of training items. Each training item includes an acquired image of a training object and a corresponding computationally generated image that contains higher spatial resolution information than the acquired image.

[0143] In step 121, a training data set is generated. To this end, a plurality of training images are captured by performing an imaging process on an imaging region of (a plurality of) corresponding training objects (e.g., the same training object for all training images or a corresponding training object for each training image). The training objects are substantially identical. For example, each of the training objects may be a product of a manufacturing process manufactured according to the same design data (e.g., a semiconductor product manufactured on a wafer, as described above with reference to FIG. 1 ). Figure 2 and Figure 3 products), such as training objects are intended to be identical, but may differ to some extent due to manufacturing defects or any other (unexpected) variations in the manufacturing process.

[0144] In an embodiment, each training item of the training data set may further include imaging parameter(s) characterizing the imaging process used to capture the corresponding image of the respective training item.

[0145] Furthermore, in step 121, the computational method is applied to the plurality of acquired training images to reconstruct a plurality of corresponding high-resolution training images (ie, the reconstructed images contain higher spatial resolution information than the acquired training images).

[0146] In an embodiment, the plurality of high resolution training images are reconstructed by performing steps 402 and 403 of process 400 (above) and generating reconstructed images based on the determined values ​​of the product model.

[0147] In another embodiment, a variation of the above computational method (i.e., steps 402 and 403 of process 400) may be applied, wherein each acquired training image is processed individually. For example, for each acquired training image, the high-resolution image X may be found by solving the optimization task

[0148]

[0149] where the symbols have the meanings described above with reference to equation (1), i.e., P is the corresponding acquired image, B is the kernel (representing, for example, the point spread function of the imaging device), W and D TV is a linear operator, M means applying a mask (and reducing the pixel size if necessary), F means the Fourier transform, and F -1 represents the inverse Fourier transform. As before, the term Data consistency is ensured.

[0150] In another embodiment, another variation of the above-described computational method (i.e., steps 402 and 403 of process 400) may be used. In an embodiment, a set of images of the same object (e.g., two images) are processed together to generate corresponding high-resolution images. Each image in the set may be acquired using a different imaging mode (e.g., the first image may be acquired at a low spatial resolution and a low noise level, and the second image may be acquired at a high spatial resolution and a high noise level). The embodiment may be particularly useful for integrated circuit inspection / measurement based on SEM imaging, where it is often observed that acquiring image data using a SEM requires a trade-off between speed, resolution, and accuracy. Acquiring SEM images with low energy (applied to the primary electron beam) can result in high spatial resolution, but can also result in a low signal-to-noise ratio (and a slow measurement speed). On the other hand, acquiring SEM images with high energy can improve measurement speed and signal-to-noise ratio at the expense of spatial resolution (due to the large interaction range, i.e., the large interaction range of the incident electrons with the sample). To illustrate this point, Fig.13 A first low resolution (and low noise) input image of a hole array structure is shown in panel 131, and a second high resolution (but noisy) input image of the same structure is shown in panel 132 (the images in panels 131, 132 are not acquired images, but rather images computationally generated from the "ground truth" image shown in panel 130 for illustrative purposes).

[0151] Furthermore, in embodiments, high-resolution images (e.g. Fig.13 Panel 133) can be reconstructed by performing a deconvolution task so that the high resolution image is reconstructed with the first input image and the second input image (the information included therein) (e.g. Figure 1Similar to the method described with reference to equation (1), in an embodiment, images of different instances of the same structure can be processed together to improve the deconvolution result. Specifically, multiple sets of images (i.e., images in each set are images acquired in different modes, and each set represents a different instance of the same structure) can be processed together to find a corresponding high-resolution image X for each input set by solving the optimization task k

[0152]

[0153] Wherein the symbols have the meanings described above with reference to equations (1) and (10), except that in order to enable the use of different imaging modes, multiple kernels B are used. i (Each kernel corresponds to a pattern.) Similarly, P k,i represents the acquired image, where the index k represents a specific set and the index i represents the imaging mode. As before, W and D TV is a linear operator, M means applying a mask (and reducing the pixel size if necessary), F means the Fourier transform, and F -1 represents the inverse Fourier transform. Data consistency is ensured. It should be appreciated that equation (1) can also be performed on a single input set (so in this case, there is no need to sum over k in equation 10).

[0154] Panel 133 shows the effect of applying equation (11) to Fig.13 The high resolution image is found by using the input images 131, 132. It is obvious that the reconstructed image 133 is very similar to the ground truth 130. This is further illustrated in panel 134, which shows that the difference between panel 130 and panel 133 is small. Therefore, the reconstructed image 133 has a high spatial resolution and a low noise level. The method described with reference to equation (11) can be used to perform step 121, that is, to calculate and generate multiple corresponding high resolution training images from multiple acquired training images. In this case, as above, the multiple training images include acquired image sets, wherein the images in each set are acquired using different imaging modes.

[0155] Although the method described above with reference to equation (11) has been illustrated using high resolution and low resolution SEM images, it should be understood that the set of imaging modes is not limited to these examples.

[0156] In general, any computational (deconvolution) method that performs an image enhancement task (i.e., generates a reconstructed image containing higher spatial resolution information than the acquired input image) can be used to generate the reconstructed image for the training dataset. For example, the computational method can be implemented using other neural network models.

[0157] In step 122, the neural network 94 is trained using the training data set. Specifically, for each training item of the training data set, the neural network 94 receives (as input) an acquired image of the corresponding training item, and processes the input image to generate (as output) a reconstructed image. The output image is then compared with the corresponding (computationally generated) high-resolution training image of the corresponding training item to determine the difference between the output image and the high-resolution training image. The network parameters are then adjusted so as to reduce the difference. In other words, the network parameters of the neural network 94 are updated so that the neural network 94 processes the acquired image to generate an output image that (substantially) matches the corresponding (computationally generated) high-resolution training image. It should be understood that several methods can be used to implement step 122 (e.g., optimization algorithms based on back-propagation and gradient descent), i.e., determining the network parameters based on the differences obtained from the training items.

[0158] In an embodiment, wherein each training item comprises at least one imaging parameter characterizing an imaging process used to capture a corresponding image of the respective training item, and the neural network 94 is configured to receive the at least one imaging parameter as a further input (in addition to the input image), for each training item, network parameters are adjusted based on a difference between the output image and the corresponding high-resolution training image and based on the corresponding at least one imaging parameter.

[0159] According to reference formula 10 and Fig.13 The described method performs Fig.12 In an embodiment of step 121 of , the neural network 94 may be configured to receive as input a set of acquired images, wherein the images were acquired using the same set of imaging modes that were used to generate the training images (e.g. Fig.13 The images 131 and 132 may be provided as inputs to the neural network). Alternatively, the neural network 94 (according to the reference formula 10 and Fig.13 The method described herein) can be configured to receive as input only images acquired in one of the imaging modes (e.g., only low-resolution, low-noise images).

[0160] Generally speaking, once trained, the neural network 94 can be used to process an input image to generate a reconstructed image having a higher resolution than the input image (as described above with reference to Fig.11). Because the images included in the training data set are related to the same (or similar) objects, the neural network learns a specific image-to-image mapping that can successfully enhance images of the same (or similar) objects used to create the training data set. Therefore, using the trained neural network 94 can be computationally lightweight compared to computational methods used to create high-resolution training images. Therefore, while the generation of the training data set can be computationally expensive, once trained (and deployed), the neural network 94 can process images quickly without requiring large amounts of computing resources (e.g., fast enough to allow real-time applications). In other words, in the above-described methods and systems, computationally expensive tasks are effectively pre-loaded into the training phase so that the system is computationally lightweight during deployment.

[0161] Once trained, the neural network 94 can be used to process images of objects similar to those represented in the training data set. The output images of the neural network 94 can be used for defect inspection. If the input image represents an object that differs from the training object by more than a certain degree, the neural network 94 may not perform the image enhancement task well, i.e., in this case, the generated output image may be (substantially) different from the reconstructed image generated from the input image by the computational method (used to generate the high-resolution training image). In general, the greater the difference between the object represented in the input image and the (multiple) training objects, the worse the accuracy of the image enhancement task performed by the neural network 94 may be. In other words, a trained neural network 94 can be used to display images of objects similar to the training objects, and if the neural network 94 is to be used with images showing substantially different objects, retraining may be required.

[0162] In embodiments where the corresponding imaging parameters are not provided (as additional inputs) to the neural network 94 during training and deployment, the accuracy of the image enhancement tasks performed by the neural network 94 may be worse if the imaging process used for the input images during deployment is (substantially) different from the imaging process used to acquire the training images. In embodiments where the corresponding imaging parameters are not provided (as additional inputs) to the neural network 94 during training and deployment (and the training images are acquired according to a wide range of different imaging parameters), the accuracy of the image enhancement tasks performed by the neural network 94 may not be worse if the imaging process used for the input images during deployment is different from the imaging process used to acquire the training images. This is because the neural network 94 (during training) may have learned to distinguish between effects on the input images caused by the imaging process and effects caused by objects represented in the input images. In this case, when the trained neural network 94 receives imaging parameters that were not included in the training data set, the neural network 94 can effectively interpolate to take into account the effects of the imaging parameters on the input images.

[0163] Some benefits of neural network 94 can be understood by considering an example workflow. A first semiconductor wafer may include multiple structures formed by a manufacturing process. The structures may be intended to be identical (i.e., they are manufactured according to the same design data). An imaging device may be used to acquire low-resolution images of the structures of the first wafer. These images may then be used to create a training data set for neural network 94. This means that these images may be used to create (possibly in a computationally expensive manner) reconstructed high-resolution images of the structures of the first wafer. Neural network 94 is then trained according to the training data set. When successive wafers are manufactured (i.e., wafers manufactured according to the same design data as the first wafer), low-resolution images of the structures of these successive wafers may be acquired using an imaging device, and neural network 94 may generate high-resolution images from these low-resolution images, for example, for defect inspection. Because processing images using a trained neural network 94 is computationally lightweight, neural network 94 may enable fast (e.g., real-time) defect detection, and thus may improve the operation of the manufacturing process.

[0164] Although neural network 94 is described in the context of defect detection for semiconductor products, it should be understood that neural network 94 may be used in a wide variety of applications.

[0165] Use according to Fig.13 The experimental results obtained by training an example neural network using the method are shown below. Figures 14 to 16 The example neural network is a model with a "U-Net" structure. The training data set includes 500 (low-resolution) SEM images showing a specific hole pattern, and correspondingly includes 500 high-resolution images, which have been computationally generated according to the optimization task described above with reference to formula 10. The training data set is used to train the example neural network.

[0166] Fig.14 Panel 138 shows an acquired SEM image as an example. Fig.13 Panel 135 of FIG. 1 shows the corresponding output image of the trained neural network. The hole structure is clearly better resolved in the output image 135 than in the acquired image 138. To illustrate the accuracy of the neural network, Fig.14 A high resolution image reconstructed using equation (10) is also shown in panel 136, and the difference between panel 135 and panel 136 is also shown in panel 137. The images in panels 135 and 136 are substantially identical, and the difference between them (as shown in panel 137) is small. Therefore, the trained neural network is able to effectively replicate the results obtained by performing deconvolution using the optimized formula of equation 10.

[0167] Fig.15 It is illustrated that a trained neural network works well even if the structures shown in the input image differ (slightly) from the structures shown in the training images. To this end, a synthetic example of an image of a different structure is created by perturbing the test input image. In particular, to create such a synthetic example 140, a copy of a region of the test input image is rotated and inserted into another region of the test input. Panels 142 and 143 show two of such synthetically generated images, and panels 141 and 144 show the corresponding output images of the neural network. Panels 141 and 144 show the altered hole structure in high resolution, demonstrating that the neural network can also perform image enhancement on images representing structures different from the training structure.

[0168] Fig.16 The usefulness of a neural network 94 for defect inspection is illustrated. In the absence of ground truth information available about what the structure should look like (e.g. because the information may be proprietary to another party), defect inspection may be based on detection differences between samples (typically there will be many samples that are manufactured correctly and a few that are defective). Fig.16 Panels 156 and 157 show SEM images acquired of two imaging portions of the wafer. The intention is to make the structures in the two portions identical. Panels 150 and 151 show the corresponding output images of the neural network, and for comparison, panels 154 and 154 show images obtained by performing deconvolution using the optimization formula of Equation 10. As above, the difference between the structures of the two imaging portions can indicate defects. Panels 152, 155, and 158 output the differences between panels 151 and 152, between panels 154 and 155, and between panels 157 and 158, respectively. The fact that panels 152 and 155 are almost identical indicates that the output image of the neural network is as suitable for defect inspection as the image obtained by performing deconvolution using the optimization formula of Equation 10. In addition, comparing panels 152 and 158 shows that the output image of the neural network shows differences between the two imaging portions that are barely visible in the raw data.

[0169] Further embodiments of the invention are presented in the following numbered clauses:

[0170] 1. A method for obtaining information describing at least one product of a manufacturing process, the method comprising: acquiring a plurality of images, the images having been captured by performing an imaging process on one or more corresponding imaging regions of the at least one product; and determining values ​​of a plurality of numerical parameters using the images and corresponding at least one imaging model characterizing the imaging process, the numerical parameters together defining a product model of a portion of the at least one product, the determination of the values ​​comprising:

[0171] forming a loss function based on the acquired plurality of images, the at least one imaging model, and the numerical parameters of the product model; and

[0172] The values ​​of the numerical parameters of the product model are determined by performing a minimization algorithm on the loss function with respect to the numerical parameters.

[0173] 2. A method according to clause 1, wherein the determining employs a loss value indicative of a difference between the image and a result of applying the corresponding at least one imaging model to the corresponding imaged region of the product model.

[0174] 3. A method according to clause 2, wherein the loss function includes the loss value as a loss term.

[0175] 4. A method according to clause 2, wherein the minimization algorithm is performed under constraints on the loss value.

[0176] 5. A method according to any one of clauses 2 to 4, wherein the loss value is the sum, across the images, of measures of the difference between the image and a corresponding estimated image for each image, the corresponding estimated image being obtained by applying the corresponding imaging model to the corresponding imaging area of ​​the product model.

[0177] 6. A method according to any one of the preceding clauses, wherein the imaging process is performed by an imaging device and is characterized by at least one imaging parameter, and the multiple images include multiple images of a single imaging area of ​​a single product, and the multiple images are captured by the imaging process having different corresponding implementations of the at least one imaging parameter, and the value of the numerical parameter is determined using a different corresponding imaging model for each implementation of the at least one imaging parameter.

[0178] 7. The method of clause 6, wherein the imaging parameters include the at least one imaging parameter selected from the group consisting of:

[0179] a distance between a sensor of the imaging device and the at least one product;

[0180] an orientation of the at least one product relative to an imaging direction in which the image is captured during the imaging process;

[0181] a translational position of the at least one product transverse to an imaging direction in which the image is captured during the imaging process;

[0182] The focal position of the imaging device relative to the product; and

[0183] The exposure time used to capture the image.

[0184] 8. A method according to any of the preceding clauses, wherein the imaging model comprises a convolution model based on an array of values ​​indicating the values ​​of corresponding points on a point spread function.

[0185] 9. A method according to any of the preceding clauses, wherein the model of the at least one product comprises at least one two-dimensional array of values ​​indicating the brightness of corresponding points of the product according to another imaging process.

[0186] 10. A method according to claim 9, when dependent on claim 8, wherein at least one of the two-dimensional arrays of values ​​corresponds to points of the product across an area of ​​the product that is larger than the imaging area, and the imaging model includes applying the convolution model to the two-dimensional array of values ​​to form convolution values, and applying a mask to restrict the convolution values ​​to the imaging area.

[0187] 11. A method according to claim 9 or claim 10, wherein the two-dimensional array of values ​​has a pixel resolution, the pixel resolution includes more intensity values ​​per unit area of ​​the product than the acquired image, and the imaging model includes a resolution reduction model for reducing the pixel resolution of the two-dimensional array of values ​​to the pixel resolution of the acquired image.

[0188] 12. A method according to any of clauses 9 to 11, wherein the loss function comprises at least one regularization term based on the norm of the intensity values.

[0189] 13. The method of clause 12, wherein at least one regularization term comprises a structure term that facilitates forming edges in the two-dimensional array of intensity values.

[0190] 14. A method according to clause 12 or 13, wherein the regularization term is based on transformation values ​​obtained by applying at least one transformation to the two-dimensional array of intensity values.

[0191] 15. A method according to clause 14, wherein the at least one transform comprises a wavelet transform and / or a transform that obtains an image gradient in the two-dimensional array of values.

[0192] 16. A method according to clause 14 or 15, wherein the regularisation term is a function of each transformed value and the sum of the transformed values.

[0193] 17. A method according to any one of clauses 9 to 16, further comprising measuring a difference between at least one pair of points in at least one of the two-dimensional arrays of values ​​to obtain a measured dimension of the at least one product.

[0194] 18. A method according to any of the preceding clauses, wherein the at least one imaging model is defined by the corresponding one or more imaging parameters, the method further comprising obtaining values ​​of the one or more imaging parameters of the at least one imaging model.

[0195] 19. A method according to clause 18, when dependent on clause 8, wherein the one or more imaging parameters comprises an array of values ​​indicative of: values ​​of corresponding points of a point spread function.

[0196] 20. A method according to clause 18 or 19, wherein the one or more imaging parameters comprise control parameters of the imaging process.

[0197] 21. A method according to any of clauses 18 to 20, wherein the minimization algorithm additionally minimizes the loss function with respect to at least some of the imaging parameters to thereby determine the values ​​of at least some of the imaging parameters.

[0198] 22. A method according to any one of clauses 18 to 21, comprising acquiring a reference image captured by the imaging process of a target having a known geometry, and determining at least one of the imaging parameters based on the reference image and the known geometry.

[0199] 23. A method according to clause 22, wherein the target comprises an object having a diameter smaller than the resolution level of the imaging process and located on a uniform background, and at least one of the imaging parameters is derived by processing the reference image as a point spread function of the object.

[0200] 24. A method according to any of the preceding clauses, wherein the loss function comprises a plurality of terms multiplied by corresponding hyperparameters, the method comprising: selecting a value of the hyperparameter by repeatedly executing the method for different corresponding trial values ​​of the hyperparameter; and selecting the value of the hyperparameter from the trial values ​​of the hyperparameter using a selection criterion based on at least one of: (i) the corresponding determined value of the numerical parameter; and (ii) the corresponding value of the term of the loss function after the minimization process.

[0201] 25. A method according to any of the preceding clauses, wherein the numerical parameters of the product model define corresponding geometrical parameters of the at least one product.

[0202] 26. The method of clause 25, wherein the numerical parameters include numerical parameters respectively indicating one of:

[0203] the location of the elongated element in the product;

[0204] The length of the elongated element in the product;

[0205] the width of the elongated element in the product;

[0206] the angle between the length direction of the elongated element in the product and a reference direction; or

[0207] The angle between the respective length directions of two elongated elements in the product.

[0208] 27. A method according to clause 25 or clause 26, wherein the at least one product comprises a plurality of layers spaced apart in a depth direction, and the numerical parameters comprise numerical parameters respectively indicating one of:

[0209] a translational offset transverse to a depth direction of two structures in said at least one product in different respective said layers;

[0210] an angular offset of said depth direction transverse to said respective length directions of two elongated structures in said at least one product in different respective layers; or

[0211] The spacing in the depth direction between two structures in the product.

[0212] 28. A method according to any of the preceding clauses, wherein the image is an optical image obtained by bright field microscopy.

[0213] 29. A method according to clause 28, when dependent on clause 7, wherein the imaging parameters include the frequency of electromagnetic radiation used in the bright field microscopy.

[0214] 30. A method according to any one of clauses 1 to 27, wherein the imaging process is scanning electron microscopy (SEM) and the image is a scanning electron microscopy (SEM) image.

[0215] 31. A method according to any of the preceding clauses, further comprising determining whether at least one abnormality criterion is satisfied with respect to any of the one or more imaging regions based on the determined value of the numerical parameter.

[0216] 32. A method according to clause 31, wherein after determining that the abnormality criterion is met with respect to any of the one or more imaging regions, further measurements are performed on the imaging region that meets the abnormality criterion.

[0217] 33. A method according to clause 31 or 32, wherein determining whether the anomaly criterion is met includes comparing the determined value of the numerical parameter with design data describing the at least one product, and determining that a defect exists in the at least one product based on the comparison.

[0218] 34. A method according to any of the preceding clauses, wherein the image has multiple imaging areas on the at least one product, each imaging area contains a corresponding structure produced by the manufacturing process according to the same design data, and the product model includes a corresponding model part of each of the imaging areas in the imaging area, and each model part is defined by a corresponding subset of the numerical parameters.

[0219] 35. A method according to clause 34, wherein the loss function includes a difference term that penalizes differences between the model parts.

[0220] 36. A method according to clause 35, wherein the difference term is a term that penalizes the rank of the concatenation of the numerical parameters defining the model part.

[0221] 37. A method according to clause 36, wherein the difference term is the nuclear norm of the concatenation of the numerical parameters defining the model part.

[0222] 38. The method according to any of the preceding clauses further comprises registering the images so that corresponding parts of the images are aligned.

[0223] 39. A method according to clause 38, wherein the registration is based on corresponding displacement vectors associated with the images.

[0224] 40. The method of clause 39, wherein the displacement vector is determined during the minimization process.

[0225] 41. A method according to any of the preceding clauses, wherein in the at least one product different ones of the imaging areas overlap, thereby forming a continuous enlarged imaging area.

[0226] 42. A method according to any one of the preceding clauses, wherein the step of forming the loss function is repeated for different corresponding subsets of the imaging area, and the step of determining the value of the numerical parameter of the corresponding model part of the product model is performed for each subset, and the defective imaging area in the imaging area is identified as such imaging area: for the identified imaging area, the value of the loss function after the minimization process is lower for the subset in which the defective imaging area is omitted than for the subset including the defective imaging part.

[0227] 43. A method according to any of the preceding clauses, which is repeatedly executed after the manufacturing process is repeatedly executed, each execution of the method being performed on the product of the most recently executed manufacturing process.

[0228] 44. A method according to clause 43, wherein in each execution of the method, the at least one product includes the product of the most recently executed manufacturing process and the product of the previous execution of the manufacturing process.

[0229] 45. A method according to any of the preceding clauses, wherein the manufacturing process is a wafer manufacturing process and at least one of the products is a manufactured wafer.

[0230] 46. ​​The method of clause 45, wherein the manufacturing process is a photolithography process.

[0231] 47. A computing system comprising a processor and a memory storing program instructions operable when executed by the processor to cause the processor to perform a method according to any of the preceding clauses.

[0232] 48. A computer program product storing program instructions operable when executed by the processor to cause the processor to perform the method according to any one of clauses 1 to 46.

[0233] 49. A method of measuring at least one product of a manufacturing process, the method comprising:

[0234] imaging the at least one product using an imaging system by an imaging process characterized by at least one imaging parameter, wherein an imaging unit of the imaging system captures a plurality of images of at least one imaging region of the at least one product for a plurality of different corresponding realizations of the at least one imaging parameter; and

[0235] The plurality of images of the at least one imaging area are used collectively to obtain a product model of the at least one product.

[0236] 50. The method of clause 49, wherein the imaging parameters are selected from the group consisting of:

[0237] a distance between the sensor and the at least one product;

[0238] an orientation of the at least one product relative to an imaging direction of the imaging process;

[0239] a translational position of the at least one product transverse to an imaging direction of the imaging process; and

[0240] The focal position of the imaging process relative to the product; and

[0241] The frequency of the electromagnetic radiation used in the imaging process.

[0242] 51. A method according to clause 49 or 50, wherein the imaging process is bright field microscopy.

[0243] 52. The method of clause 51, wherein the imaging parameters include the frequency of electromagnetic radiation used in the bright field microscopy.

[0244] 53. The method according to any one of clauses 49 to 52, further comprising a drive system for moving the at least one product and / or for moving the imaging unit of the imaging system to change the imaging parameters.

[0245] 54. An imaging system comprising an imaging unit configured to perform an imaging procedure and a processor configured to control the imaging system to perform the method according to any of clauses 49 to 53.

[0246] 55. A method according to any one of clauses 1 to 46 or 49 to 53, wherein the product model comprises a plurality of reconstructed images having a one-to-one correspondence with a plurality of acquired images, wherein the reconstructed images represent one or more corresponding imaging regions of the at least one product with a higher spatial resolution than a plurality of corresponding acquired images.

[0247] 56. The method of clause 55, further comprising the step of training a neural network model having a plurality of network parameters, the neural network model being configured to receive as input an image of an imaging region of a target product of the manufacturing process and to generate as output a reconstructed image of the imaging region of the target product, the generated output image representing the target product at a higher spatial resolution than the input image, the training step comprising:

[0248] generating a training data set including a plurality of training items, each training item including one of a plurality of acquired images of the first product and a corresponding reconstructed image, and

[0249] The neural network model is trained using the training data set.

[0250] 57. A method according to clause 56, wherein the imaging process for capturing the image of the at least one product is performed by an imaging device and the imaging process is characterized by at least one imaging parameter, and wherein each training item also includes an implementation of the at least one imaging parameter, and the at least one imaging parameter characterizes the imaging process for capturing the corresponding image of the at least one product included in the training item.

[0251] 58. A method according to clause 57, wherein the image of the target product has been captured using the imaging device by performing the imaging process, and the neural network model is configured to receive as input the image of the primary product and an implementation of the at least one imaging parameter, wherein the at least one imaging parameter characterizes the imaging process used to capture the image of the primary product.

[0252] 58. A method according to any one of clauses 56 to 58, wherein training the neural network model using the training data set includes iteratively adjusting the network parameters to reduce the difference between each of the reconstructed images of the training data set and the corresponding output image generated by inputting the corresponding acquired image into the neural network model.

[0253] 60. A method according to any one of clauses 56 to 59, wherein the neural network model comprises at least one of an autoencoder, a variational autoencoder, and a U-Net architecture.

[0254] 61. A method of training a neural network model having a plurality of network parameters and configured to generate a reconstructed image of a target object as a product of a manufacturing process from an input image of the target object, the reconstructed image representing the target object at a higher spatial resolution than the input image, the method comprising:

[0255] (i) Generate a training data set including multiple training items by the following operations:

[0256] acquiring a plurality of training images, the training images having been captured by performing the imaging process on corresponding training products of a manufacturing process other than the target product, and the training images comprising substantially the same spatial resolution as the input image;

[0257] generating a corresponding high-resolution training image for each of the plurality of training images, wherein the corresponding high-resolution training image has a higher spatial resolution than the corresponding training image, and

[0258] forming the plurality of training items, each training item comprising one training image among the plurality of training images and the corresponding high-resolution training image; and

[0259] (ii) using a loss function to train the neural network model using the training data set, wherein the loss function includes a term that represents the difference between the output of the neural network model when receiving one of the training images and the corresponding high-resolution training image.

[0260] 62. The method of clause 61, wherein the target object and the training object have been manufactured according to substantially the same design data.

[0261] 63. A method according to clause 61 or clause 62, wherein the imaging process for capturing training images is performed by an imaging device and is characterized by at least one imaging parameter, and wherein each training item also includes an implementation of at least one imaging parameter, wherein the imaging parameter characterizes the imaging process for capturing the corresponding training image included in the training item.

[0262] 64. A method according to clause 63, wherein the input image has been captured using an imaging device by performing the imaging process, and the neural network model is configured to receive as input the input image and an implementation of at least one of the imaging parameters, wherein the imaging parameters characterize the imaging process used to capture the input image.

[0263] 65. A method according to any one of clauses 61 to 64, wherein the neural network model comprises at least one of an autoencoder, a variational autoencoder, and a U-Net architecture.

[0264] 66. A method according to any one of clauses 61 to 65, wherein computationally generating a corresponding high-resolution training image for each of the plurality of training images comprises performing the method according to clause 55.

[0265] 67. A method according to any one of clauses 61 to 66, wherein computationally generating a corresponding high-resolution training image for each of the plurality of training images comprises applying a deconvolution method to the training image.

[0266] 68. A method according to clause 67, wherein the deconvolution method is an iterative deconvolution method.

[0267] 69. A method of generating a reconstructed image of a target object from an input image of the target object using a trained neural network according to any one of clauses 61 to 68, the method comprising: receiving the input image, and processing the input image using the neural network to generate the reconstructed image.

Claims

1. A method of measuring at least one product of a manufacturing process, the method comprising: imaging the at least one product using an imaging system by an imaging process characterized by at least one imaging parameter, wherein an imaging unit of the imaging system captures a plurality of images of at least one imaging region of the at least one product for a plurality of different corresponding realizations for the at least one imaging parameter; as well as The plurality of images of the at least one imaging area are used collectively to obtain a product model of the at least one product.

2. The method of claim 1, wherein the imaging parameters are selected from the group consisting of: a distance between the sensor and the at least one product; an orientation of the at least one product relative to an imaging direction of the imaging process; a translational position of the at least one product transverse to an imaging direction of the imaging process; and The focal position of the imaging process relative to the product; and The frequency of the electromagnetic radiation used in the imaging process.

3. The method according to claim 1 or 2, wherein the imaging process is bright field microscopy.

4. The method of claim 3, wherein the imaging parameters include the frequency of electromagnetic radiation used in the bright field microscopy.

5. The method according to any one of claims 1 to 4, further comprising a drive system for moving the at least one product and / or for moving the imaging unit of the imaging system to change the imaging parameters.

6. An imaging system comprising an imaging unit configured to perform an imaging process and a processor configured to control the imaging system to perform the method according to any one of claims 1 to 5.

7. A method according to any one of claims 1 to 5, wherein the product model includes a plurality of reconstructed images, and the plurality of reconstructed images have a one-to-one correspondence with the plurality of acquired images, wherein the reconstructed images represent one or more corresponding imaging regions of the at least one product with a higher spatial resolution than the plurality of corresponding acquired images.

8. The method according to claim 7, further comprising the step of training a neural network model having a plurality of network parameters, the neural network model being configured to receive an image of an imaging region of a target product of the manufacturing process as input and to generate a reconstructed image of the imaging region of the target product as output, the generated output image representing the target product at a higher spatial resolution than the input image, the training step comprising: generating a training data set including a plurality of training items, each training item including one of a plurality of acquired images of the first product and a corresponding reconstructed image, and The neural network model is trained using the training data set.

9. A method according to claim 8, wherein the imaging process for capturing the image of the at least one product is performed by an imaging device, and the imaging process is characterized by at least one imaging parameter, and wherein each training item also includes an implementation of the at least one imaging parameter, and the at least one imaging parameter characterizes the imaging process for capturing the corresponding image of the at least one product included in the training item.

10. A method according to claim 9, wherein the image of the target product has been captured using the imaging device by performing the imaging process, and the neural network model is configured to receive as input the image of the primary product and the implementation of the at least one imaging parameter, wherein the at least one imaging parameter characterizes the imaging process used to capture the image of the primary product.

11. The method according to any one of claims 8 to 10, wherein training the neural network model using the training data set comprises iteratively adjusting the network parameters to reduce the difference between each of the reconstructed images of the training data set and a corresponding output image generated by inputting the corresponding acquired image into the neural network model.

12. The method according to any one of claims 8 to 11, wherein the neural network model comprises at least one of an autoencoder, a variational autoencoder, and a U-Net architecture.

13. The method of claim 1 or claim 2, wherein the imaging process is scanning electron microscopy (SEM) and the image is a scanning electron microscopy (SEM) image.

14. A computing system comprising a processor and a memory storing program instructions operable when executed by the processor to cause the processor to perform a method according to any one of claims 1 to 5 or claims 7 to 13.

15. A computer program product storing program instructions which, when executed by a processor, are operable to cause the processor to perform a method according to any one of claims 1 to 5 or any one of claims 7 to 13.

Images