Generation of synthetic semiconductor images

Abstract

Methods, systems, and media are provided for generating composite semiconductor image data. In some embodiments, the method includes generating a set of composite segmented images, where each composite segmented image in the set represents a segmented semiconductor measurement image. The method includes generating a set of virtual images using a first trained GAN, where each virtual image corresponds to one of the composite segmented images in the set of composite segmented images. The method may also include constructing a training set including a plurality of training samples, where each training sample includes a composite segmented image from the set of composite segmented images and a corresponding virtual image from the set of virtual images, where the training set is usable to train a downstream model configured to model a semiconductor manufacturing process.

Description

Incorporation by Reference

[0001] A PCT Request Form is being filed contemporaneously herewith as part of this application. Each application to which this application claims benefit or priority as identified in the contemporaneously filed PCT Request Form is incorporated herein by reference in its entirety for all purposes. [Background technology]

[0002] Machine learning models (neural networks, deep neural networks, etc.) can be useful for modeling semiconductor processes. However, such models can be difficult to acquire and / or generate because they require large data sets to robustly train the models.

[0003] The background art provided herein is intended to provide a general background to the present disclosure, and the work of the inventors named herein, to the extent described in this background art, along with aspects of the description that would not normally be considered prior art at the time of filing, are not admitted expressly or impliedly as prior art to the present disclosure. Summary of the Invention

[0004] A system, method, and medium for generating composite semiconductor image data are provided.

[0005] In some embodiments, a method for generating composite semiconductor image data includes generating a set of composite segmented images, each composite segmented image of the set representing a segmented semiconductor measurement image. The method includes generating a set of virtual images using a first trained GAN, each virtual image corresponding to one of the composite segmented images in the set of composite segmented images. The method includes constructing a training set including a plurality of training samples, each training sample including a composite segmented image from the set of composite segmented images and a corresponding virtual image from the set of virtual images, the training set being usable for training a downstream model configured to model a semiconductor manufacturing process.

[0006] In some examples, the synthetic segmented image set is generated using a second trained GAN. In some examples, the first trained GAN and the second trained GAN are trained simultaneously. In some examples, simultaneously training the first trained GAN and the second trained GAN includes updating weights for a first classifier and / or a first generator associated with the first trained GAN based on a classifier loss and / or a generator loss for a second classifier and / or a second generator associated with the second trained GAN.

[0007] In some examples, generating the set of composite segmented images comprises generating each composite segmented image using a semiconductor process model. In some examples, the semiconductor process model is a model configured to represent a particular semiconductor manufacturing process. In some examples, the particular semiconductor manufacturing process is an etching process, and the semiconductor process model generates images representing an etching profile as a result of the etching process assuming a set of process conditions provided as input to the semiconductor process model. In some examples, the method further comprises applying a texture mapping transformation technique to at least one composite segmented image of the set of composite segmented images generated by the semiconductor process model, wherein the at least one composite segmented image having the texture mapping transformation technique applied is included in the set of composite segmented images.

[0008] In some examples, generating the set of virtual images comprises generating a corresponding grayscale image from a given synthetic segmented image of the set of synthetic segmented images and providing the corresponding grayscale image to the first trained GAN. In some examples, generating the corresponding grayscale image comprises applying a randomly generated color palette to the given synthetic segmented image.

[0009] In some examples, the segmented semiconductor metrology image corresponds to an electron microscopy (EM) image, a scanning electron microscopy (SEM) image, a critical dimension SEM image (CD-SEM), or a transmission electron microscopy (TEM) image.

[0010] In some examples, the downstream model trained with the training set is configured to label interfaces between materials in the input image that have the same type as the segmented semiconductor measurement image.

[0011] In some examples, the downstream model trained with the training set is configured to provide dimensional measurements of one or more wafer features included in an input image having the same type as the segmented semiconductor measurement image.

[0012] In some examples, the downstream model trained with the training set is configured to provide at least one measurement associated with an interface between materials in an input image having the same type as the segmented semiconductor measurement image.

[0013] In some examples, the first trained GAN utilizes multiple sets of classifiers and generators, each set of classifiers and generators corresponding to a different layer of a pyramid structure, each layer configured to utilize images of different sizes. In some examples, the classifiers and generators of each set of the multiple sets of classifiers and generators are trained sequentially.

[0014] According to some embodiments, a computer program product is provided, comprising a non-transitory computer-readable medium having computer-executable instructions for causing a computer system to execute a method for generating composite semiconductor image data. The method includes generating a set of composite segmented images, each composite segmented image of the set representing a segmented semiconductor measurement image. The method includes generating a set of virtual images using a first trained GAN, each virtual image corresponding to one of the composite segmented images in the set of composite segmented images. The method includes constructing a training set including a plurality of training samples, each training sample including a composite segmented image from the set of composite segmented images and a corresponding virtual image from the set of virtual images, the training set being usable for training a downstream model configured to model a semiconductor manufacturing process.

[0015] In some examples, the synthetic segmented image set is generated using a second trained GAN. In some examples, the first trained GAN and the second trained GAN are trained simultaneously. In some examples, simultaneously training the first trained GAN and the second trained GAN includes updating weights for a first classifier and / or a first generator associated with the first trained GAN based on a classifier loss and / or a generator loss for a second classifier and / or a second generator associated with the second trained GAN.

[0016] In some examples, generating the set of composite segmented images comprises generating each composite segmented image using a semiconductor process model. In some examples, the semiconductor process model is a model configured to represent a particular semiconductor manufacturing process. In some examples, the particular semiconductor manufacturing process is an etching process, and the semiconductor process model generates images representing an etching profile as a result of the etching process assuming a set of process conditions provided as input to the semiconductor process model. In some examples, the method further comprises applying a texture mapping transformation technique to at least one composite segmented image of the set of composite segmented images generated by the semiconductor process model, wherein the at least one composite segmented image having the texture mapping transformation technique applied is included in the set of composite segmented images.

[0017] In some examples, generating the set of virtual images comprises generating a corresponding grayscale image from a given synthetic segmented image of the set of synthetic segmented images and providing the corresponding grayscale image to the first trained GAN. In some examples, generating the corresponding grayscale image comprises applying a randomly generated color palette to the given synthetic segmented image.

[0018] In some examples, the segmented semiconductor metrology image corresponds to an electron microscopy (EM) image, a scanning electron microscopy (SEM) image, a critical dimension SEM image (CD-SEM), or a transmission electron microscopy (TEM) image.

[0019] In some examples, the downstream model trained with the training set is configured to label interfaces between materials in the input image that have the same type as the segmented semiconductor measurement image.

[0020] In some examples, the downstream model trained with the training set is configured to provide dimensional measurements of one or more wafer features included in an input image having the same type as the segmented semiconductor measurement image.

[0021] In some examples, the downstream model trained with the training set is configured to provide at least one measurement associated with an interface between materials in an input image having the same type as the segmented semiconductor measurement image.

[0022] In some examples, the first trained GAN utilizes multiple sets of classifiers and generators, each set of classifiers and generators corresponding to a different layer of a pyramid structure, each layer configured to utilize images of different sizes. In some examples, the classifiers and generators of each set of the multiple sets of classifiers and generators are trained sequentially. [Brief explanation of the drawings]

[0023] [Figure 1] FIG. 1 illustrates an example of a system for generating a composite semiconductor image, according to some embodiments.

[0024] [Figure 2] FIG. 1 illustrates a generative adversarial network (GAN) that may be utilized within a system for synthetic semiconductor image generation, according to some embodiments.

[0025] [Figure 3] FIG. 1 illustrates an example of a GAN with a pyramidal structure, in accordance with some embodiments.

[0026] [Figure 4] 1 is a flowchart illustrating an example of a process for generating a training set of synthetic semiconductor images, according to some embodiments.

[0027] [Figure 5] 1 is a flowchart illustrating an example of a process for generating a virtual semiconductor image according to some embodiments.

[0028] [Figure 6] 1 is a flowchart illustrating an example of a process for generating a synthetic segmented image using a semiconductor process model, according to some embodiments.

[0029] [Figure 7] FIG. 1 illustrates an example of a computer system that can be used to implement certain embodiments described herein. DETAILED DESCRIPTION OF THE INVENTION

[0030] In the following description, numerous specific details are set forth to provide a thorough understanding of the presented embodiments. The disclosed embodiments may be practiced without some or all of these specific details. Additionally, detailed descriptions of well-known process operations are omitted to avoid unnecessarily obscuring the disclosed embodiments. While the disclosed embodiments are described in connection with specific embodiments, it should be understood that they are not intended to limit the disclosed embodiments.

[0031] Systems, methods, and media for generating composite semiconductor image data are disclosed herein. In some embodiments, a training set may be generated, where the training set includes a plurality (e.g., hundreds or thousands) of training samples. The training samples may include synthetically generated content that is a synthetically generated version of an image or image information acquired using a given semiconductor metrology technique. Examples of semiconductor metrology techniques include electron microscopy (EM), scanning electron microscopy (SEM), critical dimension SEM (CD-SEM), transmission electron microscopy (TEM), etc. By way of example, the training samples may include virtual images that are synthetically generated versions of images that may be acquired using the semiconductor metrology technique. In some embodiments, the training samples may include a virtual image (e.g., a synthetically generated version of an image that may be acquired using the given metrology technique) and a corresponding segmented image. The segmented image may include an indication of clusters of pixels assigned to the same classification or category, where each classification or category represents a particular wafer feature, material, etc.

[0032] By generating a training set that includes synthetically generated training samples, a training set of hundreds or thousands of training samples can be generated without the time- and resource-intensive measurement methods of acquiring the training samples. The training set can be used to train one or more downstream models that are used, for example, to provide measurements related to interfaces between materials, to provide measurements of one or more wafer features, etc. Because a training set of synthetically generated training samples can be substantially larger than a training set limited to samples acquired using physical measurement techniques (e.g., EM, SEM, CD-SEM, TEM, etc.), downstream models trained using a training set of synthetically generated training samples can be trained more quickly (e.g., by reaching stopping criteria more quickly) and can be more robust (e.g., because a wider data distribution range is covered within the synthetically generated data set). Furthermore, because downstream models trained using a training set of synthetically generated samples are more robust, semiconductor manufacturing processes that rely on the downstream models can be enabled to operate more efficiently, with higher throughput, reduced wafer waste, etc.

[0033] FIG. 1 illustrates an example system for generating a composite semiconductor image. As shown, a training sample pair 102 may be generated. The training sample pair 102 may include a synthetic segmented image 104 and a virtual image 106. As discussed above, a "segmented image," as used herein, refers to an image in which each pixel is assigned to a class or category that indicates, for example, the type of material (e.g., interface, fin, etc.) with which the object corresponding to that pixel is associated. In other words, a "segmented image" may refer to a cluster of pixels that together form an object or element of the image. Conventionally, a composite image may be generated using a real image (e.g., a photograph). In contrast, a "synthetic segmented image," as used herein, refers to a segmented image generated without any such real image. A "virtual image," as used herein, is a synthetically generated image that appears to have been captured using a particular imaging technology, rather than a real image acquired using the imaging technology. As described below, a virtual image may be generated using a synthetic segmented image, such that the virtual image and the synthetic segmented image used to generate the virtual image form a pair. It should be understood that the synthetically generated image is constructed to include similar features, textures, etc. of a semiconductor image acquired using a given measurement technique (e.g., SEM, TEM, CD-SEM, etc.), and therefore both synthetically generated images include a wide range of potential features, textures, etc. relevant to the semiconductor field.

[0034] While FIG. 1 illustrates the generation of a single training sample pair (including one synthetic segmented image (104) and one virtual image (106)), in some embodiments, a complete training set may be generated, where the complete training set includes multiple training sample pairs (e.g., 100 training sample pairs, 1000 training sample pairs, or any other suitable number N).

[0035] The synthetic segmented image 104 may be generated by a synthetic segmented image generation system 108. In some embodiments, the synthetic segmented image generation system 108 may comprise a generative adversarial network (GAN). Note that the architecture of a GAN and training of a GAN are described in more detail below with reference to FIGS. 2 and 3. In some such embodiments, the GAN may be trained using one or more segmented images generated using real images acquired using semiconductor imaging techniques. Examples of imaging techniques include electron microscopy (EM), scanning electron microscopy (SEM), critical dimension SEM (CD-SEM), transmission electron microscopy (TEM), etc. The one or more segmented images used to train the GAN may be generated using a machine learning model (e.g., a convolutional neural network, etc.) trained to generate segmented images using real images as input. The GAN may receive input from a random seed or random noise image, which may be used to control feature dimensions and variability of features in the synthetic segmented image generated by the GAN.

[0036] Alternatively, in some embodiments, the composite segmentation image generation system 108 may include a semiconductor process model. Such a semiconductor process model may take as input parameter values ​​corresponding to parameters of a particular manufacturing process being modeled (e.g., an etch process, a deposition process, etc.). The semiconductor process model may generate as output a composite segmentation image representing the resulting segmentation image when the manufacturing process is modeled given the input process parameter values. Note that, in some embodiments, various parametric techniques may be applied to the composite segmentation image to account for feature variations, as described below in connection with FIG. 5 .

[0037] The composite segmented image 104 generated by the composite segmented image generation system 108 may be provided to a grayscale image conversion block 110. The grayscale image conversion block 110 may generate a grayscale image corresponding to the composite segmented image 104. A more detailed technique describing the generation of a grayscale image from the composite segmented image is described below in conjunction with FIG.

[0038] The grayscale image generated by the grayscale image conversion block 110 may be provided to a virtual image generation system 112. The virtual image generation system 112 may be configured to take the grayscale version of the composite segmented image generation system 108 as input and generate a virtual image (e.g., virtual image 106) corresponding to the composite segmented image as output.

[0039] It should be understood that the training sample pairs, including synthetic segmentation images and corresponding virtual images, may be used to train any number of different downstream models. In some embodiments, each downstream model may be a single segmentation model used in connection with a particular dataset and / or a particular application. In some examples, using a given trained segmentation model, an image mask may be inferred, and multiple measurements N may be derived from the image mask. For example, a downstream model may be trained to label different materials in input images acquired using the same imaging technology as the virtual images in the training set. As another example, a downstream model may be trained to provide dimensional measurements of one or more wafer features in input images acquired using the same imaging technology as the virtual images in the training set using image masks derived from a downstream model that is itself a trained segmentation model trained using a synthetically generated training set. As yet another example, a downstream model may be trained to provide at least one measurement associated with an interface between materials in input images acquired using the same imaging technology as the virtual images in the training set. It should be noted that the downstream model may be any suitable type of model, such as a neural network, a deep neural network, or a convolutional neural network. By generating a training set that includes synthetic training samples, arbitrarily large training sets may be generated, which may allow for more robust training of downstream models. Furthermore, by using synthetic training data, resource-intensive imaging techniques are not required to generate each training sample, which may allow for more rapid training of downstream models.

[0040] As described above in connection with FIG. 1 , synthetic segmented images and / or virtual images may be generated using a GAN. Generally, a GAN is a type of neural network that can be used for unsupervised learning. A GAN may include two neural network models: a discriminator network and a generator network. The generator network may be trained to generate fake samples, while the discriminator network may be trained to distinguish between fake and real samples. Thus, by training both the generator network and the discriminator network of a GAN, the GAN may be trained to generate realistic fake (also referred to as "synthetic") samples. While a GAN may be trained by training both the generator network and the discriminator network at inference time, it should be noted that the GAN may also generate realistic synthetic data using the output of only the generator network. In other words, the discriminator network may be used alone during training, while the trained generator network is used at inference time.

[0041] FIG. 2 illustrates the architecture and training of an example GAN. As shown, a generator network 202 may receive a random input 204 as an input. The generator network 202 may generate a synthetic (i.e., fake) sample 206 as an output. Referring now to a classifier network 212, a real sample 210 may be selected from a set of real samples (e.g., real images 208). The classifier network 212 may be configured to receive the real sample 210 and the synthetic sample 206 as inputs and generate an output indicating which of the two inputs is the real sample. An error associated with the output of the classifier network 212 may be used to determine a classifier loss 214 and a generator loss 216. For example, the generator loss 216 may be relatively high if the classifier network 212 correctly identifies the real sample 210 as a real sample, effectively penalizing the generator network 202 for not generating sufficiently realistic synthetic samples. In contrast, the classifier loss 216 may be relatively high if the classifier network 212 inaccurately classifies real samples, effectively penalizing the classifier network 212 for not accurately distinguishing between real and synthetic inputs. By simultaneously updating the generator network 202 and the classifier network 212, the generator network 202 may be trained to generate highly realistic synthetic samples. Note that once training of the GAN is complete, the trained GAN may comprise a trained generator network that is now configured (e.g., after training) to generate realistic synthetic samples.

[0042] In some examples, a GAN (e.g., a GAN used to generate a synthetic segmented image and / or a GAN used to generate a virtual image from the synthetic segmented image) may have a pyramidal structure including a set of layers. Using such a pyramidal structure, the GAN may be trained using as few as one real image. Each layer may include a generator network and a classifier network. Each layer may operate on patches of the real image used to train the GAN, such that lower-level layers operate on patches having lower (e.g., coarser) resolution and higher-level layers operate on patches having higher (e.g., finer) resolution. Thus, by training all layers of the GAN, the GAN as a whole may learn to synthetically generate both coarse and fine features in the synthetic image. In some embodiments, for a given layer of the GAN, the generator network may receive both a random seed image (e.g., noise) having a size corresponding to the patch on which that layer of the GAN operates and a synthetically generated image generated by the generator network of a preceding (e.g., lower-level) GAN. For a given layer, the discriminator network may receive synthetically generated images of the generator network from the same layer. In some embodiments, the generator network and the discriminator network may be trained consecutively for different layers for a given image, such that a lower-level layer network (corresponding to a lower or coarser resolution) is trained before a higher-level layer network (corresponding to a higher or finer resolution). In particular, since a higher-level layer network may depend on the output of the generator network of a preceding lower-level network, a higher-level layer network may be trained after and / or based on a lower-level layer network for a given image. Furthermore, it should be noted that at each layer, the generator network may receive as input a noise image having the same size as the image generated by the generator network at that layer.In layers that are not at the lowest level of resolution (e.g., smallest image size), the generator network may receive two inputs corresponding to a noise image and a synthetic image generated by the generator network of the previous layer. One example of a GAN with a pyramidal structure is a SinGAN network, although it should be noted that other types of GANs with pyramidal structures may also be used.

[0043] 3 illustrates an example of a GAN with a pyramidal structure, according to some embodiments. As shown, each layer from 0 to N includes a generator network and a discriminator network. In particular, for a given layer i, the corresponding generator network is G i and the corresponding classifier network is D i Layer 0 represents the layer with the highest (e.g., finest) resolution, and Layer N represents the layer with the lowest (e.g., coarsest) resolution.

[0044] Such a pyramidal structure may be used to train the generator network and the classifier network using as few as one training image. Such training may be performed by sampling various patches of a single training image, with different layers of the pyramidal structure operating on patches having different sizes. FIG. 3 illustrates effective patch sizes 302a, 302b, and 302c, each corresponding to a different layer of the pyramidal structure. As shown, smaller patch sizes correspond to higher (e.g., finer) resolution and are associated with larger-sized images received as input by the generator network and corresponding larger composite images generated by the generator network at that layer. Note that the number of layers N may be selected to optimally learn the granular features of the image. In some embodiments, N may be three layers, four layers, five layers, etc. As an example, for N=4, image sizes of 1024x1024 pixels for layer 0, 512x512 pixels for layer 1, 256x256 pixels for layer 2, and 128x128 pixels for layer 4 may be used.

[0045] It should be noted that while a GAN may be trained using as few as one image in certain GANs having a pyramidal structure (such as that shown and described above in connection with FIG. 3), in some embodiments, the GAN may be trained with multiple (e.g., 2, 5, 10, 20, 100, etc.) images. Furthermore, when synthetic segmented images are generated using a GAN (e.g., rather than using a semiconductor processing model or other technique to generate the synthetic segmented images), the synthetic segmented image GAN and the virtual image GAN (which generates virtual images based on the corresponding synthetic segmented images) may be trained simultaneously. By training both GANs simultaneously, rather than first training the synthetic segmented image GAN and then subsequently training the virtual image GAN, significant time can be saved by training each GAN in parallel.

[0046] As described above in connection with FIG. 1 , in some embodiments, a training set may be constructed that can be utilized to train a downstream model. The training set may be constructed by generating a set of training samples, each training sample including a pair of images. The pair of images may include a synthetic segmented image and a corresponding synthetically generated virtual image. The synthetically generated virtual image may be a synthetic image that appears to be an image acquired using a semiconductor metrology technique (e.g., EM, SEM, CD-SEM, TEM, etc.). The synthetic segmented image may be generated using a GAN (e.g., using a trained generator network) or a semiconductor processing model. The virtual image may be generated based on the synthetic segmented image (e.g., as described in more detail below in connection with FIG. 5 ). For example, the virtual image may be generated by providing the synthetic segmented image as an input to a GAN (e.g., the GAN shown and described above in connection with FIG. 2 and / or FIG. 3 ), specifically as an input to a trained generator network of the GAN.

[0047] 4 is a flowchart illustrating an example process 400 for generating a training set, according to some embodiments. In some implementations, the blocks of process 400 may be performed by a server device. In some embodiments, the blocks of process 400 may be performed in an order other than the order shown in FIG. 4. In some embodiments, two or more blocks of process 400 may be performed substantially in parallel. In some embodiments, one or more blocks of process 400 may be omitted.

[0048] At step 402, process 400 may begin by generating a set of composite segmented images, each representing a segmented semiconductor metrology image. Examples of semiconductor metrology types include EM, SEM, CD-SEM, TEM, etc. The composite segmented images may include indications of clusters of pixels assigned to the same category (e.g., material type, wafer feature, etc.). In some embodiments, the composite segmented images may be generated by a GAN (e.g., by a trained generator network). In some examples, the GAN may have a pyramidal structure as shown and described above in connection with FIG. 3. Alternatively, in some embodiments, the composite segmented images may be generated using a semiconductor process model, as shown and described in more detail in connection with FIG. 6.

[0049] At step 404, process 400 can use the GAN (e.g., using a trained generator network) to generate a set of virtual images, each corresponding to one synthetic segmented image in the set of synthetic segmented images. In other words, for a given pair of synthetic segmented image and corresponding virtual image, the virtual image can be a synthetic image having textures and features associated with a given semiconductor acquisition technique (e.g., EM, SEM, CD-SEM, TEM, etc.). Continuing with this example, the corresponding synthetic segmented image can be a segmented version of the virtual image, e.g., showing clusters of pixels assigned to the same category (e.g., wafer feature category, material category, etc.).

[0050] In some embodiments, each virtual image may be generated by generating a grayscale version of the segmented image. The grayscale version of the segmented image may then be provided to a trained GAN (e.g., a trained generator network), which is configured to generate the virtual image as an output. A more detailed technique for generating the virtual images is shown and described below in connection with FIG. 5.

[0051] At step 406, process 400 may construct a training set including a set of training samples, each training sample including a synthetic segmented image and a corresponding virtual image, which can be used to train a downstream model configured to model a semiconductor manufacturing process. Note that the training set may also include original images (e.g., original images used to generate the synthetic images). For example, as described above in connection with FIG. 1 , the training set may be constructed by collecting multiple training samples, each including a synthetic segmented image and a corresponding virtual image. The training set may include hundreds or thousands of training samples. Note that in some embodiments, the training set may include a combination of real images (e.g., acquired using a semiconductor metrology technique) and segmented images constructed based on the real images, and synthetic training samples (e.g., a synthetic segmented image and a corresponding virtual image). Note that in some embodiments, all of the images in the training set may be associated with the same semiconductor metrology technique (e.g., EM, SEM, CD-SEM, TEM, etc.).

[0052] As described above in connection with block 404 of FIG. 4 , a virtual image may be generated based on the synthetic segmented image using a GAN. In particular, a grayscale version of the synthetic segmented image may be generated. The grayscale image may be generated by applying a color palette to the synthetic segmented image. The color palette may be a randomly generated color palette. In some embodiments, a parametric deformation technique may be applied to the synthetic segmented image before generating the grayscale version of the synthetic segmented image. The parametric deformation technique may introduce variations (such as variations across a set of wafer features) into the synthetic segmented image. By applying the parametric deformation technique to the synthetic segmented image, the training set may include a more representative set of wafer-to-wafer variations, thereby enabling more robust training of downstream models. Note that in some embodiments, the parametric deformation technique may be applied when a semiconductor processing model is used to generate the synthetic segmented image. Examples of parametric deformation techniques that can be utilized include texture mapping and image deformation.

[0053] 5 is a flowchart illustrating an example process 500 for generating a virtual image based on a composite segmented image, according to some embodiments. In some implementations, the blocks of process 500 may be performed by a server device. In some embodiments, the blocks of process 500 may be performed in an order other than the order shown in FIG. 5. In some embodiments, two or more blocks of process 500 may be performed substantially in parallel. In some embodiments, one or more blocks of process 500 may be omitted.

[0054] Process 500 may begin in step 502 by obtaining a synthetic segmentation image. Similar to that described above in connection with block 402 of FIG. 4, the synthetic segmentation image may include an indication of clusters of pixels assigned to the same category (e.g., material type, wafer feature, etc.). In some embodiments, the synthetic segmentation image may be generated by a GAN (e.g., by a trained generator network). In some examples, the GAN may have a pyramidal structure as shown and described above in connection with FIG. 3. Alternatively, in some embodiments, the synthetic segmentation image may be generated using a semiconductor processing model, as shown and described in more detail in connection with FIG. 6.

[0055] In some embodiments, at step 504, process 500 can apply parametric deformation techniques to the composite segmented image to account for variations in geometric features. Such variations may include variations among features of various wafer dimensions, variations in angle or tilt of features, etc. Examples of parametric deformation techniques include shape deformation. Examples of shape deformation include texture mapping and image deformation. In some embodiments, the parametric deformation techniques may be applied to the composite segmented image generated using a semiconductor processing model. In some implementations, block 504 may be omitted.

[0056] At step 506, process 500 may generate a grayscale image of the composite segmented image, which is provided to a trained GAN (e.g., a trained generator network) to generate a corresponding virtual image. In some embodiments, the grayscale image may be generated by applying a color palette (e.g., a random color palette) to the composite segmented image. Note that if a parametric deformation technique is applied at block 504, the grayscale image may be generated using the composite segmented image after application of the deformation technique. As described above in connection with FIGS. 1 and 4, the grayscale image may be provided as an input to a trained GAN (e.g., a trained generator network).

[0057] As described above in connection with FIGS. 4 and 5 , in some examples, the composite segmentation image may be generated using a semiconductor manufacturing process model. Such a process model may simulate physical and / or chemical processes occurring during a manufacturing process (e.g., an etching process, a deposition process, etc.). The process model may be a behavioral model and / or a model of surface reactions (e.g., a surface dynamics model). In some embodiments, the process model may take as input process parameter values ​​that represent knobs associated with the semiconductor process being simulated. The semiconductor process model may then generate as output a composite segmentation image that represents the results of a processed wafer from the manufacturing process using the given process parameter values.

[0058] FIG. 6 is a flow chart illustrating an example process 600 for generating a synthetic segmented image using a semiconductor process model, according to some embodiments. In some examples, the blocks of process 600 may be performed on a server device. In such examples, the server device may be the same or different from the server device used to perform the blocks of processes 400 and / or 500. In some embodiments, the blocks of process 600 may be performed in an order other than the order shown in FIG. 6. In some examples, two or more blocks of process 600 may be performed substantially in parallel. In some examples, one or more blocks of process 600 may be omitted.

[0059] In step 602, process 600 may begin by obtaining one or more process parameters associated with the manufacturing process to be simulated. The process parameters may include parameters associated with the process temperature, parameters associated with the process gas (e.g., gas species, gas mixture, gas flow rate, gas partial pressure, etc.), parameters associated with the plasma utilized (e.g., plasma power, plasma species, etc.), and / or any other suitable process parameters. The process parameters may be obtained, for example, using a user interface.

[0060] In step 604, the process 600 can provide one or more process parameters to a manufacturing process simulation model, where the model is configured to generate a composite segmented image as an output.

[0061] It should be noted that in some embodiments, process 600 may be performed to generate a relatively small number of synthetic segmented images (e.g., one, two, three, etc.). The small number of synthetic segmented images may then be manipulated to generate a larger set of synthetic segmented images, for example, by performing one or more parametric deformation techniques to introduce plausible feature variations. The larger set of synthetic segmented images may then be used to generate corresponding virtual images. In this manner, a relatively computationally intensive process simulation model (e.g., as used in process 600) may be run relatively few times and used to generate a much larger synthetic training set, which may then be used to train one or more downstream models.

[0062] Background of the Disclosed Computer Embodiments Systems including the manufacturing tools described herein may include logic for automated control of components.

[0063] The analysis logic may be designed and implemented in any of a variety of ways. For example, the logic can be implemented in hardware and / or software. Examples are provided in the Controller section of this specification. Hardware-implemented control logic may be provided in any of a variety of forms, such as hard-coded logic in digital signal processors, application-specific integrated circuits, and other devices with algorithms implemented as hardware. The analysis logic may also be implemented as software or firmware instructions configured to run on a general-purpose processor. System control software may be provided by "programming" in a computer-readable programming language.

[0064] The computer program code for controlling the processing within the processing steps may be written in any conventional computer-readable programming language, such as assembly language, C, C++, Pascal, Fortran, Python, etc. Compiled object code or script is executed by a processor to perform the tasks specified in the program. Alternatively, as noted above, the program code may be hard-coded.

[0065] The integrated circuits used in the logic may include chips in the form of firmware that store program instructions, digital signal processors (DSPs), chips defined as application specific integrated circuits (ASICs), and / or one or more microprocessors or microcontrollers that execute program instructions (e.g., software). The program instructions may be instructions communicated in the form of various individual settings (or program files) that define operating parameters for performing a particular analysis or image analysis application.

[0066] 7 is a block diagram illustrating an example computing device 700 suitable for use in implementing some embodiments of the present disclosure. For example, device 700 may be suitable for performing some or all of the functionality for generating a composite segmented image and / or generating a composite image as described herein.

[0067] The computing device 700 may include a bus 702 that directly or indirectly connects with memory 704, one or more central processing units (CPUs) 706, one or more graphics processing units (GPUs) 708, a communications interface 710, input / output (I / O) ports 712, input / output components 714, a power supply 716, and one or more presentation components 718 (e.g., displays). In addition to the CPU 706 and GPU 708, the computing device 700 may include additional logic devices not shown in FIG. 7, including, but not limited to, an image signal processor (ISP), a digital signal processor (DSP), an ASIC, an FPGA, etc.

[0068] While the various blocks in FIG. 7 are illustrated as being connected via bus 702 via wires, this is not intended to be limiting and is for clarity only. For example, in some embodiments, a presentation component 718 (e.g., a display device) may be considered an I / O component 714 (e.g., if the display is a touchscreen). As another example, the CPU 706 and / or the GPU 708 may include memory (e.g., the memory 704 may be representative of a storage device in addition to the memory of the GPU 708, the CPU 706, and / or other components). In other words, the computing devices in FIG. 7 are merely exemplary. No distinction is made between categories such as “workstation,” “server,” “laptop,” “desktop,” “tablet,” “client device,” “mobile device,” “handheld device,” “electronic control unit (ECU),” “virtual reality system,” and / or other device or system types, as all are considered to be within the scope of the computing devices in FIG. 7.

[0069] Bus 702 may represent one or more buses, such as an address bus, a data bus, a control bus, or a combination thereof. Bus 702 may include one or more bus types, such as an Industry Standard Architecture (ISA) bus, an Extended Industry Standard Architecture (EISA) bus, a Video Electronics Standards Association (VESA) bus, a Peripheral Component Interconnect (PCI) bus, a Peripheral Component Interconnect Express (PCIe) bus, and / or another type of bus.

[0070] Memory 704 may include any of a variety of computer-readable media. Computer-readable media may be any available media that can be accessed by computing device 700. Computer-readable media may include both volatile and nonvolatile media, and both removable and non-removable media. By way of example, and not limitation, computer-readable media may include computer storage media and / or communication media.

[0071] Computer storage media may include volatile and nonvolatile and / or removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules, and / or other data types. For example, memory 704 may store computer-readable instructions (e.g., representing programs and / or program elements, such as an operating system). Computer storage media may include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVDs) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage, or other magnetic storage devices, or any other medium available to store the desired information and accessible by computer device 600. As used herein, computer storage media does not include the signals themselves.

[0072] Communication media may embodi computer-readable instructions, data structures, program modules, and / or other data types in a modulated data signal (such as a carrier wave or other transport mechanism) and include any information delivery media. The term "modulated data signal" may mean a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media may include wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared and other wireless media. Combinations of any of the above are also included within the scope of computer-readable media.

[0073] The CPU 706 may be configured to execute computer-readable instructions to control one or more components of the computing device 700 to perform one or more of the methods and / or processes described herein. The CPUs 706 may each include one or more cores (e.g., 1, 2, 4, 8, 28, 72, etc.) capable of simultaneously operating multiple software threads. The CPUs 706 may include any type of processor, and may include different types of processors depending on the type of computing device 700 implemented (e.g., a processor with fewer cores for a mobile device and a processor with more cores for a server). For example, depending on the type of computing device 700, the processor may be an ARM processor implemented using reduced instruction set computing (RISC) or an x86 processor implemented using complex instruction set computing (CISC). The computing device 700 may include one or more CPUs 706 in addition to one or more microprocessors or auxiliary coprocessors (e.g., math coprocessors).

[0074] The GPU 708 may be used by the computing device 700 to render graphics (e.g., 3D graphics). The GPU 708 may include many (e.g., tens, hundreds, or thousands) of cores capable of operating many software threads simultaneously. The GPU 708 may generate pixel data for an output image in response to rendering commands (e.g., rendering commands from the CPU 706 received via a host interface). The GPU 708 may include graphics memory (e.g., display memory) for storing pixel data. The display memory may be included as part of the memory 704. The GPU 708 may include two or more GPUs operating in parallel (e.g., via a link). When combined, each GPU 708 can generate pixel data for a different portion of the output image or for different output images (e.g., a first GPU for a first image and a second GPU for a second image). Each GPU may include its own memory or may share memory with other GPUs.

[0075] In examples where the computing device 700 does not include a GPU 708, the CPU 706 may be used to render graphics.

[0076] The communications interface 710 may include one or more receivers, transmitters, and / or transceivers that enable the computing device 700 to communicate with other computing devices over electronic communications networks, including wired and / or wireless communications. The communications interface 710 may include components and functionality to enable communication over any of a number of different networks, such as a wireless network (e.g., Wi-Fi, Z-Wave, Bluetooth, Bluetooth LE, ZigBee, etc.), a wired network (e.g., communication over Ethernet), a low-power wide area network (e.g., LoRaWAN, SigFox, etc.), and / or the Internet.

[0077] The I / O ports 712 may enable computing device 700 to be logically connected to other devices, such as I / O components 714, presentation components 718, and / or other components, some of which may be built-in (e.g., integrated) within computing device 700. Examples of I / O components 714 include a microphone, mouse, keyboard, joystick, trackpad, satellite dish, scanner, printer, wireless device, etc. The I / O components 714 may provide a natural user interface (NUI) that processes air gestures, voice, or other physiological input produced by a user. In some examples, the input may be transmitted to an appropriate network element for further processing. The NUI may implement any combination of voice recognition, stylus recognition, facial recognition, biometric recognition, both on-screen and adjacent-to-screen gesture recognition, head and eye tracking, and touch recognition associated with the display of computing device 700 (discussed in more detail below). The computing device 700 may include a depth camera (such as a stereoscopic camera system), an infrared camera system, an RGB camera system, touchscreen technology, and combinations thereof for gesture detection and recognition. Additionally, the computing device 700 may include an accelerometer or gyroscope (e.g., as part of an inertial measurement unit (IMU)) to enable detection of movement. In some examples, the output of the accelerometer or gyroscope may be used by the computing device 700 to provide immersive augmented or virtual reality.

[0078] The power supply 716 may include a hardwired power supply, a battery power supply, or a combination thereof. The power supply 716 may provide power to the computing device 700 to enable operation of the components of the computing device 700.

[0079] The presentation component 718 may include a display (e.g., a monitor, a touch screen, a television screen, a heads-up display (HUD), other display types, or a combination thereof), speakers, and / or other presentation components. The presentation component 718 may receive data from other components (e.g., the GPU 708, the CPU 706, etc.) and output data (e.g., as images, video, audio, etc.).

[0080] The present disclosure may be described in the general context of computer code or machine-usable instructions (e.g., computer-executable instructions, such as program modules) being executed by a computer or other machine (e.g., a personal data assistant or other handheld device). Generally, program modules, including routines, programs, objects, components, data structures, etc., refer to code that performs particular tasks or implements particular abstract data types. The present disclosure may be implemented in a variety of system configurations, including handheld devices, consumer electronics, general-purpose computers, more specialized computing devices, etc. The present disclosure may also be implemented in distributed computing environments where tasks are performed by remote processing devices linked through a network.

[0081] Further considerations As used in this specification and the appended claims, the singular forms "a," "an," and "the" include plural referents unless content and context dictate otherwise. For example, reference to "a cell" includes a combination of two or more such "cells." Unless otherwise indicated, the conjunction "or" is used in its proper sense as a Boolean logic operator, encompassing both feature selections in an alternative (A or B, where selection of A is mutually exclusive with B) and feature selections in a conjunction (A or B, where both A and B are selected).

[0082] It should be understood that, when used herein, phrases such as "for each <item> of one or more <items>," "for each <item> of one or more <items>," and the like, include both single-item groups and multiple-item groups; i.e., the phrase "for each" is used in the sense used in programming languages ​​to refer to each item in any collection of items. For example, if the collection of items referred to is a single item, then "each" refers only to that single item and does not imply that there must be at least two of those items (even though dictionary definitions of "each" often define the term to refer to "every one of two or more"). Similarly, the terms "set" or "subset" should not, in and of themselves, be considered to necessarily include multiple items; it is understood that a set or subset may include only one member or multiple members (unless the context suggests otherwise).

[0083] The use of ordinal numbers (e.g., (a), (b), (c), etc., or the like) in this disclosure and in the claims should not be understood to indicate any particular order or sequence unless such order or sequence is explicitly indicated. For example, if there are three steps labeled (i), (ii), and (iii), these steps may be performed in any order (or simultaneously, if not otherwise appropriate) unless otherwise indicated. For example, if step (ii) involves manipulation of an element produced in step (i), step (ii) may be considered to be performed some time after step (i). Similarly, if step (i) involves manipulation of an element produced in step (ii), it should be understood that the opposite is true. It should be understood that the use of the ordinal number marker "first" (e.g., "first item") herein should not be read as implying, implicitly or inherently, that a "second" thing (e.g., "second item") is necessarily present.

[0084] Various computer elements, such as processors, memories, instructions, routines, models, or other components, may be described or claimed as being "configured" to perform a task. In this context, the term "configured" is used to include structure by indicating that a component includes structure (e.g., stored instructions, circuits, etc.) that performs one or more tasks during operation. Thus, a unit / circuit / component may be referred to as being configured to perform a task even if the particular component is not necessarily currently operating (e.g., not on).

[0085] A component used in conjunction with the phrase "configured to" may refer to hardware (e.g., a circuit, a memory storing executable program instructions to perform an operation, etc.). Additionally, "configured to" may refer to a general-purpose structure (e.g., a general-purpose circuit) operated by software and / or firmware (e.g., an FPGA or general-purpose processor running software) operating to perform the tasks described above. Additionally, "configured to" may refer to one or more memories or memory elements storing computer-executable instructions for performing the tasks described above. Such memory elements may include memory on a computer chip with processing logic. In some contexts, "configured to" may also include adapting a manufacturing process (e.g., a semiconductor manufacturing facility) to produce a device (e.g., an integrated circuit) adapted to perform or execute one or more tasks.

[0086] Although the present embodiments have been described in some detail for purposes of clarity, it will be apparent that certain changes and modifications may be made within the scope of the appended claims. It should be noted that there are many other ways of implementing the processes, systems, and apparatus of the present embodiments. Therefore, the present embodiments are to be considered as illustrative and not restrictive, and the embodiments are not limited to the details set forth herein.

Claims

1. 1. A method for generating composite semiconductor image data, comprising: generating a set of composite segmented images, each composite segmented image of the set representing a segmented semiconductor measurement image; generating a set of virtual images using the first trained GAN, each virtual image corresponding to one of the synthetic segmented images in the set of synthetic segmented images; 1. A method comprising: constructing a training set including a plurality of training samples, each training sample including a synthetic segmented image from the set of synthetic segmented images and a corresponding virtual image from the set of virtual images, the training set being usable for training a downstream model configured to model a semiconductor manufacturing process.

2. The method of claim 1 , wherein the synthetic segmented image set is generated using a second trained GAN.

3. 3. The method of claim 2, wherein the first trained GAN and the second trained GAN are trained simultaneously.

4. 4. The method of claim 3, wherein simultaneously training the first trained GAN and the second trained GAN comprises updating weights for a first classifier and / or a first generator associated with the first trained GAN based on a classifier loss and / or a generator loss for a second classifier and / or a second generator associated with the second trained GAN.

5. 5. The method of claim 1, wherein generating the set of synthetic segmented images comprises generating each synthetic segmented image using a semiconductor process model.

6. 6. The method of claim 5, wherein the semiconductor process model is a model configured to represent a particular semiconductor manufacturing process.

7. 7. The method of claim 6, wherein the particular semiconductor manufacturing process is an etch process, and the semiconductor process model generates an image representing an etch profile as a result of the etch process assuming a set of process conditions provided as input to the semiconductor process model.

8. 6. The method of claim 5, further comprising applying a texture mapping transformation technique to at least one synthetic segmented image of the set of synthetic segmented images generated by the semiconductor processing model, wherein the at least one synthetic segmented image to which the texture mapping transformation technique has been applied is included in the set of synthetic segmented images.

9. 5. The method of claim 1, wherein generating the virtual image set comprises: generating a corresponding grayscale image from a given composite segmented image of the set of composite segmented images; providing the corresponding grayscale image to the first trained GAN; A method comprising:

10. 10. The method of claim 9, wherein generating the corresponding grayscale image comprises applying a randomly generated color palette to the given composite segmented image.

11. 5. The method of claim 1, wherein the segmented semiconductor metrology image corresponds to an electron microscopy (EM) image, a scanning electron microscopy (SEM) image, a critical dimension SEM image (CD-SEM), or a transmission electron microscopy (TEM) image.

12. 5. The method of claim 1, wherein the downstream model trained with the training set is configured to label interfaces between materials in input images having the same type as the segmented semiconductor measurement image.

13. 5. The method of claim 1, wherein the downstream model trained by the training set is configured to provide dimensional measurements of one or more wafer features included in an input image having the same type as the segmented semiconductor measurement image.

14. 5. The method of claim 1, wherein the downstream model trained with the training set is configured to provide at least one measurement associated with an interface between materials in an input image having the same type as the segmented semiconductor measurement image.

15. 5. The method of claim 1, wherein the first trained GAN utilizes multiple sets of classifiers and generators, each set of classifiers and generators corresponding to a different layer of a pyramid structure, and each layer is configured to utilize images of different sizes.

16. 16. The method of claim 15, wherein each set of classifiers and generators in the plurality of sets of classifiers and generators is trained consecutively.

17. 1. A computer program product comprising a non-transitory computer-readable medium provided with computer-executable instructions for causing a computer system to perform a method for generating composite semiconductor image data, the method comprising: generating a set of composite segmented images, each composite segmented image of the set representing a segmented semiconductor measurement image; generating a set of virtual images using the first trained GAN, each virtual image corresponding to one of the synthetic segmented images in the set of synthetic segmented images; 1. A computer program product comprising: constructing a training set including a plurality of training samples, each training sample including a synthetic segmented image from the set of synthetic segmented images and a corresponding virtual image from the set of virtual images, the training set being usable for training a downstream model configured to model a semiconductor manufacturing process.

18. 20. The computer program product of claim 17, wherein the synthetic segmented image set is generated using a second trained GAN.

19. 20. The computer program product of claim 18, wherein the first trained GAN and the second trained GAN are trained simultaneously.

20. 20. The computer program product of claim 19, wherein simultaneously training the first trained GAN and the second trained GAN comprises updating weights for a first classifier and / or a first generator associated with the first trained GAN based on a classifier loss and / or a generator loss for a second classifier and / or a second generator associated with the second trained GAN.

21. 21. The computer program product of any one of claims 17 to 20, wherein generating the set of synthetic segmented images comprises generating each synthetic segmented image using a semiconductor process model.

22. 22. The computer program product of claim 21, wherein the semiconductor process model is a model configured to represent a particular semiconductor manufacturing process.

23. 23. The computer program product of claim 22, wherein the particular semiconductor manufacturing process is an etch process, and wherein the semiconductor process model generates an image representing an etch profile as a result of the etch process assuming a set of process conditions provided as inputs to the semiconductor process model.

24. 22. The computer program product of claim 21, wherein the method further comprises applying a texture mapping transformation technique to at least one synthetic segmented image of the set of synthetic segmented images generated by the semiconductor processing model, and the at least one synthetic segmented image to which the texture mapping transformation technique has been applied is included in the set of synthetic segmented images.

25. 21. The computer program product of claim 17, wherein generating the virtual image set comprises: generating a corresponding grayscale image from a given composite segmented image of the set of composite segmented images; providing the corresponding grayscale image to the first trained GAN; A computer program product comprising:

26. 26. A computer program product as recited in claim 25, comprising generating the corresponding grayscale image and applying a randomly generated color palette to the given composite segmented image.

27. 21. The computer program product of any one of claims 17 to 20, wherein the segmented semiconductor metrology image corresponds to an electron microscopy (EM) image, a scanning electron microscopy (SEM) image, a critical dimension SEM image (CD-SEM), or a transmission electron microscopy (TEM) image.

28. 21. The computer program product of claim 17, wherein the downstream model trained with the training set is configured to label interfaces between materials in input images having the same type as the segmented semiconductor measurement image.

29. 21. The computer program product of claim 17, wherein the downstream model trained with the training set is configured to provide dimensional measurements of one or more wafer features included in an input image having the same type as the segmented semiconductor metrology image.

30. 21. The computer program product of claim 17, wherein the downstream model trained with the training set is configured to provide at least one measurement associated with an interface between materials in an input image having the same type as the segmented semiconductor measurement image.

31. 21. The computer program product of claim 17, wherein the first trained GAN utilizes multiple sets of classifiers and generators, each set of classifiers and generators corresponding to a different layer of a pyramid structure, each layer configured to utilize images of different sizes.

32. 32. The computer program product of claim 31, wherein each set of classifiers and generators in the plurality of sets of classifiers and generators is trained consecutively.

Images