Substrate image segmentation using artificial intelligence (AI) in semiconductor manufacturing
A single AI model for semiconductor manufacturing efficiently segments non-periodic and novel patterns using reference images, addressing resource inefficiencies and enhancing production accuracy.
Patent Information
- Application Number
- PCT/CN2025/095044
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2025-05-15
- Publication Date
- 2026-11-19
AI Technical Summary
Conventional image segmentation methods in semiconductor manufacturing struggle with non-periodic patterns, deformed objects, and novel patterns, requiring multiple segmentation processes and substantial computational resources, leading to inefficiencies and resource consumption.
A single AI model is trained to predict segmentation masks for various pattern types using reference pattern images and segmentation masks, allowing it to handle irregular shapes and new patterns without additional preprocessing, reducing the need for multiple models.
The AI model efficiently generates accurate segmentation masks for diverse patterns, saving computing resources and improving production efficiency by reducing the need for multiple models.
Smart Images

Figure CN2025095044_19112026_PF_FP_ABST
Abstract
Description
SUBSTRATE IMAGE SEGMENTATION USING ARTIFICIAL INTELLIGENCE (AI) IN SEMICONDUCTOR MANUFACTURINGTECHNICAL FIELD
[0001] Embodiments of the present disclosure relate, in general, to manufacturing systems and more particularly to systems and methods for substrate image segmentation using artificial intelligence (AI) in semiconductor manufacturing.BACKGROUND
[0002] Image segmentation can be performed on a pattern image depicting a pattern of a substrate to ensure that semiconductor manufacturing systems are producing quality substrates. However, non-periodic patterns (e.g., patterns that do not repeat at regular intervals or lack consistent structure) , patterns including deformed objects (e.g., non-circular holes) , and novel patterns (e.g., patterns not previously encountered) may be difficult to segment due to varying object characteristics (e.g., different shapes, sizes, etc. ) that make it challenging to accurately identify all the objects in the pattern. As such, image segmentation of pattern images can involve performing multiple segmentation processes or training multiple AI models to identify different types of objects in a pattern, which can require substantial memory and consumes significant computational resources.SUMMARY
[0003] Some of the embodiments described cover substrate image segmentation in a manufacturing system. The method includes receiving, from a client device, a first input including a first reference pattern image depicting a first pattern associated with a first substrate processed by manufacturing equipment, a first reference segmentation mask representing one or more first segmented regions of the first pattern depicted in the first reference pattern image, and a target pattern image depicting a second pattern, where the second pattern corresponds to the first pattern, and where the second pattern is associated with a second substrate processed by the manufacturing equipment. The method further includes providing the first input to an artificial intelligence (AI) model trained using a set of pattern images depicting patterns of different pattern types and obtaining one or more first outputs of the AI model, where the one or more first outputs indicate a first predicted target segmentation mask representing one or more second segmented regions of the second pattern depicted in the target pattern image. The method further includes providing, to the client device, the first predicted target segmentation mask for presentation.
[0004] In some implementations, the method further includes determining one or more measurements of the one or more second segmented regions depicted in the first predicted target segmentation mask, where the one or more measurements are associated with a size and location of the one or more second segmented regions. The method further includes updating one or more settings of a process recipe based on the one or more measurements.
[0005] In some implementations, the method further includes receiving, from the client device, a second input including a second reference pattern image depicting a third pattern associated with a third substrate, where a type of the third pattern is distinct from a type of the first pattern, a second reference segmentation mask representing one or more third segmented regions of the third pattern depicted in the second reference pattern image, and the target pattern image further depicting a fourth pattern, where the fourth pattern corresponds to the third pattern, and where the fourth pattern is associated with the second substrate. The method further includes providing the second input to the AI model and obtaining one or more second outputs of the AI model, where the one or more second outputs indicate a second predicted target segmentation mask representing one or more fourth segmented regions of the fourth pattern depicted in the target pattern image, and where the fourth segmented regions are distinct from the second segmented regions. The method further includes providing, to the client device, the second predicted target segmentation mask for presentation.
[0006] In some implementations, the method further includes generating a third target segmentation mask based on the first predicted target segmentation mask and the second predicted target segmentation mask, where the third target segmentation mask represents the one or more second segmented regions and the one or more fourth segmented regions. The method further includes providing, to the client device, the third target segmentation mask for presentation.
[0007] In some implementations, the first reference segmentation mask is generated by at least one of threshold edge detection, template matching, or running an object detection model.
[0008] In some implementations, a type of the second pattern depicted in the target pattern image is non-periodic pattern type.
[0009] In some implementations, the first reference pattern image and the target pattern image depict a pattern of a pattern type that is distinct from a set of pattern types of patterns included in training data provided to train the AI model.
[0010] In some implementations, the first reference pattern image and the target pattern image are one of transmission electron microscopy (TEM) images or scanning transmission electron microscopy (STEM) images.
[0011] In some implementations, a system for substrate image segmentation in a manufacturing system includes a memory and a set of one or more processing devices coupled to the memory. The set of one or more processing devices is to generate training data for training an artificial intelligence (AI) model to predict a segmentation mask for pattern images depicting patterns on a substrate. To generate the training data, the set of one or more processing devices is to generate a training input including a reference pattern image depicting a first pattern, a reference segmentation mask associated with one or more first segmented regions of the first pattern, and a target pattern image depicting a second pattern of a historical substrate, where the second pattern corresponds to the first pattern. The set of one or more processing devices is further to generate a target output for the training input, where the target output indicates a target segmentation mask associated with one or more second segmented regions of the target pattern image. The set of one or more processing devices is further to provide the training data to train the machine learning model on (i) a set of training inputs including the training input and (ii) a set of target outputs including the target output.
[0012] In some implementations, the reference segmentation mask is generated by at least one of threshold edge detection, template matching, or running an object detection model.
[0013] In some implementations, the first pattern and the second pattern include at least one of holes, bitlines, scanning transmission electron microscopy (STEM) gate-all-around (GAA) transistor structures, transmission electron microscopy (TEM) GAA transistor structures, or blobs.
[0014] In some implementations, the reference pattern image and the target pattern image are one of TEM images or STEM images.
[0015] In some implementations, at least one of the reference pattern image or the target pattern image are preprocessed, where preprocessing includes at least one of rotating at least one of the reference pattern image or the target pattern image, adding noise to at least one of the reference pattern image or the target pattern image, or adjusting a brightness gradient of at least one of the reference pattern image or the target pattern image.
[0016] In some implementations, adding noise further includes modifying an intensity value of one or more pixels in an image.
[0017] In some implementations, the set of one or more processing devices is further to extract one or more first image descriptors from the reference pattern image, extract one or more second image descriptors from the reference segmentation mask, determine the first pattern based on the one or more first image descriptors and the one or more second image descriptors, extract one or more third image descriptors from the target pattern image, determine the second pattern based on the one or more third image descriptors and the first pattern; and identify the target segmentation mask based on the second pattern and the third image descriptors.BRIEF DESCRIPTION OF THE DRAWINGS
[0018] The present disclosure is illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings in which like references indicate similar elements. It should be noted that different references to “an” or “one” embodiment in this disclosure are not necessarily to the same embodiment, and such references mean at least one.
[0019] FIG. 1 depicts an illustrative system architecture, according to aspects of the present disclosure.
[0020] FIG. 2 is a block diagram of an example masking engine, according to aspects of the present disclosure.
[0021] FIG. 3 is a flow chart of an example method for training an artificial intelligence (AI) model, according to aspects of the present disclosure.
[0022] FIG. 4 is a flow chart of an example method for image segmentation using AI, according to aspects of the present disclosure.
[0023] FIG. 5 depicts a block diagram of an illustrative computer system operating in accordance with one or more aspects of the present disclosure.DETAILED DESCRIPTION OF EMBODIMENTS
[0024] Implementations described herein provide systems and methods for image segmentation using artificial intelligence (AI) . In semiconductor manufacturing, image segmentation is used to identify the location and / or size of an object (e.g., a hole, bitline, transistor, etc. ) depicted in a pattern image, or an image representing one or more objects on a substrate. As electronic device manufacturing is a quickly growing industry that involves highly precise and complex operations, small variations or errors in production (e.g., shifts in the location of objects, changes in the size of an object, etc. ) can accumulate and yield significant losses, reduced performance, or higher costs (e.g., production costs to address substrate defects or meet manufacturing criteria) . For example, a change in the length of a transistor gate can affect the speed, leakage of electrical current, and / or power consumption of a substrate. As such, image segmentation is used to determine whether a substrate satisfies performance criteria (e.g., speed, power consumption, etc. ) and determine if there are defects based on the size and / or location of objects in the substrate.
[0025] Conventional systems use threshold edge detection and / or template matching to identify objects depicted in pattern images. A pattern is a recurring design or arrangement of a type of object (e.g., holes, bitlines, etc. ) . Threshold edge detection classifies pixels in the pattern image based on an intensity threshold. For example, pixels with an intensity above the threshold are classified as edges, while pixels with an intensity below the threshold are classified as non-edges. An edge detection algorithm (e.g., Sobel, Canny, Prewitt, etc. ) is an algorithm that detects areas of an image with high intensity gradients, which is used to identify locations in the image with significant intensity differences. By identifying the edges of an object, threshold edge detection can be used to identify the location and size of the object in the pattern image. Template matching can identify objects by using a template image, or an image of a pattern of one or more objects, to search for similar objects in the pattern image. The template image can be compared with different regions of the pattern image to identify a region of the pattern image that is the most similar to the template image. For example, a similarity score can be calculated for different regions of the pattern image based on the correlation, mean squared error, and / or normalized cross-correlation with the template image. The region of the pattern image with the highest similarity score can be considered as the most similar to the template image.
[0026] However, noise (e.g., random variation in pixel values and / or intensity) in a pattern image, defects in object shapes (e.g., holes that are not circular) , and / or non-periodic patterns can cause threshold edge detection and template matching to falsely identify and / or fail to identify objects in a pattern image. For example, variation in the pixel intensity of a pattern image can cause threshold edge detection to report an object in a pattern image when there is no object present in that location. Defects in object shapes can cause template matching to fail to identify an object because the shape of the object was not similar to the shape of the objects in the pattern of the template image. For example, template matching may not identify a hole that is not circular because the similarity score with the circular holes in the template image is low. Lastly, template matching may fail to identify objects when a pattern image depicts a non-periodic pattern, or a pattern that does not repeat at regular intervals or have a consistent structure, because template images depict fixed patterns and do not adjust to account for objects of different sizes and / or distances from each other. As such, performing image segmentation using threshold edge detection and / or template matching can require preprocessing images to reduce noise and / or performing one of the processes multiple times to account for defects in object shapes and / or non-periodic patterns, which can consume a significant amount of memory and computing resources (e.g., memory space, processing cycles, etc. ) .
[0027] Another conventional solution involves training an AI model to identify an object in a pattern image. For example, an AI model can be trained to receive a pattern image depicting a pattern of objects and predict the location and size of one or more objects based on the input pattern image (e.g., by generating a bounding box around the object and measuring the size of the bounding box) . However, conventional systems use multiple AI models that are each trained for one of the different patterns (e.g., one model trained to detect holes, one model trained for bitlines, etc. ) . As such, a system would train and maintain a separate AI model for each type of pattern. If a pattern image contains more than one type of pattern, then multiple models may need to be executed to identify different objects. The training, maintenance, and use of multiple AI models can consume computing resources, such as memory space and processing cycles. Additionally, the AI models can fail to identify patterns and / or objects that are not included in the training data.
[0028] Aspects of the present disclosure address the above noted and other deficiencies by providing systems and methods for image segmentation using an AI model that is applicable to different pattern types. A system can train such an AI model (e.g., a machine learning model) to predict a segmentation mask, or an image or binary representation that defines regions of interest corresponding to objects, based on a pattern image of a substrate, a reference pattern image, and a reference segmentation mask corresponding to the reference pattern image. A pattern image can include an image generated using transmission electron microscopy (TEM) and / or scanning transmission electron microscopy (STEM) . A pattern can include holes, bitlines, gate-all-around (GAA) transistor structures, or blobs. The reference segmentation mask can be a segmentation mask generated using threshold edge detection, template matching, and / or by running an object detection model.
[0029] In some instances, the AI model can be trained using data that includes a set of training inputs and a set of target outputs. A training input can include a target pattern image of a historical substrate, a reference pattern image, and a reference segmentation mask. A target output for the training input can include a target segmentation mask associated with one or more objects in the target pattern image. In some embodiments, the target pattern image and / or the reference pattern image can be preprocessed, which can include rotating, adding noise (e.g., changing the intensity of one or more pixels) , or adjusting a brightness gradient of the image. The AI model can be trained to generate a segmentation mask for multiple pattern images depicting different patterns. For example, one training input can include a target pattern image depicting holes, a reference pattern image depicting similar holes, and a reference segmentation mask depicting the similar holes and the training output can include a target segmentation mask depicting the holes in the target pattern image. Another training input can include a target pattern image depicting bitlines, a reference pattern image depicting similar bitlines, and a reference segmentation mask depicting the similar bitlines. The training output can be a reference segmentation mask depicting the bitlines in the target pattern image.
[0030] Upon training the AI model, the AI model can be applied to a pattern image depicting a pattern including one or more objects on a substrate processed at a manufacturing system. For example, the system can receive, from a client device, an input including a target pattern image, a reference pattern image, and a reference segmentation mask. The reference pattern image and reference segmentation mask can include a pattern (e.g., a hole) that is similar to the pattern in the target pattern image. The system can provide the input to the trained AI model and obtain one or more outputs, including a target segmentation mask corresponding to the pattern depicted in the target pattern image. The system can then provide the target segmentation mask to the client device. In some embodiments, the system can predict a segmentation mask for a pattern that it has not been trained with. For example, the input can include a target pattern image depicting a pattern the AI model has not been trained on (e.g., star-shaped objects) , a reference pattern image depicting a similar pattern (e.g., a different pattern or a portion of the target pattern depicting star-shaped objects) , and a reference segmentation mask depicting the similar pattern. The AI model can then generate a target segmentation mask that depicts the pattern in the target image based on the inputs.
[0031] In some embodiments, the target pattern image can include more than one pattern. For example, a target pattern image may depict holes and bitlines. The system can determine which pattern to generate a segmentation mask for based on the reference pattern image and reference segmentation mask. For a reference pattern image and a reference segmentation mask depicting holes, the system can generate a segmentation mask depicting the holes in the target pattern image. For a reference pattern image and reference segmentation mask depicting bitlines, the system can generate a segmentation mask depicting the bitlines in the target pattern image. In some embodiments, the system can combine the two generated segmentation masks to generate a new segmentation mask depicting multiple patterns (e.g., both the holes and bitlines) .
[0032] In some embodiments, the system can determine one or more measurements (e.g., the size of an object, the critical dimension of an object) associated with one or more objects depicted in the target segmentation mask. The measurements can be provided for display on a client device, in some embodiments. In other embodiments, the system can determine one or more parameters to change to tune a process recipe based on the measurements. For example, if one or more holes in a pattern are determined to be too large, then the system can suggest reducing the amount of etching performed on a substrate and / or adjusting the flow rate of reactive gases (e.g., fluorine, chlorine, etc. ) to reduce excessive material removal.
[0033] Aspects of the present disclosure address deficiencies of the conventional technology by providing techniques for image segmentation using an AI model that is applicable to different pattern types. As described herein, an AI model can be trained to predict a segmentation mask associated with a pattern on a substrate based on a pattern image depicting the pattern on the substrate, a reference pattern image depicting a similar pattern, and a reference segmentation mask depicting the similar pattern. Accordingly, the AI model has a higher tolerance to image noise and defects because it uses the reference pattern image and reference segmentation mask when generating the target segmentation mask, allowing it to account for irregular shapes. For example, the AI model can generate a segmentation mask for a target pattern image depicting non-circular holes based on a reference pattern image and reference segmentation mask depicting similar non-circular holes. Additionally, the AI model can be used to generate segmentation masks for non-periodic patterns and patterns it has not been trained on because it can identify new objects based on the reference pattern image and reference segmentation mask (e.g., based on image descriptors of the reference image and reference segmentation mask) , even if the pattern depicted in the references is not identical to the pattern in the target pattern image. For example, the references can depict holes that are located at a specific distance from each other, which the AI model can use to identify similar holes, which may be of a different size and / or different distances from each other, in the target pattern image. As such, the AI model can identify patterns without an identical template, improving the accuracy of generated segmentation masks without preprocessing the images and / or performing additional processes to identify defective and / or new objects, saving computing resources. Additionally, the system can generate segmentation masks for different patterns using a single AI model. The reduction in AI models reduces the computing resources spent to train and maintain multiple models. In view of the above, embodiments of the present disclosure enable the system to generate a segmentation mask of a pattern depicted in a target pattern image using a single AI model, which improves the overall efficiency of the system by reducing computing resources, which can be allocated to other systems and processes.
[0034] FIG. 1 depicts an illustrative system architecture 100, according to aspects of the present disclosure. System architecture 100 can include a client device 120, manufacturing equipment 124, metrology equipment 128, a predictive server 112 (e.g., to generate predictive data, to provide model adaptation, to use a knowledge base, etc. ) , and / or a data store 140. The predictive server 112 can be part of a predictive system 110. The predictive system 110 can further include server machines 170 and 180. In some embodiments, system architecture 100 can be included as part of or otherwise connected to a manufacturing system for processing substrates.
[0035] Manufacturing equipment 124 can produce products, such as electronic devices, following a recipe or performing runs over a period of time. Manufacturing equipment 124 can include a process chamber. Manufacturing equipment 124 can perform a process for a substrate (e.g., a wafer, etc. ) at the process chamber. Examples of substrate processes include a deposition process to deposit a film on a surface of the substrate, an etch process to form a pattern on the surface of the substrate, a polishing process to polish a material on the surface of the substrate, etc. Manufacturing equipment 124 can perform each process according to a process recipe. A process recipe defines a particular set of operations to be performed for the substrate during the process and can include one or more settings associated with each operation. For example, a deposition process recipe can include a temperature setting for the process chamber, a pressure setting for the process chamber, a flow rate setting for a precursor for a material included in the film deposited on the substrate surface, etc. Substrates that are processed according to a process recipe (e.g., for manufacturing a portion of an electronic device, etc. ) are referred to herein as production substrates.
[0036] Manufacturing equipment 124 can include one or more sensors that capture data for a substrate being processed at the manufacturing system. In some embodiments, the manufacturing equipment 124 and the sensors can be part of a sensor system that includes a sensor server (e.g., field service server (FSS) at a manufacturing facility) and sensor identifier reader (e.g., front opening unified pod (FOUP) radio frequency identification (RFID) reader for sensor system) . Sensor data may include a value of one or more of temperature (e.g., heater temperature) , spacing (SP) , pressure, high frequency radio frequency (HFRF) , RF bias, voltage of electrostatic chuck (ESC) , electrical current, flow, power, voltage, etc. Sensor data may be associated with or indicative of manufacturing parameters such as hardware parameters, such as settings or components (e.g., size, type, etc. ) of the manufacturing equipment 124, or process parameters of the manufacturing equipment 124. The sensor data can be provided while the manufacturing equipment 124 is performing manufacturing processes (e.g., equipment readings when processing products) . The sensor data can be different for each substrate. In some embodiments, sensor data can include trace data collected during performance of one or more processes (e.g., substrate processes, maintenance processes, etc. ) at manufacturing equipment 124. Trace data refers to data that indicates how components in a process chamber are operating and / or a state of an environment within a process chamber before, during, or after performance of an operation.
[0037] Manufacturing equipment 124 can include an inspection station where substrates are inspected after being processed in the process chamber. In some embodiments, the inspection station can be located at or included in a process tool of manufacturing equipment 124. A process tool refers to or can include specialized manufacturing equipment located in a processing chamber that performs fabrication steps (e.g., deposition) on substrates. In some embodiments, the inspection station can be located in the vacuum environment of the process tool. In other embodiments, the inspection station can be located outside the vacuum environment (e.g., as external equipment from the process tool) .
[0038] Metrology equipment 128 provides metrology data associated with substrates (e.g., production substrates, seasoning substrates, etc. ) processed by manufacturing equipment 124. The metrology data can include a value of one or more of film property data (e.g., wafer spatial film properties) , dimensions (e.g., thickness, height, etc. ) , dielectric constant, dopant concentration, density, defects, etc. In some embodiments, the metrology data can further include a value of one or more surface profile property data (e.g., an etch rate, an etch rate uniformity, a critical dimension of one or more features included on a surface of the substrate, a critical dimension uniformity across the surface of the substrate, an edge placement error, etc. ) . The metrology data can be of a finished or semi-finished product. The metrology data can be different for each substrate. Metrology equipment 128 can be configured to generate metrology data associated with a substrate before or after a substrate process and / or a maintenance process. In some embodiments, metrology equipment 128 can be part of a metrology system that includes a metrology server (e.g., a metrology database, metrology folders, etc. ) and metrology identifier reader (e.g., FOUP RFID reader for metrology system) .
[0039] Metrology equipment 128 can be integrated with a station of the process tool of manufacturing equipment 124. In some embodiments, metrology equipment 128 can be coupled to or be a part of a station of the process tool that is maintained under a vacuum environment (e.g., a process chamber, a transfer chamber, etc. ) . Such metrology equipment 128 is referred to as integrated metrology equipment. Accordingly, the substrate can be measured by the integrated metrology equipment while the substrate is in the vacuum environment. For example, after a process (e.g., an etch process, a deposition process, etc. ) is performed for the substrate, the metrology data for the substrate can be generated by the integrated metrology equipment without the processed substrate being removed from the vacuum environment. In other or similar embodiments, metrology equipment 128 can be coupled to or be a part of the process tool station that is not maintained under a vacuum environment (e.g., a factory interface module, etc. ) . Such metrology equipment is referred to as inline metrology equipment. Accordingly, the substrate is measured by the inline metrology equipment outside of the vacuum environment.
[0040] In additional or alternative embodiments, metrology equipment 128 can include metrology measurement devices that are separate (i.e., external) from manufacturing equipment 124. For example, metrology equipment 128 can be standalone equipment that is not coupled to any station of manufacturing equipment 124. For a measurement to be obtained for a substrate using external metrology equipment, a user of a manufacturing system (e.g., an engineer, an operator) can cause a substrate processed at manufacturing equipment 124 to be removed from manufacturing equipment 124 and transferred to metrology equipment 128 for measurement. In some embodiments, metrology equipment 128 can transfer metrology data generated for the substrate to the client device 120 coupled to metrology equipment 128 via network 130 (e.g., for presentation to a manufacturing user, such as an operator or an engineer) . In other or similar embodiments, the manufacturing system user can obtain metrology data for the substrate from metrology equipment 128 and can provide the metrology data to computer system architecture via a graphical user interface (GUI) of client device 120.
[0041] The client device 120 may include a computing device such as personal computers (PCs) , laptops, mobile phones, smart phones, tablet computers, netbook computers, network connected televisions ( “smart TVs” ) , network-connected media players (e.g., Blu-ray player) , a set-top box, over-the-top (OTT) streaming devices, operator boxes, etc. In some embodiments, the metrology data may be received from the client device 120. In some embodiments, client device 120 displays a graphical user interface (GUI) , where the GUI enables the user to provide, as input, metrology measurement values for substrates processed at the manufacturing system. In other or similar embodiments, client device 120 can display another GUI that enables the user to provide, as input, an indication of a type of substrate to be processed at the manufacturing system, a type of process to be performed for the substrate, and / or a type of equipment at the manufacturing system. In yet other or similar embodiments, client device 120 can display another GUI that that presents sensor data collected by the sensors before, during, or after performance of a process (e.g., a substrate process, a maintenance process, etc. ) . It should be noted that one or more GUIs of client device 120 can provide and / or receive any data described herein.
[0042] Data store 140 can be a memory (e.g., random access memory) , a drive (e.g., a hard drive, a flash drive) , a database system, or another type of component or device capable of storing data. Data store 140 can include multiple storage components (e.g., multiple drives or multiple databases) that can span multiple computing devices (e.g., multiple server computers) . The data store 140 can store data associated with processing a substrate at manufacturing equipment 124. For example, data store 140 can store data collected by the sensors at manufacturing equipment 124 before, during, or after a substrate process (referred to as process data) . Process data can refer to historical process data (e.g., process data generated for a previous substrate processed at the manufacturing system) and / or current process data (e.g., process data generated for a current substrate processed at the manufacturing system) . Current process data can be data for which predictive data is generated. In some embodiments, data store can store metrology data including historical metrology data (e.g., metrology measurement values for a prior substrate processed at the manufacturing system) . The data store 140 can also store contextual data associated with one or more substrates processed at the manufacturing system. Contextual data can include a recipe name, recipe operation number, preventive maintenance indicator, operator, etc. In some embodiments, contextual data can also include an indication of a difference between two or more process recipes or process operations. In some embodiments, data store 140 can additionally or alternatively store image data collected (e.g., by imaging component 126) for a substrate before, during, or after processing of the substrate, as described herein.
[0043] In some embodiments, data store 140 can be configured to store data that is not accessible to a user of the manufacturing system. For example, process data, spectral data, non-spectral data, and / or positional data obtained for a substrate being processed at the manufacturing system may not be accessible to a user of the manufacturing system. In some embodiments, all data stored at data store 140 is inaccessible by a user (e.g., an operator) of the manufacturing system. In other or similar embodiments, a portion of data stored at data store 140 is inaccessible by the user while another portion of data stored at data store 140 is accessible by the user. In some embodiments, one or more portions of data stored at data store 140 are encrypted using an encryption mechanism that is unknown to the user (e.g., data is encrypted using a private encryption key) . In other or similar embodiments, data store 140 includes multiple data stores where data that is inaccessible to the user is stored in one or more first data stores and data that is accessible to the user is stored in one or more second data stores.
[0044] Computing system 150 can include a masking engine 152 that predicts a segmentation mask, a binary or multi-class image where each pixel of the image is labeled to indicate whether it is part of a region of interest (e.g., an area within the image that is identified for additional measurements and / or analysis, such as determining the shape of the region) , by applying an AI model (e.g., model 190) to one or more images (e.g. images generated using transmission electron microscopy (TEM) , which generates an image based on electron density differences detected by measuring the interaction of electrons from a high energy electron beam with a substrate, and / or scanning transmission electron microscopy (STEM) , which generates an image using a focused electron beam by detecting the interactions of transmitted electrons with the substrate) of a reference pattern image depicting a first pattern corresponding to a substrate processed by manufacturing equipment (e.g., manufacturing equipment 124) , a reference segmentation mask corresponding to the reference pattern image that represents one or more regions of interest (e.g., objects that make up the first pattern) , and a target pattern image depicting a second pattern, where the second pattern is similar to the first pattern. A pattern is a recurring design or arrangement of a type of object (e.g., holes, bitlines, transistor structures, etc. ) . Each pattern can include one or more pattern types, which refer to classifications based on the shape and / or distribution of the objects (e.g., periodic arrangements of a single type of object, non-periodic distributions) . The reference pattern image and reference segmentation mask are used to determine the type of pattern to be identified and segmented (e.g., labeled as regions of interest) in the target pattern image. For example, the reference pattern image and segmentation mask can depict a first pattern including holes, which the masking engine 152 can use to identify a second pattern including holes in the target pattern image. In some embodiments, the holes in the second pattern can be of a different size and / or spread (e.g. the proximity between objects) than the holes in the first pattern. The masking engine 152 can receive the reference pattern image, reference segmentation mask, and target pattern image from a client device 120, in some embodiments. The masking engine 152 can provide the one or more images to the model 190 trained to predict a segmentation mask for the target pattern image based on the reference pattern image, reference segmentation mask, and the target pattern image. In some embodiments, masking engine 152 can include or have access to a predictive component 114 (e.g., of predictive system 110) which provides the reference pattern image, reference segmentation mask, and target pattern image as an input to the model 190. In other or similar embodiments, predictive system 110 can train the model 190 to predict segmentation masks for pattern images, as described below with respect to FIG. 3.
[0045] In some embodiments, predictive system 110 includes server machine 170 and server machine 180. Server machine 170 includes a training set generator 172 that is capable of generating training data sets (e.g., a set of data inputs and a set of target outputs) to train, validate, and / or test an AI model190. Some operations of training set generator 172 are described in detail below with respect to FIG. 3. In some embodiments, the training set generator 172 can partition the training data into a training set, a validating set, and a testing set. In some embodiments, the predictive system 110 generates multiple sets of training data.
[0046] Server machine 180 includes a training engine 182, a validation engine 184, a selection engine 186, and / or a testing engine 188. An engine can refer to hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, processing device, etc. ) , software (such as instructions run on a processing device, a general purpose computer system, or a dedicated machine) , firmware, microcode, or a combination thereof. Training engine 182 can be capable of training an AI model 190. The AI model 190 can refer to the model artifact that is created by the training engine 182 using the training data that includes training inputs and corresponding target outputs (correct answers for respective training inputs) . The training engine 182 can find patterns in the training data that map the training input to the target output (the answer to be predicted) , and provide the patterns to the AI model 190 that captures these patterns. In some embodiments, the AI model 190 uses one or more of support vector machine (SVM) , Radial Basis Function (RBF) , clustering, supervised machine learning, semi-supervised machine learning, unsupervised machine learning, k-nearest neighbor algorithm (k-NN) , linear regression, random forest, neural network (e.g., artificial neural network, a recurrent neural network, a convolutional neural network, etc. ) , clustering techniques (e.g., hierarchical clustering techniques) , association techniques (e.g., apriori techniques) , classification techniques (e.g., decision trees, random forest techniques, etc. ) , a variational recurrent auto-encoder, etc. It should be noted that although some embodiments of the present disclosure describe model 190 as an AI model, such embodiments can be applied to any type of AI model, non-AI based model (e.g., a statistical model, a physical model, etc. ) , and / or a hybrid model (e.g., implementing AI techniques and non-AI techniques) .
[0047] The validation engine 184 can be capable of validating a trained AI model 190 using a corresponding set of features of a validation set from training set generator 172. The validation engine 184 can determine an accuracy of each of the trained AI models 190 based on the corresponding sets of features of the validation set. The validation engine 184 can discard a trained AI model 190 that has an accuracy that does not meet a threshold accuracy. In some embodiments, the selection engine 186 can be capable of selecting a trained AI model 190 that has an accuracy that meets a threshold accuracy. In some embodiments, the selection engine 186 can be capable of selecting the trained AI model 190 that has the highest accuracy of the trained AI models 190.
[0048] The testing engine 188 can be capable of testing a trained AI model 190 using a corresponding set of features of a testing set from training set generator 172. For example, a first trained AI model 190 that was trained using a first set of features of the training set can be tested using the first set of features of the testing set. The testing engine 188 can determine a trained AI model 190 that has the highest accuracy of all of the trained AI models based on the testing sets.
[0049] Predictive server 112 includes a predictive component 114 that is capable of providing data as an input to a trained model 190 and obtaining one or more outputs of the trained model 190. As described herein, predictive component 114 can be a component of or otherwise associated with masking engine 152 and can provide the reference pattern image, reference segmentation mask, and target pattern image as an input to trained model 190. Predictive component 114 can obtain one or more outputs of trained model 190, which can include a predicted segmentation mask corresponding to a pattern depicted in the target pattern image, as described herein.
[0050] As illustrated by FIG. 1, the client device 120, manufacturing equipment 124, metrology equipment 128, predictive server 112, data store 140, server machine 170, and server machine 180 can be coupled to each other via a network 130. In some embodiments, network 130 is a public network that provides client device 120 with access to predictive server 112, data store 140, and other publicly available computing devices. In some embodiments, network 130 is a private network that provides client device 120 access to manufacturing equipment 124, metrology equipment 128, data store 140, and other privately available computing devices. Network 130 can include one or more wide area networks (WANs) , local area networks (LANs) , wired networks (e.g., Ethernet network) , wireless networks (e.g., an 802.11 network or a Wi-Fi network) , cellular networks (e.g., a Long-Term Evolution (LTE) network) , routers, hubs, switches, server computers, cloud computing networks, and / or a combination thereof.
[0051] It should be noted that in some other implementations, the functions of server machines 170 and 180, as well as predictive server 112, can be provided by a fewer number of machines. For example, in some embodiments, server machines 170 and 180 can be integrated into a single machine, while in some other or similar embodiments, server machines 170 and 180, as well as predictive server 112, can be integrated into a single machine.
[0052] In general, functions described in one implementation as being performed by server machine 170, server machine 180, and / or predictive server 112 can also be performed on client device 120. In addition, the functionality attributed to a particular component can be performed by different or multiple components operating together.
[0053] In embodiments, a “user” can be represented as a single individual. However, other embodiments of the disclosure encompass a “user” being an entity controlled by one or more users and / or an automated source. For example, a set of individual users federated as a group of administrators can be considered a “user. ”
[0054] FIG. 2 is a block diagram of an example masking engine 152, according to aspects of the present disclosure. As described above, masking engine 152 can obtain a predicted segmentation mask associated with a target pattern image depicting a pattern on a substrate processed using manufacturing equipment 124. The masking engine 152 can provide the predicted segmentation mask for display via the UI of the client device 120. In some embodiments, the masking engine 152 can determine one or more measurements corresponding to the size and / or location of a region of interest in the predicted segmentation mask and update one or more settings of a process recipe based on the measurements. As illustrated by FIG. 2, masking engine 152 can include a mask component 210, a measurement component 212, a predictive component 114, and / or a process control component 214. In some embodiments, masking engine 152 can be connected to the predictive system 110 and / or memory 250 (e.g., via network 130) . The predictive system 110 can be connected to a client device 120 in some embodiments (e.g., via network 130) . In other or similar embodiments, memory 250 can include any memory of or accessible to a component of system 100. For example, memory 250 can include, or be included in, data store 140, a memory of client device 120, and so forth.
[0055] As described herein, masking engine 152 can obtain a predicted segmentation mask corresponding to a target pattern image based on one or more outputs of an AI model 190. The model 190 may be trained based on historical data associated with system 100 (or another system) to predict segmentation masks for a pattern image based on a given target pattern image, a reference pattern image, and reference segmentation mask. Details regarding training the model 190 are provided herein with respect to FIG. 3.
[0056] FIG. 3 is block diagram depicting an example method 300 for training an AI model, according to aspects of the present disclosure. Method 300 is performed by processing logic that can include hardware (circuitry, dedicated logic, etc. ) , software (such as is run on a general-purpose computer system or a dedicated machine) , firmware, or some combination thereof. In one implementation, method 300 can be performed by a computer system, such as computer system architecture 100 of FIG. 1. In other or similar implementations, one or more operations of method 300 can be performed by one or more other machines not depicted in the figures. In some aspects, one or more operations of method 300 can be performed by training set generator 172.
[0057] For simplicity of explanation, method 300 is depicted and described as a series of acts. However, acts in accordance with this disclosure can occur in various orders and / or concurrently, and with other acts not presented and described herein. Furthermore, not all illustrated acts can be performed to implement the methods in accordance with the disclosed subject matter. In addition, those skilled in the art will understand and appreciate that the methods could alternatively be represented as a series of interrelated states via a state diagram or events. Additionally, it should be appreciated that the methods disclosed in this specification are capable of being stored on an article of manufacture to facilitate transporting and transferring such methods to computing devices. The term article of manufacture, as used herein, is intended to encompass a computer program accessible from any computer-readable device or storage media.
[0058] At block 310, processing logic initializes a training set T to an empty set (e.g., {} ) . At block 312, processing logic identifies a reference pattern image depicting a first pattern, a reference segmentation mask associated with one or more first segmented regions (e.g., regions of interest) of the first pattern, and a target pattern image depicting a second pattern of a historical substrate, where the second pattern corresponds to the first pattern (e.g., includes objects similar to those included in the first pattern) . Pattern types of the patterns can include holes, bitlines, gate-all-around transistor structures, and / or blobs (e.g., high contrast regions in the image that appear as a bright or dark spot) in periodic and / or non-periodic distributions. The AI model 190 can be trained using sets of reference images, reference segmentation masks, and target pattern images depicting different types of patterns. In some embodiments, the processing logic (e.g., training set generator 172) can identify or otherwise obtain the one or more historical images from pattern image data 254 and segmentation mask data 252 in memory 250. In other or similar embodiments, the pattern image data 254 and / or segmentation mask data 252 can be provided by a user of a system 100 (e.g., a user of a client device 120) and / or a developer or operator of system 100 and / or client device 120, in some embodiments. In some embodiments, the reference pattern image and / or target pattern image can be TEM or STEM images. As described above, TEM images refer to images generated by detecting electron interactions from a broad electron beam transmitted through a substrate, while STEM images refer to images generated by detecting electron interactions from a focused electron beam as the electrons interact with a substrate.
[0059] In additional or alternative embodiments, processing logic can perform one or more preprocessing operations with respect to the target pattern image and / or reference pattern image to increase the accuracy of the AI model 190 by improving the ability of the model to adapt to and correctly generate segmentation masks for images of different image qualities. Training set generator 172 can retrieve the preprocessed images from memory 250, in some embodiments. Preprocessing can involve rotating and / or decreasing the resolution of an image. In some embodiments, the reference pattern image and / or target pattern image can be rotated (e.g., in a clockwise or counterclockwise direction relative to the center of the image) . In other or similar embodiments, preprocessing can include adding noise (e.g., modifying an intensity value of one or more pixels in an image) to the reference pattern image and / or target pattern image. Preprocessing, in some embodiments, can also include adjusting an overall light intensity of an image by uniformly increasing or decreasing the brightness value of all the pixels in an image. In some embodiments, processing logic can perform one or more preprocessing operations using one or more separate AI models (e.g., preprocessing model 256) trained to perform one or more tasks pertaining to a respective preprocessing operation.
[0060] In some embodiments, the reference segmentation mask can be a segmentation mask generated by one of threshold edge detection and / or template matching. As described above, threshold edge detection classifies pixels in a pattern image based on an intensity threshold, while template matching identifies objects using a template image to search for similar patterns in a pattern image. In other or similar embodiments, the reference segmentation mask can be generated using an AI model. For example, a reference pattern image can be provided as input to an object detection model, which can generate bounding boxes around objects detected in the image. A reference segmentation mask can be created using measurements (e.g., object size, distribution between objects) obtained from the generated boxes.
[0061] At block 314, processing logic obtains a target segmentation mask associated with the one or more second regions (e.g., regions of interest) of the target pattern image. For example, the segmentation mask can identify one or more objects (e.g., holes) corresponding to the second pattern depicted in the target pattern image. In some embodiments, the target segmentation mask can be identified from segmentation mask data 252 stored at memory 250. For example, one or more processing devices can extract image descriptors (e.g., shape, size, location, etc. ) of one or more image features (e.g., objects) depicted in the reference pattern image and the reference segmentation mask. Based on the image descriptors for the reference pattern image and reference segmentation mask, processing logic can identify a first pattern based on the image features depicted in the reference pattern image that are labeled as regions of interest in the reference segmentation mask. Processing logic can then extract one or more image descriptors from the target pattern image and determine a second pattern corresponding to the target pattern image based on the first pattern. For example, processing logic can identify the image descriptors of a reference pattern image that are labeled as regions of interest in the reference segmentation mask to determine a first pattern including holes. The processing logic can then identify image descriptors corresponding to the target pattern image that are similar to the image descriptors in the reference pattern image that are identified as regions of interest in the reference segmentation mask. Processing logic can then compare the image descriptors of the target pattern image with image descriptors of one or more reference segmentation masks that correspond to patterns on historical substrates to identify a segmentation mask that corresponds to the target pattern image. For example, the segmentation mask can identify regions of interest that correspond to image descriptors in the target pattern image that are similar to image descriptors in the reference pattern image that were identified as regions of interest in the reference segmentation mask. In some embodiments, one or more processing devices can generate image vectors, or one-dimensional arrays where each element of the array contains a numerical value that represents the characteristics (e.g., the intensity or brightness) of a pixel in an image, to represent the features of the reference and target pattern images through numerical values that describe each pixel of the images. The image vectors of the target pattern image can be compared (e.g., using cosine similarity, Euclidean distance, etc. ) with image vectors from reference pattern image to identify image vectors in the target pattern image that are similar (e.g., have similar intensity and / or brightness values) to image vectors in the reference pattern image that are labelled as regions of interest in the reference segmentation mask. Processing logic can compare the image vectors of the target pattern image with one or more reference segmentation masks each depicting patterns of historical substrates to identify a segmentation mask with regions of interest that correspond to image vectors in the target pattern image.
[0062] At block 316, processing logic generates a training input based on the reference pattern image, reference segmentation mask, and target pattern image. In some embodiments, the training input can include the reference pattern image, a reference segmentation mask, and a target pattern image. In other embodiments, the training input can include image feature data (e.g., image descriptors or image vectors) extracted from portions of the reference pattern image, reference segmentation mask, and target pattern image. In some embodiments, the target pattern image can depict a pattern corresponding to a historical substrate.
[0063] At block 318, processing logic generates a target output based on the target segmentation mask. At block 320, processing logic generates a mapping between the training input and the target output. At block 322, processing logic adds the mapping to the training set T. At block 324, processing logic determines whether the training set T includes a sufficient amount of training data to train an AI model. It should be noted that in some implementations, the sufficiency of training set T can be determined based simply on the number of mappings in the training set, while in some other implementations, the sufficiency of training set T can be determined based on one or more other criteria (e.g., a measure of diversity of the training examples, etc. ) in addition to, or instead of, the number of input / output mappings. Responsive to determining the training set does not include a sufficient amount of training data to train the AI model, method 300 returns to block 312. Responsive to determining the training set, T, includes a sufficient amount of training data to train the AI model, method 300 continues to block 326.
[0064] At block 326, processing logic provides training set T to train the AI model. In one implementation, the training set T is provided to training engine 182 of server machine 180 to perform the training. In the case of a neural network, for example, input values (e.g., the training input) of a given input / output mapping are input to the neural network, and output values (e.g., the target output) of the input / output mapping are stored in the output nodes of the neural network. The connection weights in the neural network are then adjusted in accordance with a learning algorithm (e.g., backpropagation, etc. ) , and the procedure is repeated for the other input / output mappings in the training set T. After block 326, the model 190 can be used to predict a segmentation mask, as described herein.
[0065] Referring back to FIG. 2, the masking engine 152 can generate a segmentation mask based on one or more outputs of the model 190. Embodiments relating to generating the segmentation mask are described with respect to FIG. 2 and FIG. 4.
[0066] FIG. 4 is a flow chart of an example method 400 for substrate image segmentation in a manufacturing system, according to aspects of the present disclosure. Method 400 is performed by processing logic that can include hardware (circuitry, dedicated logic, etc. ) , software (such as is run on a general-purpose computer system or a dedicated machine) , firmware, or some combination thereof. In one implementation, method 400 can be performed by a computer system, such as computer system architecture 100 of FIG. 1. In other or similar implementations, one or more operations of method 400 can be performed by one or more other machines not depicted in the figures. In some aspects, one or more operations of method 300 can be performed by masking engine 152 and / or predictive component 114.
[0067] For simplicity of explanation, method 400 is depicted and described as a series of acts. However, acts in accordance with this disclosure can occur in various orders and / or concurrently, and with other acts not presented and described herein. Furthermore, not all illustrated acts can be performed to implement the methods in accordance with the disclosed subject matter. In addition, those skilled in the art will understand and appreciate that the methods could alternatively be represented as a series of interrelated states via a state diagram or events. Additionally, it should be appreciated that the methods disclosed in this specification are capable of being stored on an article of manufacture to facilitate transporting and transferring such methods to computing devices. The term article of manufacture, as used herein, is intended to encompass a computer program accessible from any computer-readable device or storage media.
[0068] At block 410, processing logic receives, from a client device (e.g., client device 120) , a first input including a first reference pattern image depicting a first pattern associated with a first substrate processed using manufacturing equipment (e.g., manufacturing equipment 124) , a first reference segmentation mask representing one or more first segmented regions (e.g., regions of interest) of the first pattern depicted in the first reference pattern image, and a target pattern image depicting a second pattern. The second pattern can correspond to the first pattern (e.g., the patterns can include similar objects) and be associated with a second substrate processed using one or more manufacturing equipment 124. In some embodiments, the reference pattern image and target pattern image can be TEM or STEM images, as described above. In some embodiments, the reference segmentation mask is generated using at least one of threshold edge detection, template matching, or running an object detection model.
[0069] Depending on the embodiment, the target pattern image can be non-periodic (e.g., the pattern does not repeat at consistent intervals) and / or include deformed objects (e.g., non-circular holes, curved bitlines, etc. ) . In other or similar embodiments, the second pattern depicted in the target pattern image can be a pattern type that is distinct from the pattern types of patterns used to train the AI model (e.g., a pattern type that was not included in the patterns used to train the AI model 190) . Additionally, in some embodiments, the first and second substrate may be the same substrate. For example, a target pattern image can depict a distinct pattern on a substrate that was not used when training the AI model 190. The reference pattern image can depict of portion of the distinct pattern on the same substrate depicted in the target pattern image (e.g., the reference pattern image depicts a subset of the objects depicted in the target pattern image) . The AI model 190 can use the image descriptors from the reference pattern image and segmentation mask to determine the distinct pattern and identify objects in the target pattern image that are part of the distinct pattern, as described with respect to FIG. 3. The predicted target segmentation mask can be generated based on the identified objects in the target pattern image.
[0070] At block 412, processing logic provides the first input to an AI model (e.g., model 190) . In some embodiments, predictive component 114 of masking engine 152 can provide the reference pattern image, reference segmentation mask, and target pattern image as the input to the model 190, in accordance with described embodiments. The model 190 can be trained using a set of pattern images depicting patterns of different pattern types (e.g., patterns including different types of objects and / or object arrangements) to generate a segmentation mask for a pattern depicted in a target pattern image, as described with respect to FIG. 3. At block 414, processing logic obtains one or more first outputs of the AI model. In some embodiments, the one or more first outputs can include a predicted target segmentation mask representing one or more segmented regions (e.g., regions of interest) in the target pattern image.
[0071] At block 416, processing logic provides, to the client device (e.g., client device 120) , the predicted target segmentation mask for presentation. In some embodiments, processing logic (e.g., measurement component 212) can determine one or more measurements (e.g., measurements of regions of interest, measurements of space between regions of interest) corresponding to the predicted target segmentation mask. For example, the measurement component 212 can determine a size of the substrate depicted in the predicted segmentation mask (e.g., obtain data corresponding to the size of the substrate from client device 120) to determine a ratio of the number of pixels that represent a measurement (e.g., 1 millimeter) in the predicted segmentation mask. The measurement component 212 can determine the amount of pixels in a region of interest and use the ratio to convert the number of pixels to the unit of measurement. In some embodiments, processing logic (e.g., the process control component 214) can update one or more process settings based on the measurements. For example, in some embodiments the process control component 214 can determine whether the measurements satisfy one or more measurement criteria 260, such as whether the regions of interest are within an acceptable size range and / or whether the regions of interest are within an acceptable distance range. Upon determining that the measurements are not within an acceptable size and / or distance range, the process control component 214 can determine one or more process settings (e.g., based on whether the measurements are below or above the size and / or distance threshold range) associated with a process recipe included in process recipe data 258. For example, if the size of a hole is determined to be larger than an acceptable size range, the process control component 214 can determine to reduce the length of an etch time setting associated with the process recipe used to process the substrate depicted in the target pattern image.
[0072] In some embodiments, the target pattern image can include more than one pattern. For example, a target pattern image can include holes and bitlines. A first predicted segmentation mask for the first pattern including holes can be generated with an input including the target pattern image and a reference pattern image and segmentation mask depicting holes. A second predicted segmentation mask for the second pattern including bitlines can be generated with an input including the target pattern image and a reference pattern image and segmentation mask depicting bitlines. In some embodiments, processing logic (e.g., mask component 210) can combine the two predicted segmentation masks to generate a third predicted segmentation mask that depicts both the first and second patterns. For example, the mask component 210 can extract the portions of each segmentation mask labeled as regions of interest and combine the extracted portions to generate a third segmentation mask where the objects in the first pattern and the objects in the second pattern are both labeled as regions of interest. The third segmentation mask can be provided for presentation on the client device 120, in some embodiments.
[0073] FIG. 5 depicts a block diagram of an illustrative computer system 500 operating in accordance with one or more aspects of the present disclosure. In alternative embodiments, the machine can be connected (e.g., networked) to other machines in a Local Area Network (LAN) , an intranet, an extranet, or the Internet. The machine can operate in the capacity of a server or a client machine in a client-server network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. The machine can be a personal computer (PC) , a tablet computer, a set-top box (STB) , a Personal Digital Assistant (PDA) , a cellular telephone, a web appliance, a server, a network router, switch or bridge, or any machine capable of executing a set of instructions (sequential or otherwise) that specify actions to be taken by that machine. Further, while only a single machine is illustrated, the term “machine” shall also be taken to include any collection of machines (e.g., computers) that individually or jointly execute a set (or multiple sets) of instructions to perform any one or more of the methodologies discussed herein. In embodiments, computing device 500 can correspond to predictive server 112 of FIG. 1 or another processing device of system 100. In other or similar embodiments, computing device 500 can correspond to computing system 150 of system 100.
[0074] The example computing device 500 includes a processing device 502, a main memory 504 (e.g., read-only memory (ROM) , flash memory, dynamic random access memory (DRAM) such as synchronous DRAM (SDRAM) , etc. ) , a static memory 506 (e.g., flash memory, static random access memory (SRAM) , etc. ) , and a secondary memory (e.g., a data storage device 528) , which communicate with each other via a bus 508.
[0075] Processing device 502 can represent one or more general-purpose processors such as a microprocessor, central processing unit, or the like. More particularly, the processing device 502 can be a complex instruction set computing (CISC) microprocessor, reduced instruction set computing (RISC) microprocessor, very long instruction word (VLIW) microprocessor, processor implementing other instruction sets, or processors implementing a combination of instruction sets. Processing device 502 can also be one or more special-purpose processing devices such as an application specific integrated circuit (ASIC) , a field programmable gate array (FPGA) , a digital signal processor (DSP) , network processor, or the like. Processing device 502 can also be or include a system on a chip (SoC) , programmable logic controller (PLC) , or other type of processing device. Processing device 502 is configured to execute the processing logic for performing operations and steps discussed herein.
[0076] The computing device 500 can further include a network interface device 522 for communicating with a network 564. The computing device 500 also can include a video display unit 510 (e.g., a liquid crystal display (LCD) or a cathode ray tube (CRT) ) , an alphanumeric input device 512 (e.g., a keyboard) , a cursor control device 514 (e.g., a mouse) , and a signal generation device 520 (e.g., a speaker) .
[0077] The data storage device 528 can include a machine-readable storage medium (or more specifically a non-transitory computer-readable storage medium) 524 on which is stored one or more sets of instructions 526 embodying any one or more of the methodologies or functions described herein. A non-transitory storage medium refers to a storage medium other than a carrier wave. The instructions 526 can also reside, completely or at least partially, within the main memory 504 and / or within the processing device 502 during execution thereof by the computer device 500, the main memory 504 and the processing device 502 also constituting computer-readable storage media.
[0078] The computer-readable storage medium 524 can also be used to store model 190 and data used to train model 190. The computer readable storage medium 524 can also store a software library containing methods that call model 190. While the computer-readable storage medium 524 is shown in an example embodiment to be a single medium, the term “computer-readable storage medium” should be taken to include a single medium or multiple media (e.g., a centralized or distributed database, and / or associated caches and servers) that store the one or more sets of instructions. The term “computer-readable storage medium” shall also be taken to include any medium that is capable of storing or encoding a set of instructions for execution by the machine and that cause the machine to perform any one or more of the methodologies of the present disclosure. The term “computer-readable storage medium” shall accordingly be taken to include, but not be limited to, solid-state memories, and optical and magnetic media.
[0079] The preceding description sets forth numerous specific details such as examples of specific systems, components, methods, and so forth in order to provide a good understanding of several embodiments of the present disclosure. It will be apparent to one skilled in the art, however, that at least some embodiments of the present disclosure can be practiced without these specific details. In other instances, well-known components or methods are not described in detail or are presented in simple block diagram format in order to avoid unnecessarily obscuring the present disclosure. Thus, the specific details set forth are merely exemplary. Particular implementations can vary from these exemplary details and still be contemplated to be within the scope of the present disclosure.
[0080] Reference throughout this specification to “one embodiment” or “an embodiment” means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. Thus, the appearances of the phrase “in one embodiment” or “in an embodiment” in various places throughout this specification are not necessarily all referring to the same embodiment. In addition, the term “or” is intended to mean an inclusive “or” rather than an exclusive “or. ” When the term “about” or “approximately” is used herein, this is intended to mean that the nominal value presented is precise within ±10%.
[0081] Although the operations of the methods herein are shown and described in a particular order, the order of operations of each method can be altered so that certain operations can be performed in an inverse order so that certain operations can be performed, at least in part, concurrently with other operations. In another embodiment, instructions or sub-operations of distinct operations can be in an intermittent and / or alternating manner.
[0082] It is understood that the above description is intended to be illustrative, and not restrictive. Many other embodiments will be apparent to those of skill in the art upon reading and understanding the above description. The scope of the disclosure should, therefore, be determined with reference to the appended claims, along with the full scope of equivalents to which such claims are entitled.
Claims
1.A method for substrate image segmentation in a manufacturing system, the method comprising:receiving, from a client device, a first input comprisinga first reference pattern image depicting a first pattern associated with a first substrate processed by manufacturing equipment,a first reference segmentation mask representing one or more first segmented regions of the first pattern depicted in the first reference pattern image, anda target pattern image depicting a second pattern, wherein the second pattern corresponds to the first pattern, and wherein the second pattern is associated with a second substrate processed by the manufacturing equipment;providing the first input to an artificial intelligence (AI) model trained using a plurality of pattern images depicting patterns of different pattern types;obtaining one or more first outputs of the AI model, wherein the one or more first outputs indicate a first predicted target segmentation mask representing one or more second segmented regions of the second pattern depicted in the target pattern image; andproviding, to the client device, the first predicted target segmentation mask for presentation.2.The method of claim 1 further comprising:determining one or more measurements of the one or more second segmented regions depicted in the first predicted target segmentation mask, wherein the one or more measurements are associated with a size and location of the one or more second segmented regions; andupdating one or more settings of a process recipe based on the one or more measurements.3.The method of claim 1, further comprising:receiving, from the client device, a second input comprisinga second reference pattern image depicting a third pattern associated with a third substrate, wherein a type of the third pattern is distinct from a type of the first pattern,a second reference segmentation mask representing one or more third segmented regions of the third pattern depicted in the second reference pattern image, andthe target pattern image further depicting a fourth pattern, wherein the fourth pattern corresponds to the third pattern, and wherein the fourth pattern is associated with the second substrate;providing the second input to the AI model;obtaining one or more second outputs of the AI model, wherein the one or more second outputs indicate a second predicted target segmentation mask representing one or more fourth segmented regions of the fourth pattern depicted in the target pattern image, and wherein the fourth segmented regions are distinct from the second segmented regions; andproviding, to the client device, the second predicted target segmentation mask for presentation.4.The method of claim 3, further comprising:generating a third target segmentation mask based on the first predicted target segmentation mask and the second predicted target segmentation mask, wherein the third target segmentation mask represents the one or more second segmented regions and the one or more fourth segmented regions; andproviding, to the client device, the third target segmentation mask for presentation.5.The method of claim 1, wherein the first reference segmentation mask is generated by at least one of threshold edge detection, template matching, or running an object detection model.6.The method of claim 1, wherein a type of the second pattern depicted in the target pattern image is non-periodic pattern type.7.The method of claim 1, wherein the first reference pattern image and the target pattern image depict a pattern of a pattern type that is distinct from a plurality of pattern types of patterns included in training data provided to train the AI model.8.The method of claim 1, wherein the first reference pattern image and the target pattern image are one of transmission electron microscopy (TEM) images or scanning transmission electron microscopy (STEM) images.9.A system for substrate image segmentation in a manufacturing system, the system comprising:a memory; anda set of one or more processing devices coupled to the memory, wherein the set of one or more processing devices is to:generate training data for training a machine learning model to predict a segmentation mask for pattern images depicting patterns on a substrate, wherein to generate the training data, the set of one or more processing devices is to:generate a training input comprising a reference pattern image depicting a first pattern, a reference segmentation mask associated with one or more first segmented regions of the first pattern, and a target pattern image depicting a second pattern of a historical substrate, wherein the second pattern corresponds to the first pattern;generate a target output for the training input, wherein the target output indicates a target segmentation mask associated with one or more second segmented regions of the target pattern image; andprovide the training data to train the machine learning model on (i) a set of training inputs comprising the training input and (ii) a set of target outputs comprising the target output.10.The system of claim 9, wherein the reference segmentation mask is generated by at least one of threshold edge detection, template matching, or running an object detection model.11.The system of claim 9, wherein the first pattern and the second pattern comprise at least one of holes, bitlines, scanning transmission electron microscopy (STEM) gate-all-around (GAA) transistor structures, transmission electron microscopy (TEM) GAA transistor structures, or blobs.12.The system of claim 9, wherein the reference pattern image and the target pattern image are one of TEM images or STEM images.13.The system of claim 9, wherein at least one of the reference pattern image or the target pattern image are preprocessed, wherein preprocessing comprises at least one of:rotating at least one of the reference pattern image or the target pattern image,adding noise to at least one of the reference pattern image or the target pattern image, oradjusting a brightness gradient of at least one of the reference pattern image or the target pattern image.14.The system of claim 13, wherein adding noise further comprises modifying an intensity value of one or more pixels in an image.15.The system of claim 9, wherein the set of one or more processing devices is further to:extract one or more first image descriptors from the reference pattern image;extract one or more second image descriptors from the reference segmentation mask;determine the first pattern based on the one or more first image descriptors and the one or more second image descriptors;extract one or more third image descriptors from the target pattern image;determine the second pattern based on the one or more third image descriptors and the first pattern; andidentify the target segmentation mask based on the second pattern and the third image descriptors.16.A non-transitory computer readable medium comprising instructions for substrate image segmentation in a manufacturing system that, when executed by a set of one or more processing devices, cause the set of one or more processing devices to:receive, from a client device, a first input comprisinga first reference pattern image depicting a first pattern associated with a first substrate processed by manufacturing equipment,a first reference segmentation mask representing one or more first segmented regions of the first pattern depicted in the first reference pattern image, anda target pattern image depicting a second pattern, wherein the second pattern corresponds to the first pattern, and wherein the second pattern is associated with a second substrate processed by the manufacturing equipment;provide the first input to an artificial intelligence (AI) model trained using a plurality of pattern images depicting patterns of different pattern types;obtain one or more first outputs of the AI model, wherein the one or more first outputs indicate a first predicted target segmentation mask representing one or more second segmented regions of the second pattern depicted in the target pattern image; andprovide, to the client device, the first predicted target segmentation mask for presentation.17.The non-transitory computer readable medium of claim 16, wherein the set of one or more processing devices is further to:determine one or more measurements of the one or more second segmented regions depicted in the first predicted target segmentation mask, wherein the one or more measurements are associated with a size and location of the one or more second segmented regions; andupdate one or more settings of a process recipe based on the one or more measurements.18.The non-transitory computer readable medium of claim 16, wherein the set of one or more processing devices is further to:receive, from the client device, a second input comprisinga second reference pattern image depicting a third pattern associated with a third substrate, wherein a type of the third pattern is distinct from a type of the first pattern,a second reference segmentation mask representing one or more third segmented regions of the third pattern depicted in the second reference pattern image, andthe target pattern image further depicting a fourth pattern, wherein the fourth pattern corresponds to the third pattern, and wherein the fourth pattern is associated with the second substrate;provide the second input to the AI model;obtain one or more second outputs of the AI model, wherein the one or more second outputs indicate a second predicted target segmentation mask representing one or more fourth segmented regions of the fourth pattern depicted in the target pattern image, and wherein the fourth segmented regions are distinct from the second segmented regions; andprovide, to the client device, the second predicted target segmentation mask for presentation.19.The non-transitory computer readable medium of claim 18, wherein the set of one or more processing devices is further to:generate a third target segmentation mask based on the first predicted target segmentation mask and the second predicted target segmentation mask, wherein the third target segmentation mask represents the one or more second segmented regions and the one or more fourth segmented regions; andprovide, to the client device, the third target segmentation mask for presentation.20.The non-transitory computer readable medium of claim 16, wherein the first reference segmentation mask is generated by at least one of threshold edge detection, template matching, or running an object detection model.
