Machine learning to determine equipment constant updates

Abstract

The method includes providing trace data associated with one or more substrate processing procedures as input to a first trained machine learning model. The input further includes equipment constants associated with the one or more substrate processing procedures. The input further includes trace data for a first processing chamber. The input further includes equipment constants for the first processing chamber. The method further includes obtaining a recommended update to the first equipment constant of the first processing chamber as output from the first trained machine learning model. The method further includes updating the first equipment constant of the first processing chamber in response to obtaining the output from the first trained machine learning model.

Description

[Technical Field]

[0001] The present disclosure relates to methods relating to machine learning models used to evaluate manufactured devices, such as semiconductor devices, and more particularly, to methods for generating and utilizing equipment constant updates to improve or standardize the performance of manufacturing chambers. [Background technology]

[0002] Products can be fabricated by performing one or more manufacturing processes using manufacturing equipment. For example, semiconductor manufacturing equipment can be used to fabricate substrates through a semiconductor manufacturing process. The products are fabricated to have specific characteristics suitable for a target application. Machine learning models are used in various process control and predictive functions associated with the manufacturing equipment. The machine learning models are trained using data associated with the manufacturing equipment. Changes can be made to the process recipe, process chamber, process steps, etc. to improve the characteristics of the fabricated products. Summary of the Invention

[0003] The following is a simplified summary of the present disclosure to provide a basic understanding of some aspects of the disclosure. This summary is not an extensive overview of the disclosure. It is not intended to identify key or critical elements of the disclosure, nor to delineate the scope of particular embodiments of the disclosure or the scope of the claims. Its sole purpose is to present some concepts of the disclosure in a simplified form as a prelude to the more detailed description that is presented later.

[0004] In one aspect of the present disclosure, a method includes providing trace data associated with one or more substrate processing procedures as input to a first trained machine learning model. The input further includes equipment constants associated with the one or more substrate processing procedures. The input further includes trace data for a first processing chamber. The input further includes equipment constants for the first processing chamber. The method further includes obtaining a recommended update to the first equipment constant of the first processing chamber as output from the first trained machine learning model. The method further includes updating the first equipment constant of the first processing chamber in response to obtaining the output from the first trained machine learning model.

[0005] In another aspect of the present disclosure, a system includes a memory and a processing device coupled to the memory. The processing device is for providing trace data associated with one or more substrate processing procedures that resulted in substrates satisfying one or more criteria as input to a first trained machine learning model. The processing device is further for providing equipment constants associated with the one or more substrate processing procedures as input to the first trained machine learning model. The processing device is further for providing trace data of a first processing chamber as input to the first trained machine learning model. The processing device is further for providing equipment constants of the first processing chamber as input to the first trained machine learning model. The processing device is further for obtaining a recommended update to a first equipment constant of the first processing chamber as output from the first trained machine learning model. The processing device is further for updating the first equipment constant of the first processing chamber in response to obtaining the output from the first trained machine learning model.

[0006] In another aspect of the present disclosure, a non-transitory machine-readable storage medium stores instructions that, when executed, cause a processing device to perform operations. The operations include providing trace data associated with one or more substrate processing procedures as input to a first trained machine learning model. The input further includes equipment constants associated with the one or more substrate processing procedures. The input further includes trace data for a first processing chamber. The input further includes equipment constants for the first processing chamber. The operations further include obtaining a recommended update to the first equipment constant of the first processing chamber as output from the first trained machine learning model. The operations further include updating the first equipment constant of the first processing chamber in response to obtaining the output from the first trained machine learning model.

[0007] The present disclosure is illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram illustrating an exemplary system architecture, according to some embodiments. [Figure 2A] FIG. 1 is a block diagram of an example dataset generator for creating datasets for one or more supervised models, according to some embodiments. [Figure 2B] FIG. 1 is a block diagram of an example dataset generator for creating datasets for one or more unsupervised models, according to some embodiments. [Figure 3] FIG. 1 is a block diagram illustrating a system for generating output data, according to some embodiments. [Figure 4A] 1 is a flow diagram of a method for generating a dataset for a machine learning model, according to some embodiments. [Figure 4B] 1 is a flow diagram of a method for updating equipment constants of a process chamber, according to some embodiments. [Figure 4C]1 is a flow diagram of a method for performing corrective actions associated with a processing chamber, according to some embodiments. [Figure 4D] 1 is a flow diagram of a method for adjusting equipment constants of a chamber in a group of chambers, according to some embodiments. [Figure 4E] 1 is a flow chart of an example method for performing corrective actions associated with one or more chambers in a group of chambers, according to some embodiments. [Figure 5A] FIG. 1 is a block diagram illustrating a system for performing operations associated with updating equipment constants of a process chamber, in accordance with some embodiments. [Figure 5B] FIG. 5 is a block diagram illustrating the operation of a corrective action recommendation model 530, according to some embodiments. [Figure 6] FIG. 1 is a block diagram illustrating a computer system according to some embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0009] Techniques related to improving the performance of manufacturing equipment by updating equipment constants are described herein. Manufacturing equipment is used to fabricate products such as substrates (e.g., wafers, semiconductors). The manufacturing equipment may include one or more fabrication or processing chambers for isolating the substrates from the environment. The properties of the fabricated substrates should meet target values ​​to facilitate a specific function. Manufacturing parameters are selected to fabricate substrates that meet the target property values. Many manufacturing parameters (e.g., hardware parameters, process parameters, etc.) contribute to the properties of the processed substrates.

[0010] As used herein, manufacturing parameters include process recipes and equipment constants. A process recipe includes parameters selected to produce process results, e.g., to enable processing of substrates characterized by one or more target characteristics. A process recipe may include parameters selected and / or adjusted based on product design, target output, target substrate metrology, etc. A process recipe may include parameters such as process temperature, process pressure, process gases, radio frequency (RF) radiation characteristics, plasma characteristics, etc. Equipment constants include parameters associated with the operation of manufacturing equipment. Equipment constants may include parameters corresponding to the implementation of a process recipe. Equipment constants may not be associated with a particular process, recipe, substrate design, target characteristics, etc. Equipment constants may be associated with manufacturing equipment, processing tools, process chambers, one or more components, etc. Equipment constants may include control settings, e.g., voltage or current settings applied to a component to achieve a target output (e.g., defined by the process recipe). Equipment constants may include operational settings, e.g., settings for the operation of a component not directly related to the process recipe output, such as the speed of a transfer robot, the voltage applied to a component to operate the component, etc.

[0011] Equipment constants can include independent values, such as the operating speed utilized for a pump or the allowable pressure given a vented or evacuated chamber. Equipment constants can include tables of values, such as a table relating input settings (e.g., process strategy inputs) to actions (e.g., voltages applied to components to achieve the inputs). Equipment constants can include functions, such as functions that can be utilized to calculate appropriate actions for target conditions (e.g., defined by the process strategy). Equipment constants can include calibration tables and / or calibration constants, such as adjustments to standard or factory settings for components. Equipment constants can include constants associated with one or more controllers. Equipment constants can include parameters associated with a proportional-integral-derivative (PID) controller, such as parameters that determine the effect on a controller output based on a controller input.

[0012] Equipment constants can form the basis for many operations of fabrication equipment, including parameters that control robot motion, chamber pressurization, chamber pumping, gas flow and mixing, temperature control, plasma generation, substrate clamping mechanisms, in-chamber metrology systems, and any other operations performed by the fabrication system.

[0013] A manufacturing system can control processing conditions (e.g., conditions within a process chamber) by specifying setpoints for characteristic values, receiving data from sensors located within the manufacturing chamber, and making adjustments to manufacturing equipment until the sensor readings match the setpoints. In some embodiments, the setpoints can be defined by a processing recipe, e.g., a process temperature, and a temperature sensor can be maintained at the defined temperature. In some embodiments, the setpoints can be defined by equipment constants, e.g., a target process temperature, and can reference a table of equipment constants that relate the target temperature (e.g., a temperature at a location not directly probed by a temperature sensor) to temperature readings by one or more sensors in the process chamber. Power provided to one or more heaters can be adjusted to maintain the setpoint temperature and the temperature of the sensor associated with the equipment constant table.

[0014] Process results may vary among multiple pieces of manufacturing equipment, tools, facilities, chambers, etc. A particular tool, chamber, etc. may produce acceptable product (meeting specific conditions or requirements), e.g., may produce substrates meeting target property value ranges more frequently than other equipment. Equipment may satisfy one or more conditions by frequently producing acceptable product for a range of times, e.g., after preventative or corrective maintenance, after seasoning, after installation, etc. Such equipment may be referred to as "golden" equipment. While the phrase "golden chamber" is used herein for brevity, golden equipment may include a golden tool, a set of golden equipment, golden manufacturing equipment, etc.

[0015] A golden trace can include trace data associated with one or more processing operations, such as operations that resulted in an acceptable product (e.g., a substrate meeting a target performance threshold, a target measurement value, etc.). The golden trace can be obtained from sensors in a golden chamber. The golden trace data can be collected during processing of the product by the golden chamber. A golden trace associated with one or more acceptable products (e.g., a substrate achieving a target characteristic value) and one or more golden chambers can represent sensor data measured during processing.

[0016] In conventional systems, operations of processing tools, chambers, equipment, etc. can be changed and / or adjusted to improve performance. Operations can be adjusted to enable production of products that meet target performance indicators (e.g., target measurements). Operations can be adjusted to increase the likelihood of producing products that meet target performance indicators. Operations can be adjusted to increase the efficiency of the manufacturing system, e.g., in terms of material consumption, usage time, energy consumption, etc. Operations can be adjusted to decrease the cost per acceptable product, including, e.g., reducing the cost of scrapping defective products, the cost of analyzing defective products, etc.

[0017] In conventional systems, operation of the manufacturing equipment can be adjusted to enable closer alignment between trace data collected from the manufacturing equipment and the golden trace data. For example, operation of a process chamber can be adjusted to more closely align the trace data with the golden trace data. Operation of the manufacturing equipment can be adjusted to enable closer similarity between the manufacturing equipment and the golden chamber.

[0018] In conventional systems, changes to the processing strategy can be made to adjust the operation of the manufacturing equipment. The processing strategy can be adjusted to improve product characteristics. The processing strategy can be adjusted to improve trace data, e.g., to make the trace data more similar to golden trace data. The processing strategy can be adjusted to improve manufacturing equipment, e.g., to make the performance of a process chamber more similar to a golden chamber.

[0019] These conventional methods have several drawbacks. Adjusting the performance of manufacturing equipment (e.g., process chambers) to improve product characteristics is an inefficient process. Updates may be made (e.g., to processing recipe parameters), substrates may be processed according to the updates, and metrology operations may be performed on the substrates. The relationship between recipe inputs and product outputs may be nonlinear and not one-to-one (e.g., one recipe component may affect multiple substrate characteristics). Manufacturing equipment performance may also be related to aging or failed components and may be improved through preventative or corrective maintenance. Determining appropriate actions to improve product performance can be difficult and increase the time to correct equipment performance. This can result in increased equipment downtime, reduced productivity, reduced yield, etc. Determining corrective actions based on product performance may include increased costs for performing measurements, for example, with stand-alone metrology equipment. Determining corrective actions based on product performance may include performing many processing runs (e.g., processing many products), increasing expenditures of energy, materials, and time, increasing costs associated with discarding defective products, increasing component wear, aging, and / or misalignment, reducing equipment productivity relative to non-productive time (e.g., reducing chamber green time), etc.

[0020] Adjusting a processing strategy to target alignment (e.g., closer similarity) of the trace data and the golden trace data also has drawbacks. The trace data is affected by many aspects of the manufacturing process. The trace data is affected by the processing strategy. The trace data is affected by equipment health, e.g., component aging, drift, etc., which may affect chamber performance and be reflected in the trace data. The trace data may also be affected by equipment constants. Adjusting a processing strategy to target alignment of the trace data and the golden trace data utilizes some of the available adjustable parameters (e.g., process knobs) to improve manufacturing equipment performance. Adjusting a processing strategy may be less efficient than other trace data alignment methods. Adjusting a processing strategy may result in increased energy use, increased material use, decreased processing efficiency, etc., compared to other trace data alignment methods. Aligning trace data by adjusting a processing strategy may cause processing equipment components to be operated more frequently, more roughly, in a manner that imposes more stress on the components, etc., compared to matching trace data via another method. For example, a temperature trace may be matched by increasing the power supplied to a heater, but increasing the power supplied to the heater may create more severe conditions for one or more components of the process chamber, reduce the lifespan of one or more components of the process chamber, increase the drift and / or aging of one or more components of the process chamber, increase overall energy use of the process chamber, etc. Reducing the lifespan of one or more components of the process chamber may result in increased frequency of reactive and / or preventive maintenance, increased chamber downtime, decreased chamber green time, etc. Reducing the lifespan of one or more components may result in increased expensive unplanned downtime, increased costs associated with the component, including component costs, expedited shipping costs, etc.

[0021] Aspects of the present disclosure may address one or more of these shortcomings associated with conventional solutions. Equipment constants of manufacturing equipment may be adjusted. Generally, the description herein refers to equipment constants of a chamber (e.g., a golden chamber) being utilized to update equipment constants of another chamber. Embodiments utilizing or updating equipment constants of a tool, a facility, a group of tools (e.g., a group of similar tools), a group of chambers (e.g., several similarly functioning chambers of a tool), etc. are also applicable. The chamber being updated may be part of the same tool as the golden chamber or a different tool. The chamber being updated may be in the same facility as the golden chamber or in a different facility.

[0022] Equipment constants are distinct from process recipe parameters. Process recipe parameters are generally related to setpoint values ​​for one or more characteristics during processing. Process recipe parameters can include characteristic value setpoints (e.g., target temperature, target pressure, etc.), time (e.g., duration to hold target temperature), setpoint changes over time (e.g., temperature increase), etc. Equipment constants are generally settings that affect the operation of manufacturing equipment. Equipment constants can include non-process setpoints, such as transfer robot speed, gas flows for venting locks, the number of processing operations between which automatic chamber cleaning and / or seasoning operations are performed, leak check result boundaries that generate alerts, etc. Equipment constants can include settings that are relevant during processing, such as chamber pumping system pressure boundaries that trigger alerts, gas control parameters for delivering process gases to the process chamber, etc. Equipment constants can include, for example, calibration tables for relating setpoint values ​​to control signals.

[0023] In some embodiments, one or more equipment constants of a golden chamber (or, where appropriate, a golden tool, golden facility, etc.) can be applied to a process chamber (e.g., a lower-performing chamber). In some embodiments, the equipment constants can be categorized by the risk incurred by updating the equipment constant. For example, an equipment constant that is unlikely to have a significant adverse effect on the process if changed (e.g., transfer robot speed, inert gas flow for venting locks, etc.) can be considered a low-risk equipment constant. An equipment constant that is likely to have a significant adverse effect on the process if changed (e.g., radio frequency (RF) control, plasma generation control, process gas mixing and delivery, pressure and / or temperature control, etc.) can be considered a high-risk. The risk of an equipment constant can be assessed by subject matter experts, statistical models, physics-based models, machine learning models, etc. Based on the assessed risk, action can be taken to update one or more equipment constants of the process chamber.

[0024] In some embodiments, one or more machine learning models may be utilized to determine corrective updates to equipment constants. The machine learning model may be utilized to evaluate the impact of one or more equipment constants, e.g., impact on product performance, impact on trace data, impact on product variability, etc. The machine learning model may be utilized to determine a target equipment constant from input data including equipment constants and an indication of equipment performance. The machine learning model may be utilized to determine outlier equipment constants, e.g., an outlier calibration table. The machine learning model may be utilized to determine the optimal time to perform maintenance on manufacturing equipment, e.g., to perform a manual calibration. The machine learning model may cause an action associated with updating the equipment constant to be performed, e.g., the machine learning model may schedule, initiate, etc. a calibration action. In some embodiments, a statistical model, a physics-based model, or another type of model may be utilized instead of a machine learning model to perform one or more of these actions.

[0025] In some embodiments, a machine learning model can be utilized to generate a golden trace. The machine learning model can be provided with trace data as a data input. The machine learning model can further be provided with product metrology data associated with the trace data as a data input. The machine learning model can be provided with data from one or more golden chambers. The machine learning model can be configured to generate one or more golden traces. In some embodiments, the machine learning model can generate lower and upper limits for the golden trace. In some embodiments, the machine learning model can generate golden traces associated with multiple chamber sensors, multiple measured conditions (e.g., temperature, pressure, etc.), etc.

[0026] In some embodiments, the machine learning model can generate recommended updates to one or more equipment constants based on the golden trace. The machine learning model can receive one or more golden traces as input. The golden trace can be generated by another machine learning model. The golden trace can include upper and lower limits of the golden trace. The golden trace can include traces associated with multiple process conditions, multiple sensors, etc. The machine learning model can further receive equipment constants from one or more process chambers, e.g., a chamber that is not a golden chamber, a chamber that does not produce golden trace data, a poorly performing chamber, etc. The machine learning model can further receive trace data, e.g., associated with the same process chamber as the received equipment constants. The machine learning model can further receive metrology data, e.g., associated with the same process chamber as the received equipment constants. The machine learning model can generate recommended changes to the equipment constants of one or more chambers as output. The machine learning model can generate a schedule for equipment constant updates, e.g., a recommended order for updating equipment constants to facilitate monitoring the impact of changes to the equipment constants. The machine learning model can generate one or more recommendations for maintenance actions (e.g., calibration, cleaning, seasoning, etc.). The machine learning model can initiate one or more maintenance actions. The machine learning model can perform optimization actions and recommend updates to equipment constants targeting, for example, improved chamber efficiency, improved chamber performance, improved energy efficiency, improved material expenditures, etc.

[0027] Aspects of the present disclosure provide technical advantages over conventional solutions: By updating equipment constants, a larger process variation space for updating processing recipes can be accessed; Chamber tuning can be performed to improve uniformity between different chambers; Chamber uniformity and / or performance can be improved across many different processing recipes, processing operations, processing types, etc. Equipment constant updates can be made based on information from a variety of sources, e.g., subject matter experts, physics-based models, statistical models, machine learning models, optimization algorithms, etc. Equipment constant updates can be made in the context of specific results, e.g., taking into account the relationship between one or more equipment constants and one or more substrate performance metrics.

[0028] In one aspect of the present disclosure, a method includes providing trace data associated with one or more substrate processing procedures as input to a first trained machine learning model. The input further includes equipment constants associated with the one or more substrate processing procedures. The input further includes trace data for a first processing chamber. The input further includes equipment constants for the first processing chamber. The method further includes obtaining a recommended update to the first equipment constant of the first processing chamber as output from the first trained machine learning model. The method further includes updating the first equipment constant of the first processing chamber in response to obtaining the output from the first trained machine learning model.

[0029] In another aspect of the present disclosure, a system includes a memory and a processing device coupled to the memory. The processing device is for providing trace data associated with one or more substrate processing procedures that resulted in substrates satisfying one or more criteria as input to a first trained machine learning model. The processing device is further for providing equipment constants associated with the one or more substrate processing procedures as input to the first trained machine learning model. The processing device is further for providing trace data of a first processing chamber as input to the first trained machine learning model. The processing device is further for providing equipment constants of the first processing chamber as input to the first trained machine learning model. The processing device is further for obtaining a recommended update to a first equipment constant of the first processing chamber as output from the first trained machine learning model. The processing device is further for updating the first equipment constant of the first processing chamber in response to obtaining the output from the first trained machine learning model.

[0030] In another aspect of the present disclosure, a non-transitory machine-readable storage medium stores instructions that, when executed, cause a processing device to perform operations. The operations include providing trace data associated with one or more substrate processing procedures as input to a first trained machine learning model. The input further includes equipment constants associated with the one or more substrate processing procedures. The input further includes trace data for a first processing chamber. The input further includes equipment constants for the first processing chamber. The operations further include obtaining a recommended update to the first equipment constant of the first processing chamber as output from the first trained machine learning model. The operations further include updating the first equipment constant of the first processing chamber in response to obtaining the output from the first trained machine learning model.

[0031] 1 is a block diagram illustrating an example system 100 (an example system architecture) according to some embodiments. System 100 includes client devices 120, manufacturing equipment 124, sensors 126, measurement equipment 128, a prediction server 112, and a data store 140. Prediction server 112 may be part of a prediction system 110. Prediction system 110 may further include server machines 170 and 180.

[0032] The sensors 126 can provide sensor data 142 associated with the manufacturing equipment 124 (e.g., associated with production by the manufacturing equipment 124 and the corresponding product, such as a substrate). The sensor data 142 can be used to ascertain the health of the equipment and / or the health of the product (e.g., product quality). The sensor data 142 can include trace data, e.g., data generated by the sensors multiple times over the duration of a process. The trace data can include values ​​associated with the time at which the associated measurement was performed. The manufacturing equipment 124 can manufacture the product according to a recipe or by performing runs over a period of time. In some embodiments, the sensor data 142 can include one or more values ​​of optical sensor data, spectral data, temperature (e.g., heater temperature), spacing (SP), pressure, high frequency radio frequency (HFRF), radio frequency (RF) match voltage, RF match current, RF match capacitor position, electrostatic chuck (ESC) voltage, actuator position, robot position, current, flow rate, power, voltage, etc. The sensor data 142 can include historical sensor data 144 and current sensor data 146. The current sensor data 146 may be associated with a product currently being processed (e.g., a substrate, a semiconductor wafer, etc.), a product recently processed, the number of products recently processed, etc. The current sensor data 146 may be used as input to a trained machine learning model, e.g., to generate prediction data 168. The historical sensor data 144 may include data stored in association with previously produced products. The historical sensor data 144 may be used to train a machine learning model, e.g., model 190. The historical sensor data 144 and / or the current sensor data 146 may include attribute data, e.g., manufacturing equipment ID or design, sensor ID, type, and / or location label, label of the manufacturing equipment status such as current faults, service life, etc.

[0033] The sensor data 142 may be associated with or indicative of manufacturing parameters, such as hardware parameters of the manufacturing equipment 124 (e.g., hardware settings or installed components, e.g., size, type, etc.) or process parameters of the manufacturing equipment 124 (e.g., heater settings, gas flows, etc.).

[0034] Data associated with some hardware parameters and / or process parameters can instead or additionally be stored as manufacturing parameters 150, which can include past manufacturing parameters (e.g., associated with past processing runs) and current manufacturing parameters. Manufacturing parameters 150 can indicate input settings to a manufacturing device (e.g., heater power, gas flow, etc.). Manufacturing parameters 150 can be or include components of a processing recipe to be executed by, for example, manufacturing equipment 124. Sensor data 142 and / or manufacturing parameters 150 can be provided while manufacturing equipment 124 is performing a manufacturing process (e.g., equipment readings while processing a product). Sensor data 142 can be different for each product (e.g., each substrate). Manufacturing parameters 150 can be different for each product design, each recipe, etc. Manufacturing parameters 150 can be customized based on manufacturing equipment 124, for example, customized for the performance of a particular process chamber. Substrates produced by manufacturing equipment 124 can have characteristics (such as film thickness, film strain, etc.) measured by metrology equipment 128, for example, by stand-alone metrology equipment. Metrology data 160 can be a component of data store 140. Metrology data 160 can include historical metrology data 164 (e.g., metrology data associated with previously processed products).

[0035] The manufacturing parameters 150 may include hardware parameters (e.g., information indicating which components are installed in the manufacturing equipment 124, component replacements, component age, software versions or updates, etc.) and / or process parameters (e.g., temperature, pressure, flow rate, flow velocity, current, voltage, gas flow, lift speed, etc.).

[0036] In some embodiments, metrology data 160 can be provided without the use of stand-alone metrology equipment, such as in-situ metrology data (e.g., metrology or proxies for metrology collected during processing), integrated metrology data (e.g., metrology or proxies for metrology collected while a product is in a chamber or under vacuum but not during a processing operation), or in-line metrology data (e.g., data collected after a substrate is removed from vacuum). Metrology data 160 can include current metrology data (e.g., metrology data associated with a currently or recently processed product), historical metrology data, etc. Historical metrology data can be utilized in training one or more machine learning models.

[0037] The equipment constants 152 may include settings, parameters, calibrations, etc. associated with the manufacturing equipment 124. The equipment constants may be stored in association with a process chamber, a process tool, process equipment, groups thereof, etc. The equipment constants may be provided to train a model. The equipment constants may be provided as inputs to one or more models. The equipment constants (e.g., equipment constant updates) may be received as outputs from one or more models.

[0038] In some embodiments, the sensor data 142, the metrology data 160, the manufacturing parameters 150, and / or the equipment constants 152 can be processed (e.g., by the client device 120 and / or the prediction server 112). Processing the data can include generating features. In some embodiments, the features are patterns (e.g., slope, width, height, peaks, etc.) within the sensor data 142, the metrology data 160, and / or the manufacturing parameters 150, or combinations of values ​​(e.g., power derived from voltage and current, etc.) from the sensor data 142, the metrology data 160, the equipment constants 152, and / or the manufacturing parameters. The data can include features, which can be used by the prediction component 114 to perform signal processing and / or to obtain prediction data 168 for taking corrective actions.

[0039] Each instance (e.g., each set) of sensor data 142 may correspond to a product (e.g., a substrate), a manufacturing device, a type of substrate produced by the manufacturing device, etc. Each instance of metrology data 160 and manufacturing parameters 150 may similarly correspond to a product, a manufacturing device, a type of substrate produced by the manufacturing device, etc. The data store may further store information relating different sets of data types, such as information indicating that a set of sensor data, a set of metrology data, a set of equipment constants, and a set of manufacturing parameters are all associated with the same product, manufacturing device, type of substrate, etc.

[0040] The golden trace data 162 can be or include sensor data designated as golden data. The golden trace data 162 can be generated or selected by a model (e.g., selected from the sensor data 142). The golden trace data 162 can be generated or selected by a machine learning model. The golden trace data 162 can include data associated with one or more measured characteristics, one or more sensors, etc. The golden trace data 162 can include upper and lower limits, e.g., guard bands, of the golden trace.

[0041] The predictive data 168 may include recommended corrective actions. The predictive data 168 may include equipment constant updates for one or more process chambers. The predictive data 168 may include scheduled updates. The predictive data 168 may include scheduled maintenance, such as scheduled and recommended preventive or corrective maintenance. The predictive data 168 may include scheduled automatic maintenance, such as component calibration, process chamber cleaning or seasoning operations, etc. In some embodiments, the predictive system 110 may generate the predictive data 168 using supervised machine learning (e.g., the predictive data 168 includes output from a machine learning model trained using labeled data, such as sensor data labeled with metrology data). In some embodiments, the predictive system 110 may generate the predictive data 168 using unsupervised machine learning (e.g., the predictive data 168 includes output from a machine learning model trained using unlabeled data, where the output may include clustering results, principal component analysis, anomaly detection, etc.). In some embodiments, the prediction system 110 can generate the prediction data 168 using semi-supervised learning (e.g., the training data can include a mix of labeled and unlabeled data, etc.).

[0042] The data store 140 may further store synthetic data. Synthetic data may be data associated with one or more types of data stored in the data store 140 (e.g., sensor data, manufacturing parameters, equipment constants, metrology data, etc.). Synthetic data may be data not generated by manufacturing equipment or sensors, data not associated with the processing of one or more substrates, etc. Synthetic data may be utilized to replace and / or augment data collected by / from the manufacturing system 100. Synthetic data may be generated by a user, e.g., a subject matter expert. Synthetic data may be generated by a model, e.g., a statistical model, a machine learning model, a recurrent neural network, etc. Synthetic data may be provided as input to a model, training input to a model, target output to a model, etc. Synthetic data may be utilized to augment a data type when available data is insufficient (e.g., for training a machine learning model), to fill gaps in trace data (e.g., to fill in unsatisfactory portions of trace data between otherwise satisfactory trace data), etc.

[0043] Client device 120, manufacturing equipment 124, sensors 126, measurement equipment 128, prediction server 112, data store 140, server machine 170, and server machine 180 can be coupled to one another via network 130 to generate predictive data 168 for implementing corrective actions. In some embodiments, network 130 can provide access to cloud-based services. Operations performed by client device 120, prediction system 110, data store 140, etc. can be performed by a virtual cloud-based device.

[0044] In some embodiments, network 130 is a public network that provides client devices 120 with access to prediction server 112, data store 140, and other publicly available computing devices. In some embodiments, network 130 is a private network that provides client devices 120 with access to manufacturing equipment 124, sensors 126, measurement equipment 128, data store 140, and other privately available computing devices. Network 130 may include one or more wide area networks (WANs), local area networks (LANs), wired networks (e.g., Ethernet networks), wireless networks (e.g., 802.11 networks or Wi-Fi networks), cellular networks (e.g., Long Term Evolution (LTE) networks), routers, hubs, switches, server computers, cloud computing networks, and / or combinations thereof.

[0045] The client device 120 may include computing devices such as a personal computer (PC), a laptop, a mobile phone, a smartphone, a tablet computer, a netbook computer, a network-connected television ("smart TV"), a network-connected media player (e.g., a Blu-ray player), a set-top box, an over-the-top (OTT) streaming device, an operator box, etc. The client device 120 may include a corrective action component 122. The corrective action component 122 may receive user input (e.g., via a graphical user interface (GUI) displayed via the client device 120) of instructions associated with the manufacturing equipment 124. In some embodiments, the corrective action component 122 transmits instructions to the prediction system 110, receives output (e.g., prediction data 168) from the prediction system 110, determines corrective actions based on the output, and causes the corrective actions to be implemented.

[0046] In some embodiments, the corrective action component 122 obtains sensor data 142 (e.g., current sensor data 146) associated with the manufacturing equipment 124 (e.g., from the data store 140, etc.) and provides the sensor data 142 (e.g., current sensor data 146) associated with the manufacturing equipment 124 to the predictive system 110. In some embodiments, the corrective action component 122 can obtain equipment constants 152 (e.g., updates to the equipment constants recommended by the predictive component 114) from the data store 140 and provide the equipment constants 152 to the manufacturing equipment 124 to update the equipment constants of the manufacturing equipment 124.

[0047] In some embodiments, the corrective action component 122 receives corrective action instructions from the predictive system 110 and causes the corrective action to be implemented. Each client device 120 may include an operating system that enables a user to one or more of create, view, or edit data (e.g., instructions associated with the manufacturing equipment 124, corrective actions associated with the manufacturing equipment 124, etc.).

[0048] In some embodiments, metrology data 160 (e.g., historical metrology data) corresponds to historical characteristic data of a product (e.g., a product processed using manufacturing parameters associated with historical sensor data 144 and historical manufacturing parameters of manufacturing parameters 150), and predictive data 168 is associated with predicted characteristic data (e.g., of a product to be or was manufactured under conditions recorded by current sensor data 146 and / or current manufacturing parameters). In some embodiments, predictive data 168 is or includes predicted metrology data (e.g., virtual metrology data, virtual composite microscope images) of a product to be or was manufactured under conditions recorded as current sensor data 146, current measurement data, current metrology data, and / or current manufacturing parameters. In some embodiments, predictive data 168 is or includes indications of anomalies (e.g., an abnormal product, an abnormal component, an abnormal manufacturing equipment 124, abnormal energy usage, abnormal equipment constants, etc.), and optionally one or more causes of those anomalies. In some embodiments, the predictive data 168 is an indication of a change or drift over time in some component, such as manufacturing equipment 124, sensor 126, metrology equipment 128, etc. In some embodiments, the predictive data 168 is an indication of the end of life of a component, such as manufacturing equipment 124, sensor 126, metrology equipment 128, etc. In some embodiments, the predictive data 168 is an indication of the progress of a processing operation being performed, for example, as used for process control.

[0049] Running a manufacturing process that results in a defective product can be costly in terms of time, energy, product, components, manufacturing equipment 124, the cost of identifying defects and discarding the defective product, etc. By inputting sensor data 142 (e.g., manufacturing parameters that are in place or will be used to manufacture the product) into the predictive system 110, receiving output of predicted data 168, and performing corrective actions based on the predicted data 168, the system 100 can have the technical advantage of avoiding the costs of producing, identifying, and discarding defective products. By updating the equipment constants of the manufacturing equipment, equipment performance can be improved, standardized, and / or made more consistent, resulting in more frequent production of products that meet target performance metrics and avoiding the costs associated with producing defective products.

[0050] Running a manufacturing process that results in a failure of a component of manufacturing equipment 124 can be costly in terms of downtime, product damage, equipment damage, expedited ordering of replacement components, etc. By inputting sensor data 142 (e.g., manufacturing parameters that are in place or will be used to manufacture a product), metrology data, measurement data, etc., receiving output of predictive data 168, and performing corrective action (e.g., predicted operational maintenance of a component, e.g., replacement, treatment, cleaning, etc.) based on the predictive data 168, system 100 can have the technical advantage of avoiding the costs of one or more of unexpected component failures, unscheduled downtime, lost productivity, unexpected equipment failures, product waste, etc. Monitoring the performance of components, e.g., manufacturing equipment 124, sensors 126, measurement devices 128, etc., over time can provide an indication of a degrading component. Monitoring equipment constants 152 over time can provide an indication of a degrading component, for example, when a recommended equipment constant falls outside a control limit, statistical limit, guard band, etc.

[0051] The manufacturing parameters may be suboptimal for producing the product, which may result in costly consequences such as increased resource (e.g., energy, coolant, gas, etc.) consumption, increased amount of time to produce the product, increased component failures, increased quantity of defective products, etc. By inputting measurement instructions into the prediction system 110, receiving the output of the prediction data 168, and performing corrective action such as updating the equipment constants (e.g., setting optimal equipment constants) of the manufacturing equipment 124 (e.g., based on the prediction data 168), the system 100 may have the technical advantage of using improved equipment constants (e.g., process equipment constants, non-process equipment constants, calibration tables, etc.) to avoid the costly consequences of suboptimal equipment performance.

[0052] The manufacturing parameters may be suboptimal for reducing the environmental impact of the manufacturing process. For example, a semiconductor manufacturing process utilizing a first set of manufacturing parameters may generate additional pollutants, waste, carbon dioxide, and / or other greenhouse gases, etc., compared to performing the process utilizing a second set of manufacturing parameters. The manufacturing process may be less sustainable (e.g., due to the use of a combination of manufacturing parameters) than another process that produces similar results. By inputting indications of manufacturing equipment performance into the prediction system 110, receiving outputs associated with corrective actions from the prediction system 110, and implementing the corrective actions, the system 100 may have the technical advantage of using improved equipment constants to reduce the environmental impact of the manufacturing process.

[0053] The manufacturing parameters may be suboptimal for the production rate of one or more products. Utilizing a particular set of parameters or particular ranges of manufacturing parameters may result in faster production, faster processing, faster processing within acceptable defect limits, etc. By inputting manufacturing equipment performance indications into the prediction system 110, receiving output associated with corrective actions from the prediction system 110, and implementing the corrective actions, the system 100 may have the technical advantage of reducing the time to process products. The system 100 may have the technical advantage of using improved equipment constants to reduce processing time per substrate, for example, within target defect limits and / or other performance metrics.

[0054] The corrective action may be associated with one or more of computational process control (CPC), statistical process control (SPC) (e.g., SPC of electronic components to determine the process under control, SPC to predict the useful life of a component, SPC to compare with a 3σ graph, etc.), advanced process control (APC), model-based process control, preventative operational maintenance, design optimization, manufacturing parameter updates, manufacturing recipe updates, equipment constant updates, feedback control, feedforward control, machine learning corrections, etc.

[0055] In some embodiments, the corrective action includes providing an alert (e.g., an alert indicating a recommended action such as scheduled maintenance or calibration, an alarm to stop or not run a manufacturing process when the predictive data 168 indicates a predicted anomaly, such as an anomaly in a product, component, or manufacturing equipment 124, etc.). In some embodiments, the execution of the corrective action includes causing an update to one or more equipment constants. In some embodiments, the execution of the corrective action includes retraining a machine learning model associated with the manufacturing equipment 124. In some embodiments, the execution of the corrective action includes training a new machine learning model associated with the manufacturing equipment 124.

[0056] In some embodiments, the corrective action includes initiating preventative operational maintenance (e.g., replacing, treating, cleaning, etc., components of the manufacturing equipment 124). In some embodiments, the corrective action includes initiating design optimization (e.g., updating equipment constants, manufacturing processes, manufacturing equipment 124, etc. for an optimized product). In some embodiments, the corrective action includes updating a recipe (e.g., changing when a manufacturing subsystem enters a sleep or active mode, changing set points for various characteristic values, etc.). In some embodiments, the corrective action includes scheduling or performing calibration, cleaning, and / or seasoning operations of the process system.

[0057] Prediction server 112, server machine 170, and server machine 180 may each include one or more computing devices such as a rack-mounted server, a router computer, a server computer, a personal computer, a mainframe computer, a laptop computer, a tablet computer, a desktop computer, a graphics processing unit (GPU), an accelerator, an application-specific integrated circuit (ASIC) (e.g., a tensor processing unit (TPU)), etc. Operations of prediction server 112, server machine 170, server machine 180, data store 140, etc. may be performed by a cloud computing service, a cloud data storage service, etc.

[0058] The prediction server 112 may include a prediction component 114. In some embodiments, the prediction component 114 may receive current sensor data 146 for performing corrective actions associated with the manufacturing equipment 124. In some embodiments, the corrective actions may include updating one or more equipment constants. The prediction component 114 may further receive additional data, such as current manufacturing parameters (e.g., received from the client device 120, retrieved from the data store 140), metrology data 160, equipment constants 152, golden trace data 162, etc., to generate output (e.g., predicted data 168) for performing corrective actions associated with the manufacturing equipment 124. In some embodiments, the prediction component 114 may use one or more trained machine learning models 190 to determine the output for performing the corrective actions based on the current data. In some embodiments, the predicted data 168 may be utilized as input to a machine learning model. A machine learning model may receive as input data output by another machine learning model.

[0059] Manufacturing equipment 124 can be associated with one or more machine learning models, such as model 190. The machine learning models associated with manufacturing equipment 124 can perform many tasks, including process control, classification, performance prediction, process updates, etc. Model 190 can be trained using data associated with manufacturing equipment 124 or products processed by manufacturing equipment 124, such as sensor data 142 (e.g., collected by sensors 126), manufacturing parameters 150 (e.g., associated with process control of manufacturing equipment 124), metrology data 160 (e.g., generated by metrology equipment 128), equipment constants 152, etc.

[0060] One type of machine learning model that can be used to perform some or all of the above tasks is an artificial neural network, such as a deep neural network. An artificial neural network generally includes a feature representation component with a classifier or recurrent layer that maps features to a desired output space. A convolutional neural network (CNN), for example, hosts multiple layers of convolutional filters. Lower layers can perform pooling to address nonlinearities, and a multi-layer perceptron is typically added on top to map upper layer features extracted by the convolutional layers to a decision (e.g., a classification output).

[0061] A recurrent neural network (RNN) is another type of machine learning model. A recurrent neural network model is designed to interpret a sequence of inputs where the inputs are inherently related to each other, for example, time trace data, serial data, etc. The output of the RNN's perceptron is fed back as input to the perceptron to generate the next output.

[0062] Deep learning is a type of machine learning algorithm that uses a series of multiple layers of nonlinear processing units for feature extraction and transformation. Each successive layer uses the output from the previous layer as input. Deep neural networks can learn in a supervised (e.g., classification) and / or unsupervised (e.g., pattern analysis) manner. Deep neural networks include a hierarchical structure of layers, with different layers learning different levels of representation corresponding to different levels of abstraction. In deep learning, each level learns to transform its input data into a slightly more abstract, synthetic representation. For example, in an image recognition application, the raw input may be a matrix of pixels; the first representation layer may abstract the pixels and encode edges; the second layer may construct and encode edge placement; the third layer may encode higher-level shapes (e.g., teeth, lips, gums, etc.); and the fourth layer may recognize trajectories. In particular, the deep learning process can learn which features are best located at which levels alone. The "deep" in "deep learning" refers to the number of layers through which data is transformed. More precisely, deep learning systems have substantial belief assignment path (CAP) depth. A CAP is a chain of transformations from input to output. The CAP describes the potential causal relationships between input and output. For feedforward neural networks, the CAP depth can be the depth of the network, which can be the number of hidden layers plus one. For recurrent neural networks, a signal can propagate through a layer more than once, and the CAP depth is potentially unlimited.

[0063] In some embodiments, multiple machine learning models can be utilized by the system 100. A first machine learning model can be utilized (e.g., by the prediction system 110) to generate golden trace data associated with a manufacturing process, a manufacturing system, a product design, a recipe, etc. The first machine learning model can be configured to receive input metrology data and tool traces (e.g., trace sensor data). The first machine learning model can be configured to output one or more golden traces. The golden trace output can be data measured by a golden chamber. The golden trace can be associated with a product that meets a target performance index. The golden trace can include an ideal or best trace. The golden trace can include upper and lower limits, upper and lower guard bands, control limits, an average trace, a median trace, etc. The golden trace can be selected, generated, etc. for a target result. For example, a manufacturing process may have several targets, such as a target energy consumption, a target environmental impact, a target processing rate, and a target performance level (e.g., a target level of defective products). The golden trace can be selected to optimize one metric, balance one or more metrics, optimize one or more metrics while keeping other metrics within target ranges, etc.

[0064] A process may include a golden trace guard band. The guard band may represent a limit on how far the trace may deviate before some action is taken. For example, trace data for a process may be considered acceptable if a data point falls within a range defined by the guard band, if a target portion of the data point falls within the guard band, if a target portion of the data point falls within a target value range of the guard band, etc. In some embodiments, the guard band may be generated statistically, e.g., by generating a synthetic trace that includes a target portion of the input data. In some embodiments, the guard band may be generated by a statistical model, a machine learning model, etc. In some embodiments, the guard band may be generated from multiple processing runs, processing chambers, etc. In some embodiments, minimum and maximum data values ​​from multiple runs (e.g., multiple golden traces) may define minimum and maximum data values ​​for the guard band. The golden trace used to define the guard band may be generated by a golden chamber, golden equipment, golden tool, etc.

[0065] In some embodiments, the golden trace is provided to a second machine learning model. The second machine learning model can be configured to recommend a corrective action, for example, to the client device 120, a user, etc. The second machine learning model can be configured to implement the corrective action. The second machine learning model can be configured to recommend and / or implement a schedule for the corrective action. The second machine learning model can be configured to recommend and / or implement an equipment constant update. The second machine learning model can receive metrology data (e.g., of a process chamber that may include the equipment constants to be updated) as an additional input. The second machine learning model can receive equipment constants (e.g., of the golden chamber, the chamber associated with the golden trace data, the chamber that may include the equipment constants to be updated, etc.) as an additional input. The second machine learning model can receive trace data (e.g., of the chamber that includes the equipment constants to be updated) as an additional input. The second machine learning model can be configured to improve the operation of one or more process chambers. The second machine learning model can improve the operation of one or more process chambers by recommending and / or implementing updates to equipment constants that improve substrate metrology, e.g., that increase the likelihood that substrates processed in the process chambers will meet target performance indicators. The second machine learning model can improve the operation of one or more process chambers by recommending and / or implementing updates to equipment constants that improve the process chamber trace data, for example, that increase the similarity between the golden trace data and the process chamber trace data.

[0066] In some embodiments, the prediction component 114 receives one or more types of data, performs signal processing to decompose the data into sets of current data, provides these sets of data as inputs to a trained model 190, and obtains output from the trained model 190 indicative of predicted data 168. The input data may include sensor data 142, manufacturing parameters 150, equipment constants 152, metrology data 160, golden trace data 162, predicted data 168, etc. In some embodiments, the predicted data is indicative of metrology data (e.g., a prediction of substrate quality). In some embodiments, the predicted data is indicative of component and / or process chamber health. In some embodiments, the predicted data is indicative of processing progress (e.g., utilized to terminate a processing operation). In some embodiments, the predicted data 168 includes golden trace data. In some embodiments, the predicted data 168 includes updated equipment constants. In some embodiments, the predicted data 168 includes a schedule of corrective actions, e.g., a schedule to update equipment constants.

[0067] In some embodiments, the various models discussed in connection with model 190 (e.g., supervised machine learning models, unsupervised machine learning models, etc.) may be combined into one model (e.g., an ensemble model) or may be separate models.

[0068] Data can be passed back and forth between several separate models included in models 190, corrective action component 122, and prediction component 114. In some embodiments, some or all of these operations can instead be performed by different devices, such as client device 120, server machine 170, server machine 180, etc. Those skilled in the art will understand that variations in data flow, which components perform which processes, which data is provided to which models, etc. are within the scope of this disclosure.

[0069] The data store 140 can be a memory (e.g., random access memory), a drive (e.g., a hard drive, a flash drive), a database system, a cloud-accessible memory system, or another type of component or device capable of storing data. The data store 140 can include multiple storage components (e.g., multiple drives or multiple databases) across multiple computing devices (e.g., multiple server computers). The data store 140 can store sensor data 142, manufacturing parameters 150, metrology data 160, golden trace data 162, and prediction data 168.

[0070] The sensor data 142 may include historical sensor data 144 and current sensor data 146. The sensor data may include a sensor data time trace over the duration of the manufacturing process, associations between data and physical sensors, preprocessed data, e.g., average and synthetic data, and data indicating sensor performance over time (i.e., for many manufacturing processes). The manufacturing parameters 150 and metrology data 160 may include similar characteristics, e.g., historical metrology data and current metrology data. The historical sensor data 144, historical metrology data, and historical manufacturing parameters may be historical data (e.g., at least a portion of these data that can be used to train the model 190). The current sensor data 146, current metrology data, and current manufacturing parameters may be current data (e.g., at least a portion of past data input into the learning model 190) from which the predictive data 168 (e.g., for performing corrective actions) is generated. The equipment constants 152 may include current equipment constants, historical equipment constants (e.g., for training a model), expected equipment constants (e.g., scheduled updates to the equipment constants), etc. Sensor data, manufacturing parameters, metrology data, etc. can include real data (e.g., measured from the manufacturing process, fabricated substrates, etc.) and synthetic (e.g., generated by machine learning models, subject matter experts, etc.) data.

[0071] In some embodiments, prediction system 110 further includes server machine 170 and server machine 180. Server machine 170 includes dataset generator 172 capable of generating datasets (e.g., a set of data inputs and a set of target outputs) for training, validating, and / or testing model 190, which may include one or more machine learning models. Some operations of dataset generator 172 are described in more detail below with respect to FIGS. 2A-2B and 4A. In some embodiments, dataset generator 172 may divide historical data (e.g., historical sensor data 144, historical manufacturing parameters, historical metrology data 164) into a training set (e.g., 60 percent of the historical data), a validation set (e.g., 20 percent of the historical data), and a test set (e.g., 20 percent of the historical data).

[0072] In some embodiments, the prediction system 110 (e.g., via the prediction component 114) generates multiple sets of features. For example, a first set of features can correspond to a first set of types of sensor data (e.g., a first set of sensors, a first combination of values ​​from the first set of sensors, a first pattern of values ​​from the first set of sensors) corresponding to each of the data sets (e.g., a training set, a validation set, and a test set), and a second set of features can correspond to a second set of types of sensor data (e.g., from a second set of sensors different from the first set of sensors, a second combination of values ​​different from the first combination, a second pattern different from the first pattern) corresponding to each of the data sets.

[0073] In some embodiments, the machine learning model 190 is provided with historical data as training data. In some embodiments, the machine learning model 190 is provided with output from another machine learning model, such as predicted data 168, as training data. The type of data provided varies depending on the intended use of the machine learning model. For example, the machine learning model can be trained by providing it with historical sensor data 144 as training inputs and corresponding metrology data 160 as target outputs. In some embodiments, a large amount of data is used to train the model 190, for example, sensor and metrology data for hundreds of boards can be used.

[0074] Server machine 180 includes a training engine 182, an authentication engine 184, a selection engine 185, and / or a test engine 186. The engines (e.g., training engine 182, authentication engine 184, selection engine 185, and test engine 186) may refer to hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, processing device, etc.), software (e.g., instructions executed on a processing device, general-purpose computer system, or dedicated machine), firmware, microcode, or a combination thereof. Training engine 182 may be capable of training a model 190 using one or more sets of features associated with a training set from dataset generator 172. Training engine 182 may generate multiple trained models 190, each corresponding to a distinct set of features (e.g., sensor data from a distinct set of sensors) of the training set. For example, a first trained model may be trained using all features (e.g., X1-X5), a second trained model may be trained using a first subset of features (e.g., X1, X2, X4), and a third trained model may be trained using a second subset of features (e.g., X1, X3, X4, and X5) that may partially overlap with the features of the first subset. The dataset generator 172 may receive the output of the trained models (e.g., prediction data 168 or equipment constants to be updated), collect that data into training, validation, and test datasets, and use those datasets to train a second model (e.g., a machine learning model configured to output prediction data, corrective actions, etc.).

[0075] The authentication engine 184 may be capable of authenticating the trained models 190 using a corresponding set of features of the authentication set from the dataset generator 172. For example, a first trained machine learning model 190 trained using a first set of features of the training set may be authenticated using the first set of features of the authentication set. The authentication engine 184 may determine the accuracy of each of the trained models 190 based on the corresponding sets of features of the authentication set. The authentication engine 184 may discard trained models 190 having an accuracy that does not meet a threshold accuracy. In some embodiments, the selection engine 185 may be capable of selecting one or more trained models 190 having an accuracy that meets the threshold accuracy. In some embodiments, the selection engine 185 may be capable of selecting the trained model 190 with the highest accuracy among the trained models 190.

[0076] The testing engine 186 may be capable of testing the trained model 190 using a corresponding set of features of a test set from the dataset generator 172. For example, a first trained machine learning model 190 trained using a first set of features of the training set may be tested using a first set of features of the test set. The testing engine 186 may determine the trained model 190 with the highest accuracy of all the trained models based on the test set.

[0077] In the case of a machine learning model, the model 190 may refer to model artifacts (ground truths for each training input) created by the training engine 182 using a training set including data inputs and corresponding target outputs. In an embodiment, the training set includes synthetic microscope images generated by the synthetic data generator 174. Patterns within the dataset that map the data inputs to the target outputs (ground truths) may be found, and mappings capturing these patterns are provided to the machine learning model 190. The machine learning model 190 may use one or more of the following: support vector machines (SVMs), radial basis functions (RBFs), clustering, supervised machine learning, semi-supervised machine learning, unsupervised machine learning, k-nearest neighbor algorithms (k-NNs), linear regression, random forests, neural networks (e.g., artificial neural networks, recurrent neural networks), etc. The synthetic data generator 174 may include one or more machine learning models, which may include one or more of the same type of model (e.g., artificial neural networks).

[0078] In some embodiments, one or more machine learning models 190 may be trained using historical data (e.g., historical sensor data 144). In some embodiments, the models 190 may have been trained using synthetic data 162 or a combination of historical and synthetic data.

[0079] The prediction component 114 can provide current data to the model 190 and can execute the model 190 on the inputs to obtain one or more outputs. For example, the prediction component 114 can provide current sensor data 146 to the model 190 and can execute the model 190 on the inputs to obtain one or more outputs. The prediction component 114 can be capable of determining (e.g., extracting) prediction data 168 from the output of the model 190. The prediction component 114 can determine (e.g., extract) confidence data from the output that indicates a confidence level that the prediction data 168 is an accurate predictor of the current sensor data 146 and / or the process associated with the input data for products produced or to be produced using the manufacturing equipment 124 with current manufacturing parameters. The prediction component 114 or the corrective action component 122 can use the confidence data to determine whether to trigger a corrective action associated with the manufacturing equipment 124 based on the prediction data 168.

[0080] The confidence data may include or indicate a confidence level that the predicted data 168 is an accurate prediction for a product or component associated with at least a portion of the input data. In one example, the confidence level is a real number between 0 and 1, inclusive, where 0 indicates no confidence that the predicted data 168 is an accurate prediction for a product processed according to the input data or component health status of the component of the manufacturing equipment 124, and 1 indicates absolute confidence that the predicted data 168 will accurately predict a characteristic of a product processed according to the input data or component health status of the component of the manufacturing equipment 124. In response to the confidence data indicating a confidence level below a threshold level for a predetermined number of cases (e.g., a percentage of cases, a frequency of cases, a total number of cases, etc.), the prediction component 114 may cause the trained model 190 to be retrained (e.g., based on current sensor data 146, current manufacturing parameters, etc.). In some embodiments, the retraining may include generating one or more datasets (e.g., via the dataset generator 172) utilizing historical data and / or synthetic data.

[0081] For purposes of illustration and not limitation, aspects of the present disclosure describe training one or more machine learning models 190 using historical data (e.g., historical sensor data 144, historical manufacturing parameters) and inputting current data (e.g., current sensor data 146, current manufacturing parameters, and current metrology data) into the one or more trained machine learning models to determine predicted data 168. In other embodiments, heuristic, physics-based, or rule-based models are used to determine predicted data 168 (e.g., without using trained machine learning models). In some embodiments, such models can be trained using historical data and / or synthetic data. In some embodiments, these models can be retrained utilizing a combination of real historical data and synthetic data. The prediction component 114 can monitor historical sensor data 144, historical manufacturing parameters, and metrology data 160. Any of the information described in connection with data inputs 210A-B of FIGS. 2A-2B can be monitored or otherwise used in the heuristic, physics-based, or rule-based models.

[0082] In some embodiments, the functionality of client device 120, prediction server 112, server machine 170, and server machine 180 may be provided by fewer machines. For example, in some embodiments, server machines 170 and 180 may be combined into a single machine, and in some other embodiments, server machine 170, server machine 180, and prediction server 112 may be combined into a single machine. In some embodiments, client device 120 and prediction server 112 may be combined into a single machine. In some embodiments, the functionality of client device 120, prediction server 112, server machine 170, server machine 180, and data store 140 may be performed by a cloud-based service.

[0083] Generally, functionality described in one embodiment as being performed by client device 120, prediction server 112, server machine 170, and server machine 180 may, in other embodiments, be performed on prediction server 112, where appropriate. Additionally, functionality attributed to one particular component may also be performed by different or multiple components operating together. For example, in some embodiments, prediction server 112 may determine corrective actions based on prediction data 168. In another example, client device 120 may determine prediction data 168 based on output from a trained machine learning model.

[0084] Additionally, the functionality of one particular component may be performed by different or multiple components working together. One or more of prediction server 112, server machine 170, or server machine 180 may be accessed as a service offered to other systems or devices via an appropriate application programming interface (API).

[0085] In embodiments, a "user" may be represented as a single individual. However, other embodiments of the present disclosure encompass a "user" that is an entity controlled by multiple users and / or automated sources. For example, a set of individual users united as an administrator group may be considered a "user."

[0086] Embodiments of the present disclosure may be applied to data quality assessment, feature enhancement, model evaluation, virtual metrology (VM), predictive maintenance (PdM), marginal optimization, process control, and the like.

[0087] 2A-2B illustrate block diagrams of example dataset generators 272A-B (e.g., dataset generators 172 of FIG. 1 ) that create datasets for training, testing, validating, etc., of a model (e.g., model 190 of FIG. 1 ), according to some embodiments. Each dataset generator 272 may be part of server machine 170 of FIG. 1 . In some embodiments, several machine learning models associated with manufacturing equipment 124 may be trained, used, and maintained (e.g., within a manufacturing facility). Each machine learning model may be associated with one dataset generator 272, multiple machine learning models may share a dataset generator 272, etc.

[0088] FIG. 2A illustrates a system 200A including a dataset generator 272A for creating datasets for one or more supervised models (e.g., model 190 of FIG. 1 ). A supervised model can be generated by providing the model with training inputs and target outputs (e.g., ground truths) associated with the training inputs. The dataset generator 272A can create a dataset (e.g., data inputs 210A, target outputs 220A) using historical data, such as historical sensor data, historical measurement data, etc. The dataset generator 272A can be utilized to generate one or more datasets for a machine learning model configured to recommend corrective actions. The dataset generator 272A can be utilized to generate one or more datasets for a machine learning model configured to implement corrective actions. The dataset generator 272A can be utilized to generate one or more datasets for a machine learning model configured to schedule updates to equipment constants of manufacturing equipment.

[0089] The dataset generator 272A can generate datasets for training, testing, and validating models. In some embodiments, the dataset generator 272A can generate datasets for machine learning models. In some embodiments, the dataset generator 272A can generate datasets for training, testing, and / or validating machine learning models configured to schedule updates to equipment constants of manufacturing equipment. The machine learning model is provided with a set of target trace data 242A and a set of historical trace data 246A as data inputs 210A. The target trace data can include golden trace data. The historical trace data can include data from a process chamber, for example, a process chamber exhibiting poor performance. The poor performance can include a threshold number or portion of products that do not meet a threshold quality indicator, a number or portion of equipment constants that are outliers compared to other chambers, etc. The machine learning model can be configured to recommend equipment constant changes predicted to make the trace data of the process chamber more similar to the target trace data as output.

[0090] In some embodiments, the dataset generator 272A can generate additional sets of data as part of the data input 210A provided to the model. The model can be configured to recommend changes to equipment constants based on the additional data. The machine learning model can be provided with one or more sets of target metrology data. The target metrology data can be associated with the golden trace. The machine learning model can be provided with one or more sets of historical metrology data, for example, product metrology data associated with sets of historical trace data. The machine learning model can be provided with one or more sets of equipment constants, for example, equipment constants to be updated, equipment constants associated with the process chamber that produced the golden trace data, etc.

[0091] The dataset generator 272A can be utilized to train a machine learning model to recommend and / or schedule corrective actions. The machine learning model can be configured to adjust equipment constants of one or more process chambers. The machine learning model can be configured to adjust the equipment constants to more closely resemble the equipment constants of a golden chamber. The machine learning model can be configured to adjust the equipment constants to enable the trace data of the process chamber to more closely resemble the golden trace data. The machine learning model can be configured to adjust the equipment constants to enable measurements of products produced by the process chamber to more closely resemble measurements of products produced by the golden chamber.

[0092] The machine learning model can generate a schedule for equipment constant updates. The machine learning model can limit the number of updates to process chambers, the number of updates to tools, the number of updates within a facility, etc. that are performed at one time. The machine learning model can limit the number of chambers or tools that are updated at one time. The machine learning model can base the equipment constant update schedule on the risk and effectiveness of updating the equipment constants. For example, the machine learning model can prioritize equipment constant updates that are low risk (e.g., less likely to increase the production of faulty products) and more likely to effectively address process chamber defects. The risk and effectiveness can be assessed based on subject matter experts, assessed by physics-based models, assessed by statistical or machine learning models, etc. The machine learning model can perform optimization operations associated with scheduling the equipment constant updates. The machine learning model can optimize the equipment constant updates to reduce the risk of reducing process chamber effectiveness. The machine learning model can optimize the equipment constant updates to increase the likelihood of improving process chamber effectiveness. The machine learning model can optimize the equipment constant updates to achieve other goals, such as reduced energy consumption, reduced material consumption, reduced process time, etc.

[0093] The dataset generator 272A can be used to generate data for any type of machine learning model, such as the machine learning architecture discussed in connection with FIG. 1 . The dataset generator 272A can be used to generate data for a machine learning model that recommends equipment constant updates. The dataset generator 272A can be used to generate data for a machine learning model that schedules equipment constant updates. The dataset generator 272A can be used to generate data for a machine learning model that performs equipment constant updates. The dataset generator 272A can be used to generate data for a machine learning model configured to identify product anomalies and / or process equipment faults. For example, the dataset generator 272A can be used to generate data for a machine learning model configured to detect outliers in equipment constants, outliers in correlations or relationships between equipment constants, trace data, and / or measurement data, etc. The dataset generator 272A can be used to generate data for a machine learning model configured to detect causal relationships, e.g., cause and effect for one or more indicators.

[0094] In some embodiments, the dataset generator 272A generates a dataset (e.g., a training set, a validation set, a test set) that includes one or more data inputs 210A (e.g., training inputs, validation inputs, test inputs). The data inputs 210A can be provided to the training engine 182, the validation engine 184, or the test engine 186. The dataset can be used to train, validate, or test a model (e.g., model 190 of FIG. 1).

[0095] In some embodiments, data input 210A can include one or more sets of data. By way of example, system 200A can create multiple sets of sensor data, where such multiple sets of sensor data can include one or more of sensor data from one or more types of sensors, combinations of sensor data from one or more types of sensors, patterns of sensor data from one or more types of sensors, etc.

[0096] In some embodiments, data input 210A can include one or more sets of data. As an example, system 200A can generate multiple sets of historical metrology data, where such multiple sets of historical metrology data can include one or more of: metrology data for a group of device dimensions (e.g., including device thickness, but not including optical data or surface roughness, etc.); metrology data derived from one or more types of sensors; a combination of metrology data derived from one or more types of sensors; a pattern of metrology data; etc. The multiple sets of data input 210A can include data describing different aspects of manufacturing, such as a combination of metrology data and sensor data, a combination of metrology data and manufacturing parameters, a combination of some metrology data, some manufacturing parameter data, and some sensor data, data associated with components of a manufacturing system, such as part quality data, etc. Data input 210A can include measured data and / or synthetic data.

[0097] In some embodiments, the dataset generator 272A may generate a first data input corresponding to the first set of target trace data 242A and the first set of historical trace data 246A for training, validating, or testing a first machine learning model. The dataset generator 272A may generate a second data input corresponding to the second set of target trace data 242B and the second set of historical trace data 246B for training, validating, or testing a second machine learning model.

[0098] In some embodiments, the dataset generator 272A generates a dataset (e.g., a training set, a validation set, a test set) that includes one or more data inputs 210A (e.g., training inputs, validation inputs, test inputs), and such dataset may include one or more target outputs 220A that correspond to the data inputs 210A. The dataset may also include mapping data that maps the data inputs 210A to the target outputs 220A. In some embodiments, the dataset generator 272A may generate data for training a machine learning model configured to output equipment constant updates by generating a dataset that includes output equipment constant data 268. The data inputs 210A may also be referred to as “features,” “attributes,” or “information.” In some embodiments, the dataset generator 272A may provide this dataset to the training engine 182, the validation engine 184, or the test engine 186, where the dataset is used to train, validate, or test a machine learning model (e.g., one of the machine learning models included in model 190, an ensemble machine learning model, etc.).

[0099] System 200B, including dataset generator 272B (e.g., dataset generator 172 of FIG. 1 ), creates a dataset for one or more machine learning models (e.g., model 190 of FIG. 1 ). Dataset generator 272B can use historical data to create a dataset (e.g., data input 210B). An example dataset generator 272B is configured to generate a dataset for a machine learning model configured to take data associated with processed products as input and generate golden trace data as output. Dataset generator 272B can provide a dataset to an unsupervised machine learning model; for example, dataset generator 272B may provide data input 210B but not a target output. Dataset generator 272B can share one or more features and / or functions with dataset generator 272A.

[0100] The dataset generator 272B can generate datasets for training, testing, and validating the machine learning model. The machine learning model is provided with a set of golden chamber data 262A (e.g., metrology data of a product processed by a golden process chamber, trace data from processing the product, etc.) as data input 210B. The machine learning model can include two or more separate models (e.g., the machine learning model can be an ensemble model). The machine learning model can be configured to generate output data including golden trace data. The golden trace data can include traces matched by process chambers other than the golden process chamber. The golden trace data can include upper and / or lower limits, for example, to define a region of acceptable trace data. The golden trace data can be from different processing runs, different golden chambers, different sensors, etc. For example, trace data from different sensors can affect measurements differently. Different chambers can perform various measurements differently, and the golden trace data can reflect improved performance of one golden chamber in a particular area (e.g., a particular subsystem, such as a pressure subsystem, an RF subsystem, etc.) relative to another golden chamber. Dataset generator 272B can generate datasets for training unsupervised machine learning models, such as models configured to receive synthetic microscope data as input and generate clustering data, outlier detection data, anomaly detection data, etc. as output. The models can be trained to generate output data based on associations between sensor data and measurement data, for example, the models can be trained to recognize how trace data from a combination of sensors correlates to the measurement data.

[0101] In some embodiments, the dataset generator 272B generates a dataset (e.g., a training set, a validation set, a test set) that includes one or more data inputs 210B (e.g., training inputs, validation inputs, test inputs). The data inputs 210B may also be referred to as "features," "attributes," or "information." In some embodiments, the dataset generator 272B may provide this dataset to the training engine 182, validation engine 184, or test engine 186, where the dataset is used to train, validate, or test a machine learning model (e.g., model 190 of FIG. 1). Some embodiments of generating a training set are further described in connection with FIG. 4A.

[0102] In some embodiments, the dataset generator 272B can generate a first data input corresponding to the first set of golden chamber data 244A for training, validating, or testing a first machine learning model, and the dataset generator 272A can generate a second data input corresponding to the second set of golden chamber data 244B for training, validating, or testing a second machine learning model.

[0103] The data input 210B for training, validating, or testing a machine learning model can include information for a particular manufacturing chamber (e.g., a particular piece of substrate manufacturing equipment). In some embodiments, the data input 210B can include information for a particular type of manufacturing equipment, e.g., manufacturing equipment sharing a particular characteristic. The data input 210B can include data associated with a particular type, e.g., intended function, design, etc., of a device produced by a particular recipe. Training a machine learning model based on the type of equipment, device, recipe, equipment, etc. can enable the trained model to generate golden trace data that is applicable in multiple settings (e.g., multiple different pieces of equipment, products, etc.).

[0104] In some embodiments, after generating a dataset and using the dataset to train, validate, or test a machine learning model, the model can be further trained, validated, or tested, or tuned (e.g., by adjusting weights or parameters associated with the model's input data, such as connection weights in a neural network). Further training, validation, testing, or tuning can be performed with additional data, for example, additional training data generated by manufacturing equipment after the model is initially trained.

[0105] FIG. 3 is a block diagram illustrating a system 300 for generating output data (e.g., predicted data 168 of FIG. 1 ), according to some embodiments. In some embodiments, a system such as system 300 can be used with a machine learning model configured to generate golden trace data (e.g., golden trace data 162 of FIG. 1 ). In some embodiments, a system such as system 300 can be used with a machine learning model to determine corrective actions associated with manufacturing equipment. In some embodiments, a system such as system 300 can be used with a machine learning model to determine faults in manufacturing equipment. In some embodiments, a system such as system 300 can be used with a machine learning model to cluster or classify equipment constants for process tools or chambers. A system such as system 300 can be used with a machine learning model to schedule updates to equipment constants for manufacturing equipment. A system such as system 300 can be used with a machine learning model having different functionality than those listed above associated with a manufacturing system.

[0106] System 300 and the accompanying description are directed to a machine learning model that receives data from one or more golden process chambers and data from one or more other process chambers as input and generates recommended corrective actions to improve the performance of the one or more other process chambers as output. The golden process chamber data may include golden trace data and golden equipment constants. The other process chamber data may include trace data and equipment constants. The input data may further include metrology data, additional chamber data, target performance data, etc. The output may further include recommended equipment constant updates, scheduled equipment constant updates, etc. With appropriate modifications, such as identification of the input and output data, machine learning models having other functionality may also be operated with systems similar to system 300.

[0107] At block 310, the system 300 (e.g., a component of the prediction system 110 of FIG. 1 ) performs data partitioning (e.g., via the data set generator 172 of the server machine 170 of FIG. 1 ) of data used in training, validating, and / or testing the machine learning model. In some embodiments, the training data 364 includes golden data, such as golden trace data, golden measurement data, or golden equipment constants. In some embodiments, the training data 364 includes historical data, such as historical measurement data, historical equipment constant data, or historical trace data. The training data 364 may undergo data partitioning at block 310 to generate a training set 302, a validation set 304, and a test set 306. For example, the training set may be 60% of the training data, the validation set may be 20% of the training data, and the test set may be 20% of the training data.

[0108] The generation of the training set 302, validation set 304, and test set 306 can be tailored to a particular application. For example, the training set can be 60% of the training data, the validation set can be 20% of the training data, and the test set can be 20% of the training data. System 300 can generate multiple sets of features for each of the training, validation, and test sets. For example, if training data 364 includes features derived from sensor data from 20 sensors (e.g., sensor 126 in FIG. 1 ) and sensor data including 10 manufacturing parameters (e.g., manufacturing parameters corresponding to the same processing run as the sensor data from the 20 sensors), the sensor data can be divided into a first set of features including sensors 1-10 and a second set of features including sensors 11-20. The manufacturing parameters can also be divided into multiple sets, e.g., a first set of manufacturing parameters including parameters 1-5 and a second set of manufacturing parameters including parameters 6-10. The training inputs, target outputs, or both can be divided into multiple sets, or none can be divided. Multiple models can be trained on different sets of data.

[0109] At block 312, the system 300 performs model training (e.g., via the training engine 182 of FIG. 1 ) using the training set 302. Training of machine learning models and / or physics-based models (e.g., digital twins) can be accomplished with supervised learning, which involves providing a training dataset including labeled inputs through the model, observing its outputs, defining an error (by measuring the difference between the output and the label values), and adjusting the model weights to minimize the error using techniques such as gradient descent and backpropagation. In many applications, repeating this process across many labeled inputs in the training dataset results in a model that can provide correct outputs when presented with inputs different from those present in the training dataset. In some embodiments, training of the machine learning model can be accomplished unsupervised, e.g., no labels or classifications may be provided during training. Unsupervised models can be configured to perform anomaly detection, obtain clustering, etc. In some embodiments, generation of golden trace data can be performed by the unsupervised machine learning model, e.g., by determining correlations between various traces and metrology metrics to determine ranges of trace data that are likely to result in acceptable metrology and / or performance of the substrate.

[0110] For each training data item in the training dataset, the training data item can be input into a model (e.g., a machine learning model). The model can then process the input training data item (e.g., multiple measured dimensions of a manufactured device, a cartoon of a manufactured device, etc.) to generate an output. The output can include, for example, a corrective action. The output can be compared to the label of the training data item (e.g., a corrective action taken to correct a problem associated with historical data). In some embodiments, an unsupervised model can be utilized to recommend the corrective action; for example, the unsupervised model can learn the relationship between equipment constants and substrate performance and provide updates to predicted equipment constants to improve substrate performance.

[0111] Processing logic may then compare the generated output (e.g., a recommended corrective action) with the label (e.g., the actual corrective action) included in the training data item. Processing logic determines an error (i.e., a classification error) based on the difference between the output and the label. Processing logic adjusts one or more weights and / or values ​​of the model based on the error.

[0112] When training a neural network, an error term or Δ can be determined for each node in the artificial neural network. Based on this error, the artificial neural network adjusts one or more of its parameters (weights for one or more inputs to the node) for one or more of its nodes. Parameters can be updated via backpropagation, so that nodes in the highest layer are updated first, followed by nodes in the next layer, and so on. An artificial neural network includes multiple layers of "neurons," each layer receiving input values ​​from neurons in the previous layer. The parameters for each neuron include weights associated with the values ​​received from each of the neurons in the previous layer. Adjusting a parameter can therefore include adjusting the weights assigned to each of the inputs to one or more neurons in one or more layers in the artificial neural network.

[0113] One or more operations of the system 300 may be performed by a statistical model. The statistical model may utilize input data to determine output data through one or more statistical operations. The operations of the system 300 may be performed by a heuristic model or a rule-based model.

[0114] The system 300 can train multiple models using multiple sets of features from the training set 302 (e.g., a first set of features from the training set 302, a second set of features from the training set 302, etc.). For example, the system 300 can train a model using a first set of features in the training set (e.g., sensor data from sensors 1-10, metrology measurements 1-10, etc.) to generate a first trained model and a second set of features in the training set (e.g., sensor data from sensors 11-20, metrology measurements 11-20, etc.) to generate a second trained model. In some embodiments, the first trained model and the second trained model can be combined to generate a third trained model (e.g., which may be a better predictor or synthetic data generator than the first or second trained models alone). In some embodiments, the sets of features used in comparing models may overlap (e.g., a first set of features is sensor data from sensors 1-15 and a second set of features is sensor data from sensors 5-20). In some embodiments, hundreds of models can be generated, including models with various permutations of features and combinations of models.

[0115] At block 314, the system 300 performs model validation (e.g., via validation engine 184 of FIG. 1 ) using the validation set 304. The system 300 may validate each of the trained models using a corresponding set of features in the validation set 304. For example, the system 300 may validate a first trained model using a first set of features in the validation set (e.g., sensor data or metrology measurements 1-10 from sensors 1-10) and validate a second trained model using a second set of features in the validation set (e.g., sensor data or metrology measurements 11-20 from sensors 11-20). In some embodiments, the system 300 may validate hundreds of models (e.g., models with various permutations of features, combinations of models, etc.) generated at block 312. At block 314, the system 300 may determine the accuracy of each of the one or more trained models (e.g., via model validation) and may determine whether one or more of the trained models have an accuracy that meets a threshold accuracy. In response to determining that none of the trained models have an accuracy that meets the threshold accuracy, flow returns to block 312, where the system 300 performs model training using different sets of features from the training set. In response to determining that one or more of the trained models have an accuracy that meets the threshold accuracy, flow proceeds to block 316. The system 300 can discard trained models that have an accuracy below the threshold accuracy (e.g., based on a validation set).

[0116] At block 316, the system 300 performs model selection (e.g., via selection engine 185 of FIG. 1 ) to determine which of the one or more trained models that meet the threshold accuracy has the highest accuracy (e.g., selected model 308 based on the validation of block 314). In response to determining that two or more of the trained models that meet the threshold accuracy have the same accuracy, flow may return to block 312, where the system 300 performs model training using further refined training sets corresponding to further refined sets of features to determine the trained model with the highest accuracy.

[0117] At block 318, the system 300 performs model testing (e.g., via the test engine 186 of FIG. 1 ) using the test set 306 to test the selected model 308. The system 300 may test the first trained model using a first set of features in the test set (e.g., sensor data from sensors 1-10) and determine that the first trained model meets a threshold accuracy (e.g., based on the first set of features in the test set 306). In response to the accuracy of the selected model 308 not meeting the threshold accuracy (e.g., the selected model 308 is overfitted to the training set 302 and / or the validation set 304 and is not applicable to other datasets, such as the test set 306), flow proceeds to block 312, where the system 300 performs model training (e.g., retraining) using a different training set corresponding to different sets of features (e.g., sensor data from different sensors). In response to determining, based on the test set 306, that the selected model 308 has an accuracy that meets the threshold accuracy, flow proceeds to block 320. At least in block 312, the model can learn patterns in the training data to make predictions or generate a schedule for updating equipment constants, and in block 318, the system 300 can apply the model to the remaining data (e.g., the test set 306) to test the predictions.

[0118] At block 320, the system 300 uses the trained model (e.g., the selected model 308) to receive current data 322 (e.g., current trace data associated with a recently processed substrate, current equipment constants for a process chamber, etc.) and determines (e.g., extracts) equipment constant data 324 (e.g., predicted data 168 of FIG. 1 ) from the output of the trained model. Corrective actions associated with the manufacturing equipment 124 of FIG. 1 can be performed taking into account the equipment constant data 324. In some embodiments, the current data 322 can correspond to the same types of features in the historical data used to train the machine learning model. In some embodiments, the current data 322 corresponds to a subset of the types of features in the historical data used to train the selected model 308 (e.g., the machine learning model can be trained using multiple metrology measurements and can be configured to generate output based on a subset of the metrology measurements).

[0119] In some embodiments, the performance of a machine learning model trained, validated, and tested by system 300 may degrade. For example, a manufacturing system associated with a trained machine learning model may undergo gradual or sudden changes. As a result of the changes in the manufacturing system, the performance of the trained machine learning model may degrade. A new model can be generated to replace the degraded machine learning model. The new model can be generated by modifying the old model by retraining, by generating a new model, etc.

[0120] In some embodiments, one or more of operations 310-320 may be performed in various orders and / or with other operations not presented and described herein. In some embodiments, one or more of operations 310-320 may not be performed. For example, in some embodiments, one or more of data partitioning of block 310, model validation of block 314, model selection of block 316, or model testing of block 318 may not be performed.

[0121] 3 illustrates a system configured to train, validate, test, and use one or more machine learning models. The machine learning models are configured to accept data (e.g., set points provided to manufacturing equipment, sensor data, measurement data, etc.) as input and provide data (e.g., prediction data, corrective action data, classification data, etc.) as output. The input and / or output data can be processed, features extracted, formatted for convenience or ease of interpretation by the model, etc. The partitioning, training, validation, selection, testing, and use of blocks of system 300 can be similarly performed to train a second model utilizing a different type of data. Retraining can also be performed utilizing current data 322 and / or additional training data 346.

[0122] 4A-4C are flow diagrams of methods 400A-C associated with training and utilizing models, according to certain embodiments. Methods 400A-C may include training and utilizing machine learning models, statistical models, rule-based models, heuristic models, physics-based models, etc. Methods 400A-C may be associated with recommending and / or implementing corrective actions. Methods 400A-C may be associated with updating equipment constants for one or more process chambers. Methods 400A-C may be performed by processing logic, which may include hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, processing device, etc.), software (e.g., instructions executed on a processing device, general-purpose computer system, or dedicated machine), firmware, microcode, or a combination thereof. In some embodiments, methods 400A-C may be performed in part by prediction system 110. Method 400A may be performed in part by prediction system 110 (e.g., server machine 170 and dataset generator 172 of FIG. 1 , dataset generators 272A-B of FIGS. 2A-2B ). According to embodiments of the present disclosure, prediction system 110 may use method 400A to generate datasets for at least one of training, validating, or testing a machine learning model. Methods 400B-C may be performed by prediction server 112 (e.g., prediction component 114) and / or server machine 180 (e.g., training, validating, and testing operations may be performed by server machine 180). In some embodiments, a non-transitory machine-readable storage medium stores instructions that, when executed by a processing device (e.g., prediction system 110, server machine 180, prediction server 112, etc.), cause the processing device to perform one or more of methods 400A-C.

[0123] For ease of explanation, methods 400A-C are shown and described as a series of operations. However, operations in accordance with the present disclosure may occur in various orders and / or simultaneously, and with other operations not shown and described herein. Moreover, not all illustrated operations may be performed to implement methods 400A-C in accordance with the disclosed subject matter. In addition, those skilled in the art will understand and appreciate that methods 400A-C may alternatively be represented as a series of interrelated states via a state diagram or events.

[0124] 4A is a flow diagram of a method 400A for generating a dataset for a machine learning model, according to some embodiments. Referring to FIG. 4A, in some embodiments, at block 401, processing logic performs method 400A for initializing a training set T to an empty set.

[0125] At block 402, processing logic generates a first data input (e.g., a first training input, a first validation input) that may include one or more of sensor, manufacturing parameter, metrology data, etc. In some embodiments, the first data input may include a first set of features for multiple types of data, and the second data input may include a second set of features for multiple types of data (e.g., as described in connection with FIG. 3). The input data may include historical data.

[0126] In some embodiments, at block 403, the processing logic optionally generates a first target output for one or more of the data inputs (e.g., the first data input). In some embodiments, the input includes one or more golden traces, and the target output includes recommended updates to the equipment constants. In some embodiments, the input further includes additional data, such as metrology data, process chamber equipment constants, and / or process chamber trace data, and the target output includes a recommended schedule of equipment constant updates. In some embodiments, no target output is generated (e.g., the unsupervised machine learning model is not provided with a target output, but rather the unsupervised machine learning model is able to group the input data or find correlations in the input data).

[0127] At block 404, processing logic optionally generates mapping data indicating an input / output mapping. The input / output mapping (or mapping data) may reference data inputs (e.g., one or more of the data inputs described herein), target outputs for the data inputs, and associations between the data inputs and the target outputs. In some embodiments, such as those associated with machine learning models that do not provide a target output, block 404 may not be performed.

[0128] In some embodiments, at block 405, processing logic adds the mapping data generated at block 404 to the dataset T.

[0129] At block 406, processing logic branches based on whether dataset T is sufficient for at least one of training, validating, and / or testing a machine learning model, such as model 190 in FIG. 1. If dataset T is sufficient, execution proceeds to block 407; otherwise, execution returns to block 402. Note that in some embodiments, whether dataset T is sufficient may be determined based solely on the number of inputs that map to outputs in the dataset, while in some other embodiments, whether dataset T is sufficient may be determined based on one or more other criteria (e.g., a measure of diversity of data examples, accuracy, etc.) in addition to or instead of the number of inputs.

[0130] At block 407, processing logic provides (e.g., to server machine 180) dataset T for training, certifying, and / or testing machine learning model 190. In some embodiments, dataset T is a training set and is provided to training engine 182 of server machine 180 to perform training. In some embodiments, dataset T is a certification set and is provided to certification engine 184 of server machine 180 to perform certification. In some embodiments, dataset T is a test set and is provided to test engine 186 of server machine 180 to perform testing. For example, in the case of a neural network, input values ​​of a given input / output mapping (e.g., numerical values ​​associated with data input 210A) are input to the neural network, and output values ​​of that input / output mapping (e.g., numerical values ​​associated with target output 220A) are stored in output nodes of the neural network. Then, according to a learning algorithm (e.g., backpropagation, etc.), the weights of the connections in the neural network are adjusted, and the procedure is repeated for other input / output mappings in dataset T. After block 407, the model (e.g., model 190) may be at least one of trained using training engine 182 of server machine 180, certified using certification engine 184 of server machine 180, or tested using test engine 186 of server machine 180. The trained model may be implemented by prediction component 114 (of prediction server 112) to generate prediction data 168 for performing signal processing, to generate golden trace data, or to execute corrective actions 124 associated with manufacturing equipment.

[0131] 4B is a flow diagram of a method 400B for updating equipment constants of a process chamber, according to some embodiments. At block 410, data is provided as input to a first trained machine learning model. The provided data includes trace data. The trace data may be golden trace data. The trace data may be associated with a substrate, a substrate processing procedure, etc. The trace data may be associated with a substrate processing procedure that resulted in a substrate satisfying one or more criteria. The trace data may be associated with a substrate processing procedure that resulted in a substrate meeting one or more performance thresholds. The data provided as input includes golden equipment constants. The data provided includes trace data of a first processing chamber. The data provided as input includes equipment constants of the first processing chamber.

[0132] In some embodiments, additional input data can be provided to the first trained machine learning model. Metrology data can be provided to the first trained machine learning model. The metrology data can include golden metrology data, metrology data associated with golden trace data, metrology data associated with the first processing chamber, etc. The first trained machine learning model can be configured to recommend adjustments to manufacturing equipment to increase the similarity between acceptable metrology data or golden metrology data and current metrology data. The first trained machine learning model can be configured to adjust equipment constants to increase the similarity between substrates processed by one or more golden process chambers and substrates processed by the first processing chamber. The first trained machine learning model can be configured to adjust equipment constants to increase the similarity between trace data from the first processing chamber and the golden trace data. In some embodiments, the first trained machine learning model is configured to adjust equipment constants of the first processing chamber to target trace data within limits defined by the golden trace data. For example, the golden trace data can define an upper golden trace limit and a lower golden trace limit, and the machine learning model can recommend equipment constant updates to increase the likelihood that the first processing chamber will produce trace data within the golden trace limits.

[0133] In some embodiments, the golden trace data can be provided to the first trained machine learning model by a second trained machine learning model. The second trained machine learning model can be configured to generate one or more sets of golden trace data. The golden trace data can include data associated with a single substrate. The golden trace data can include data associated with multiple substrates. The golden trace data can include data from a single chamber (e.g., a single golden chamber). The golden trace data can include data from multiple process chambers.

[0134] In some embodiments, the second trained machine learning model can be provided with metrology data as input. The metrology data can be associated with acceptable products, e.g., products that meet one or more performance thresholds. The metrology data can be associated with measurements of substrates processed by a golden process chamber, e.g., a golden chamber. The second trained machine learning model can further be provided with trace data as input. The trace data input can be trace data associated with the metrology data, e.g., trace sensor data collected while processing a substrate associated with the input metrology data. The trace data input can be trace data for one or more golden chambers. The second machine learning model can be configured to generate golden traces according to one or more criteria. The golden trace data can be associated with one or more substrates that satisfy one or more criteria, one or more performance thresholds, etc. For example, the second machine learning model can be configured to select golden trace data based on the likelihood of a process indicated by the trace data resulting in a substrate that meets one or more performance thresholds. The second machine learning model can generate a mapping between the trace data and the metrology values. The second trained machine learning model can generate a mapping between the trace data and substrate performance. A second trained machine learning model can utilize these mappings to generate upper and lower golden trace data.

[0135] The golden trace data can be selected to correspond to upper and lower bounds on trace data values ​​corresponding to a target likelihood that a substrate will meet one or more performance thresholds. The upper golden trace can include the highest trace value of the plurality of input trace data, e.g., the highest trace value corresponding to an acceptable end product. The upper golden trace can include a statistical upper bound on the plurality of traces, e.g., based on quartile or standard deviation analysis. The lower bound trace data can be selected similarly to the upper bound trace data. The upper and lower bounds can generate a guard band for the golden trace data. The second trained machine learning model can order the plurality of trace data, e.g., organize the plurality of trace data from lowest to highest. The second trained machine learning model can base the order on the mean, median, difference at each time step from the mean of the plurality of traces for that time step, or another metric. The second trained machine learning model can be configured to generate golden trace data according to one or more configuration settings. For example, the second trained machine learning model can be configured to respond to specific substrate defects and generate upper and lower bounds on the golden trace data to avoid producing substrates containing the target defects.

[0136] In some embodiments, after the training operation, the second trained machine learning model can be provided with additional data. The second trained machine learning model can be retrained with the additional data (e.g., data associated with substrates processed after the initial training operation). The second trained machine learning model can be provided with input data and can adjust (e.g., retrain) one or more weights or biases based on the input data. The second trained machine learning model can assign a higher weight to newer data (e.g., data associated with more recently generated substrates) than older data. The second trained machine learning model can assign a higher weight to newer data than older data.

[0137] At block 412, processing logic obtains a recommended update to a first equipment constant of a first processing chamber as output from the first trained machine learning model. The processing logic may obtain multiple recommended updates to multiple equipment constants of the first processing chamber. The processing logic may further obtain one or more recommended updates to equipment constants of a second processing chamber. The processing logic may receive a schedule of the recommended updates, for example, the processing logic may receive an order in which to update the equipment constants.

[0138] At block 414, processing logic updates first equipment constants of the first processing chamber in response to obtaining output from the first trained machine learning model. Updating the equipment constants can include changing values ​​associated with operation of the first processing chamber. Updating the equipment constants can include scheduling a maintenance operation, such as a calibration operation. Updating the equipment constants can include initiating a maintenance operation, such as a calibration operation.

[0139] 4C is a flow diagram of a method 400C of performing corrective actions associated with a processing chamber, according to some embodiments. At block 420, a processing device (e.g., processing logic) receives first trace data associated with a first processing chamber. The first processing chamber satisfies one or more performance indicators. The first processing chamber may be a golden chamber. The first trace data may be or may include golden trace data. The first trace data and / or the first processing chamber may be associated with one or more processing operations that satisfy one or more conditions. The first trace data and / or the first processing chamber may be associated with one or more processing operations that resulted in a substrate satisfying the performance indicators.

[0140] The processing logic can further receive first metrology data for the first substrate. The first substrate can be associated with first trace data. The first substrate can satisfy one or more performance metrics. The first substrate can be part of a set of substrates having acceptable characteristics, e.g., the first substrate can be correlated to a designation of the first processing chamber as a golden chamber. The processing logic can further receive a first set of equipment constants associated with the first processing chamber.

[0141] At block 422, processing logic generates target trace data based on the first trace data associated with the first processing chamber. The target trace data can be or include golden trace data. The target trace data can include upper and lower limits. The target trace data can include guard bands. The target trace data can be generated based on multiple sets of trace data. The target trace data can be generated based on data from multiple processing chambers. The target trace data can be generated based on data associated with multiple substrates. The target trace data can constitute and / or define a range of trace data values. The target trace data can constitute and / or define a range of trace data values ​​that satisfy one or more performance indicators. The target trace data can constitute and / or define a range of trace data values ​​likely to be associated with a manufacturing process that satisfies one or more performance indicators.

[0142] At block 424, the processing logic receives second trace data associated with a second processing chamber. The second processing chamber does not meet one or more performance indicators. The second processing chamber may not perform at a target level of energy efficiency, time efficiency, environmental impact, target profitability, etc. The processing logic may further receive second metrology data for a second substrate associated with the second trace data. The processing logic may further receive a second set of equipment constants associated with the second processing chamber. The processing logic may further receive third trace data associated with a third processing chamber. The third trace data and the third processing chamber may share one or more characteristics with the second trace data and the second processing chamber. The third processing chamber may not meet the same set of performance indicators as the second processing chamber. The third processing chamber may not meet a different set of performance indicators than the second processing chamber.

[0143] At block 426, the processing logic generates a first recommended corrective action associated with the second processing chamber. The first recommended corrective action is generated based on the target trace data and the second trace data. The first recommended corrective action includes updating one or more equipment constants of the second processing chamber. The first recommended corrective action can be generated in response to the second trace data differing from the target trace data. The first recommended corrective action can be generated in response to the second trace data differing from the target trace data by a target value, a target percentage, or the like. The first recommended corrective action can be generated in response to the second trace data including a value outside an acceptable range (e.g., a target number value). The acceptable range can be defined by, include, or be based on the target trace data, etc.

[0144] Generating the first recommended corrective action can be performed by further considering metrology data, e.g., metrology data for the first substrate and the second substrate. Generating the first recommended corrective action can be performed by further considering equipment constants, e.g., a first set of equipment constants and a second set of equipment constants. Generating the first recommended corrective action can be performed as part of generating a plurality of recommended corrective actions. The plurality of actions can include actions targeted at one or more optimization goals (e.g., minimizing environmental impact, maximizing process throughput, etc.). The plurality of actions can include updates to a plurality of equipment constants. The plurality of actions can include actions targeted at a plurality of processing chambers. Generating the first recommended corrective action can include generating a schedule for implementing the two or more recommended corrective actions. The corrective actions can be scheduled according to the updates. The corrective actions can be scheduled such that one or more manufacturing processes are performed between updates, e.g., to monitor differences in processing chamber performance due to the first update and then perform a second update. The corrective actions can be scheduled such that at least one substrate is processed by the processing chamber between corrective actions associated with the processing chamber. The corrective actions can be scheduled so that at least one substrate is processed by the processing chamber between updates to the equipment constants of the processing chamber. The multiple corrective actions can include corrective actions targeted to multiple processing chambers, such as one or more corrective actions associated with a second processing chamber and one or more corrective actions associated with a third processing chamber. The corrective actions can be utilized, such as when uniforming a group of processing chambers to perform according to one or more performance indicators.

[0145] Generating the first recommended corrective action can be performed by a trained model. Generating the first recommended corrective action can be performed by multiple models, an ensemble model, etc. Generating the first recommended corrective action can include operations performed by one or more statistical models, one or more rule-based models, one or more heuristic models, one or more machine learning models, etc.

[0146] Generating the first recommended corrective action (e.g., as part of multiple recommended corrective actions) may include providing the target trace data and the second trace data to a trained model (e.g., a trained machine learning model, an ensemble model, etc.). Generating the first recommended corrective action may further include receiving output from the trained model. The output may indicate one or more recommended corrective actions (e.g., including the first recommended corrective action). Generating the first recommended corrective action may further include scheduling execution of the first recommended corrective action.

[0147] At block 428, processing logic executes the first recommended corrective action.

[0148] 4D is a flow diagram of a method for adjusting equipment constants of chambers in a group of chambers, according to some embodiments. At block 430, a processing device receives data indicative of performance of a plurality of process chambers. The plurality of process chambers may be a group of process chambers. The plurality of process chambers may be of one or more processing tools. The plurality of process chambers may be included in one or more facilities, such as a manufacturing facility. The processing device may be, for example, a central server associated with the manufacturing facility.

[0149] At block 432, processing logic provides data indicative of the performance of the plurality of process chambers to a model. The model may be a trained machine learning model. The model may be a statistical model, a rule-based model, a heuristic model, a physics-based model, etc. The model may be an ensemble model, e.g., which may include one or more individual models, one or more trained machine learning models, multiple types of models, etc. The model may recommend a corrective action. The model may recommend an equipment constant update. The model may recommend an equipment constant update and / or a schedule for performing the corrective action. The model may update a previous schedule for performing the corrective action.

[0150] The data indicative of the performance of the plurality of process chambers can include trace data. The data indicative of the performance of the plurality of process chambers can include metrology data. The data indicative of the performance of the plurality of process chambers can include metrology data of substrates fabricated in a process chamber of the plurality of process chambers.

[0151] The data indicative of the performance of the multiple process chambers may include data associated with each of the multiple chambers. One or more chambers may be determined to be performing satisfactorily. One or more chambers may be determined to be satisfying one or more performance indicators. One or more chambers may be determined to be satisfying one or more performance indicator thresholds. The performance indicators may include trace data indicators, metrology indicators, energy usage indicators, environmental impact indicators, etc. One or more chambers satisfying a performance indicator may be referred to as golden chambers. One or more chambers satisfying a performance indicator may be referred to as golden chambers associated with those indicators. For example, a chamber satisfying a performance indicator associated with a gas flow system may be referred to as a golden chamber for the gas flow system indicator. Trace data from one or more chambers satisfying a performance indicator may be referred to as golden trace data. Trace data from one or more chambers satisfying a performance indicator may be used to generate golden trace data. The golden trace data may be associated with a performance standard, e.g., a performance level that the chamber should meet. The performance standard may include target metrology data, a range of target metrology data, etc. The performance standard may include target trace data, golden trace data, a range of trace data, etc. Data associated with one or more chambers that meet the performance indicators can be used to generate one or more standards of performance. A chamber can be considered to be operating acceptably if the chamber performance meets the one or more standards of performance. The chamber can have corrective actions associated with the chamber taken in response to the chamber performance not meeting the performance standards. A process chamber can trigger a corrective action if the process chamber performance does not meet the standard associated with the golden chamber.A process chamber can trigger a recommended corrective action when data indicative of the process chamber's performance (e.g., trace data, metrology data) does not meet one or more performance criteria. A corrective action can be recommended and / or implemented in association with a process chamber when the process chamber's performance differs from a performance standard, the performance of a target chamber, the performance of a golden chamber, etc. A corrective action can target one or more differences between the process chamber's performance and a performance indicator, a performance standard, the performance of another process chamber, etc. A corrective action can target reducing one or more differences between the performance of two process chambers, between a process chamber's performance indicator and a performance standard, etc.

[0152] At block 434, processing logic receives output from the model. The output includes a first recommended equipment constant update associated with a first process chamber of the plurality of process chambers. The output further includes a second recommended equipment constant update associated with a second process chamber of the plurality of process chambers.

[0153] At block 436, processing logic updates a first equipment constant for the first process chamber. The processing logic further updates a second equipment constant for the second process chamber. Updating the first and second equipment constants is performed taking into account the first recommended equipment constant update and the second recommended equipment constant update. In some embodiments, the first equipment constant and the second equipment constant can be the same constant for different process chambers. For example, the same calibration table associated with a particular system, subsystem, or chamber component can be updated for two process chambers. The updates can be the same or different for the process chambers. In some embodiments, the first equipment constant and the second equipment constant can be associated with different equipment constants, such as constants associated with different operations, characteristics, systems, and / or components of the process chambers.

[0154] 4E is a flow diagram of an example method 400E of performing corrective actions associated with one or more chambers in a group of chambers, according to some embodiments. FIG. 4E and the related description are meant to be illustrative rather than limiting and provide additional clarity to example applications related to the present disclosure.

[0155] At block 440, process operations are performed on multiple substrates using multiple process chambers. The process chambers may be a group of chambers. A process operation may be one or more process steps, may include multiple sub-operations, etc. A process operation may include operations performed in one or more process chambers, one or more types of process chambers, etc. In some embodiments, a target process operation may include operations performed between the time a substrate is introduced into a chamber and the time it is removed from the chamber. A process operation may be a small portion of the overall process for manufacturing a substrate.

[0156] At block 442, data indicative of the performance of multiple process chambers is received. The data can include trace data. The data can include metrology data. The data can include recipe data. The data can include equipment constants. This data can be used to identify one or more chambers that meet performance indicators, such as product metrology indicators or trace data indicators. This data can be used to identify one or more golden chambers associated with a target process operation. This data can be used to generate golden trace data. This data can be analyzed to standardize performance across a group of chambers, improve performance of a group of chambers, adjust chamber performance goals and / or indicators, etc. This data can be analyzed to determine chamber outliers. This data can be analyzed to determine one or more chambers are outliers with respect to performance, equipment constants, etc.

[0157] At block 444, differences between data associated with various process chambers are compared. For example, differences in trace data, metrology data, equipment constant data, etc. can be considered. The impact of data differences can be determined. Differences in metrology data can be related to differences in trace data. Differences in metrology and / or trace data can be related to and / or mitigated by differences in equipment constants or recipes. Determining correlations among equipment constants, recipes, trace data, and metrology data can be performed by a model. Determining correlations among data can be performed based on subject matter experts. Determining correlations among data can be performed by a trained machine learning model. For example, one chamber may produce substrates with one or more characteristics indicative of non-ideal performance of the pressure system, while another chamber may produce substrates with characteristics indicative of non-ideal performance of the radio frequency (RF) system. To change / improve performance of a chamber among multiple process chambers, associated equipment constants can be updated.

[0158] At block 446, corrective actions can be recommended and / or implemented. The corrective actions can be associated with one or more of the multiple process chambers. The corrective actions can include updating equipment constants for one or more process chambers. The equipment constants can be updated to reduce differences between chamber performance for multiple chambers in a group of chambers. The equipment constants can be updated to reduce differences between metrology data for processed substrates associated with different chambers. The equipment constants can be updated to reduce differences between trace data associated with different chambers. The equipment constants can be updated to different values ​​for different chambers, for example, based on chamber performance. The equipment constants can be updated to different values ​​for different chambers due to slight differences between chambers, such as aging components, manufacturing variations in components (e.g., within manufacturing tolerances), etc. Equipment constant updates can be scheduled, for example, making the lowest-risk changes first, making the changes most likely to be effective first, spacing the changes to allow for processing substrates between equipment constant updates and analyzing the data associated with those substrates, etc. The corrective actions can target outliers. Corrective actions can be targeted to chambers that are outliers in terms of performance, equipment constants, and the like.

[0159] FIG. 5A is a block diagram illustrating a system 500A for performing operations associated with updating equipment constants of a process chamber, according to some embodiments. It will be appreciated that system 500A is an exemplary system, and other systems including different data flows are within the scope of this disclosure. System 500A includes two trained machine learning models. System 500A includes a golden trace generation model 506 and a corrective action recommendation model 510. In some embodiments, the operations of one or both of these models can be performed by a physics-based model, a statistical model, a rule-based model, etc. These model operations of system 500A can also be performed by more or fewer models. For example, golden trace generation model 506 and corrective action recommendation model 510 can be combined into a single ensemble model.

[0160] Golden chamber trace data 504 and golden chamber metrology data 502 are provided to a golden trace generation model 506. The golden chamber trace data 504 and golden chamber metrology data 502 can be associated with one or more golden chambers. A golden chamber can be a chamber that produced an acceptable product, e.g., a product that meets a performance threshold. A golden chamber can be a chamber that met a performance threshold for a period of time, and the golden data from the chamber can be from the period during which the chamber met the performance threshold. A golden chamber can be a chamber that met a performance threshold with a target likelihood, e.g., a target percentage of substrates processed by the chamber that met the performance threshold. The golden chamber trace data 504 can include trace data from the golden chamber, trace data from the golden chamber while the golden chamber was processing a product that met the performance threshold, etc. The golden chamber metrology data 502 can include metrology data of a product processed by the golden chamber. The golden chamber metrology data 502 can include metrology data of a product that meets the performance threshold. The golden chamber metrology data 502 and the golden chamber trace data 504 can be associated with the same set of products. The golden trace generation model 506 can be or include a machine learning model. The golden trace generation model 506 can be or include a physics-based model. The golden trace generation model 506 can be or include a heuristic model. The golden trace generation model 506 can be or include a rule-based model. The golden trace generation model 506 can be or include a statistical model.

[0161] The golden trace generation model 506 can be configured to generate golden trace data 508. The golden trace data 508 can include data associated with one or more processed products. The golden trace data 508 can include data associated with one or more sensors of manufacturing equipment. The golden trace data 508 can include a single trace for one sensor; for example, the golden trace data 508 can represent an “ideal” or “best” trace from the provided golden chamber trace data 504. The golden trace data 508 can include multiple traces for one sensor; for example, the golden trace data 508 can include upper and lower golden traces. Trace data (e.g., from other process chambers) that falls within the boundaries of the upper and lower golden trace data is not considered anomalous. The upper and lower golden traces can define upper and lower limits, guard bands, etc. The golden trace data 508 can include data from multiple processing runs, multiple process chambers, etc. Selecting traces for the golden trace data 508 may include extracting correlations between the trace data and metrology data, for example, mapping the effect of the trace data on the metrology data. Selecting traces for the golden trace data 508 may include selecting trace data associated with products that exhibit acceptable metrology indicators of a type that correlates with the selected trace data. The golden trace data may be measured data or synthetic data. The synthetic golden trace data may be generated by subject matter expertise. The synthetic golden trace data may be generated by a model. The synthetic golden trace data may be generated by a machine learning model, such as a recurrent neural network. The synthetic golden trace data may be generated by a statistical or heuristic model. The synthetic golden trace data may be generated by a physics-based model.The synthetic golden trace data can be generated by a digital twin model (e.g., a virtual representation of a physical piece of equipment such as a manufacturing chamber).

[0162] The golden trace data 508 is provided to a corrective action recommendation model 510. The corrective action recommendation model 510 can recommend a corrective action, schedule the execution of the corrective action, cause the execution of the corrective action, etc. The corrective action recommendation model 510 can generate as an output data including a recommended corrective action 520. The corrective action recommendation model 510 can recommend updates to one or more equipment constants of one or more process chambers in a bank of process chambers.

[0163] The corrective action recommendation model 510 can receive additional inputs. The model can receive fleet trace data 516. The model can receive golden chamber equipment constants 518. The model can receive fleet equipment constants 514. The model can receive fleet metrology 512. The model can receive golden chamber metrology data 502. The fleet metrology 512 and golden chamber metrology data 502 can optionally be provided to the corrective action recommendation model 510. In some embodiments, the corrective action recommendation model 510 can be configured to recommend corrective actions to increase the similarity between the processed product and the product represented in the golden metrology data. In some embodiments, the corrective action recommendation model 510 can be configured to recommend corrective actions to increase the similarity between the trace data of one or more chambers in a fleet of process chambers and the golden trace data.

[0164] 5B is a block diagram illustrating the operation of a corrective action recommendation model 530, according to some embodiments. The corrective action recommendation model 530 may be the corrective action recommendation model 510 of FIG. 5A. The corrective action recommendation model 530 may be a single model, a family of models, an ensemble model, etc. The corrective action recommendation model 530 may include one or more machine learning models, heuristic models, rule-based models, statistical models, etc. In some embodiments, some operations of the corrective action recommendation model 530 may be performed by a user and / or a subject matter expert.

[0165] The corrective action recommendation model 530 includes a trace-to-measurement correlation 532. The trace-to-measurement correlation 532 can include one or more models. The trace-to-measurement correlation 532 can identify relationships between measurement outputs of a manufacturing process and trace data. The trace-to-measurement correlation 532 can identify causal relationships between trace sensor data and measurement data. The trace-to-measurement correlation 532 can predict measurement data based on the trace data. Similar models can also be applied to other metrics, such as correlating trace data to environmental impact, energy use, throughput, etc.

[0166] The corrective action recommendation model 530 includes equipment constant and trace correlations 534. Parameter and trace correlations can include correlating manufacturing parameters with trace data. Parameter and trace correlations 534 can include determining cause and effect relationships between manufacturing parameters and trace data. Parameter and trace correlations 534 can include determining the impact of strategies, set points, equipment constants, equipment components, etc. on the trace data. Similar models can also be applied to other metrics, such as correlating parameter data with environmental impacts, energy use, throughput, etc.

[0167] The corrective action recommendation model 530 includes chamber differentiation 536. The chamber differentiation 536 can include determining differences between chambers, tools, processing equipment, etc. The chamber differentiation 536 can determine how different chambers differ in response to changing parameters, equipment constants, trace data, etc. The chamber differentiation 536 can, for example, allow the corrective action recommendation model 530 to compensate for differences in installed components, variations in manufacturing tolerances of components of manufacturing equipment, differences in chamber age, etc.

[0168] The corrective action recommendation model 530 includes a parameter update schedule 538. The parameter update schedule 538 can perform operations directed to determining where, when, under what conditions, etc., to perform parameter updates. The parameter update schedule 538 can include scheduling updates to equipment constants. Updating equipment constants (e.g., as opposed to updating a recipe) can allow the same recipe to be performed on multiple chambers, with chamber differences being compensated for by applying equipment constant updates on a chamber-by-chamber basis. The parameter update schedule 538 can include determining the risk of a parameter change (e.g., how likely a parameter change is to adversely affect process performance). The parameter update schedule 538 can include determining the effectiveness of a parameter change (e.g., how likely a parameter change is to have an intended impact on trace data, metrology data, or another output indicator). The parameter update schedule 538 can include scheduling conditional updates, e.g., some updates can be scheduled to be performed conditionally over a period of time with respect to trace data or another output indicator. The parameter update schedule 538 can include selecting different updates for different chambers. After multiple processing runs, the data can be referenced to determine the effectiveness of various updates, schedule additional updates, and the like.

[0169] The corrective action recommendation model 530 can include additional components. The corrective action recommendation model 530 can include fewer components. The corrective action recommendation model 530 can be configured to recommend a corrective action. The corrective action recommendation model 530 can be configured to recommend and / or implement parameter updates. The corrective action recommendation model 530 can be configured to recommend and / or implement equipment constant updates. The corrective action recommendation model 530 can be configured to perform chamber matching, fleet matching, and / or process optimization procedures.

[0170] 6 is a block diagram illustrating a computer system 600, according to some embodiments. In some embodiments, computer system 600 can be connected to other computer systems (e.g., via a network such as a local area network (LAN), an intranet, an extranet, or the Internet). Computer system 600 can operate in the capacity of a server or a client computer in a client-server environment, or as a peer computer in a peer-to-peer or distributed network environment. Computer system 600 can be provided by a personal computer (PC), a tablet PC, a set-top box (STB), a personal digital assistant (PDA), a cellular phone, a web appliance, a server, a network router, switch, or bridge, or any device capable of executing (sequentially or otherwise) a set of instructions that specify actions to be taken by that device. Furthermore, the term "computer" is intended to include any group of computers that individually or collectively execute one or more sets of instructions to perform any one or more of the methods described herein.

[0171] In a further aspect, computer system 600 may include a processing device 602, a volatile memory 604 (e.g., random access memory (RAM)), a non-volatile memory 606 (e.g., read-only memory (ROM) or electrically erasable programmable ROM (EEPROM)), and a data storage device 618, which may communicate with each other via a bus 608.

[0172] The processing device 602 may be provided by one or more processors, such as a general-purpose processor (e.g., a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a microprocessor implementing other types of instruction sets, or a microprocessor implementing a combination of instruction set types), or a specialized processor (e.g., an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), or a network processor).

[0173] Computer system 600 may further include a network interface device 622 (e.g., coupled to a network 674). Computer system 600 may also include a video display unit 610 (e.g., an LCD), an alphanumeric input device 612 (e.g., a keyboard), a cursor control device 614 (e.g., a mouse), and a signal generating device 620.

[0174] In some embodiments, the data storage device 618 may include a non-transitory computer-readable storage medium 624 (e.g., a non-transitory machine-readable medium) capable of storing instructions 626 encoding any one or more of the methods or functions described herein, the instructions 626 encoding the components of FIG. 1 (e.g., the prediction component 114, the corrective action component 122, the model 190, etc.) and including instructions for performing the methods described herein.

[0175] The instructions 626 may also reside, completely or partially, within the volatile memory 604 and / or the processing device 602 during execution thereof by the computer system 600, and thus the volatile memory 604 and the processing device 602 may also constitute machine-readable storage media.

[0176] While the illustrative example shows computer-readable storage medium 624 as a single medium, the term "computer-readable storage medium" is intended to include a single medium or multiple media (e.g., centralized or distributed databases and / or associated caches and servers) that store one or more executable sets of instructions. The term "computer-readable storage medium" is also intended to include any tangible medium that is capable of storing or encoding a set of instructions, for execution by a computer, that cause a computer to perform any one or more of the methodologies described herein. The term "computer-readable storage medium" is intended to include, but is not limited to, solid-state memory, optical media, and magnetic media.

[0177] The methods, components, and features described herein can be implemented by discrete hardware components or integrated into the functionality of other hardware components, such as ASICS, FPGAs, DSPs, or similar devices. In addition, the methods, components, and features can be implemented by firmware modules or functional circuits within a hardware device. Furthermore, the method components and features can be implemented in any combination of hardware devices and computer program components, or can be implemented within a computer program.

[0178] Unless specifically stated otherwise, terms such as "receive," "execute," "provide," "obtain," "cause," "access," "determine," "add," "use," "train," "reduce," "generate," "correct," and the like refer to actions and processes performed or implemented by a computer system that manipulate and transform data represented as physical (electronic) quantities in computer system registers and memory into other data similarly represented as physical quantities in computer system memory or registers or other such information storage, transmission, or display devices. Also, as used herein, terms such as "first," "second," "third," "fourth," and the like are meant to be labels for distinguishing different elements and may not have an ordering meaning according to numerical designations.

[0179] The examples described herein also relate to apparatus for performing the methods described herein. This apparatus may be specially constructed to perform the methods described herein, or may comprise a general-purpose computer system selectively programmed by a computer program stored in the computer system. Such a computer program may be stored in a computer-readable tangible storage medium.

[0180] The methods and illustrative examples described herein are not inherently related to any particular computer or other apparatus. Various general-purpose systems can be used in accordance with the teachings described herein, or it may prove convenient to construct more specialized apparatus to perform the methods and / or each of the individual functions, routines, subroutines, or operations described herein. Example structures for a variety of these systems have been set forth in the description above.

[0181] The above description is intended to be illustrative, not limiting. While the present disclosure has been described with reference to specific illustrative examples and embodiments, it will be understood that the present disclosure is not limited to the described examples and embodiments. The scope of the present disclosure should be determined with reference to the following claims, along with the full scope of equivalents to which such claims are entitled.

Claims

1. As input to the first trained machine learning model, trace data associated with one or more substrate processing procedures that resulted in substrates satisfying one or more criteria; equipment constants associated with the one or more substrate processing procedures; trace data for the first processing chamber; and providing equipment constants for the first processing chamber; obtaining, as output from the first trained machine learning model, a recommended update to a first equipment constant of the first process chamber; updating the first equipment constants of the first processing chamber in response to obtaining the output from the first trained machine learning model; and A method comprising:

2. 10. The method of claim 1, wherein the input to the first trained machine learning model further comprises metrology data for one or more substrates processed in the processing chamber.

3. providing metrology data of a first one or more substrates processed in a processing chamber and trace data associated with the first one or more substrates processed in the processing chamber as input to a second trained machine learning model; obtaining the trace data associated with the one or more substrate processing procedures as output from the second trained machine learning model; The method of claim 1 further comprising:

4. 4. The method of claim 3, further comprising providing metrology data of a second one or more substrates processed in a processing chamber as input to the second trained machine learning model, the second one or more substrates being processed after the first one or more substrates, and wherein the metrology data of the second one or more substrates is weighted more heavily than the metrology data of the first one or more substrates when generating output by the second trained machine learning model.

5. 10. The method of claim 1, further comprising obtaining a recommended update to a second equipment constant of the first processing chamber as output from the first trained machine learning model; and scheduling the update of the second equipment constant, wherein the update of the second equipment constant is scheduled to occur after updating the first equipment constant.

6. providing trace data of a second processing chamber and equipment constants of the second processing chamber as inputs to the first trained machine learning model; obtaining, as output from the first trained machine learning model, recommended updates to second equipment constants of the second processing chamber; and scheduling an update of the second equipment constant of the second processing chamber, wherein the update of the second equipment constant of the second processing chamber is scheduled to occur after updating the first equipment constant. The method of claim 1.

7. The method of claim 1 , wherein the trace data associated with the one or more substrate processing procedures includes an upper limit trace and a lower limit trace.

8. The upper limit trace is The trace containing the highest value among multiple traces, a trace representing a statistical upper bound on said plurality of traces; or The method of claim 7 , including one or more of the traces representing expected upper bounds on traces associated with the plurality of traces.

9. 1. A system comprising a memory and a processing device coupled to the memory, the processing device comprising: As input to the first trained machine learning model, trace data associated with one or more substrate processing procedures that resulted in substrates satisfying one or more criteria; equipment constants associated with the one or more substrate processing procedures; trace data for the first processing chamber; and providing equipment constants for the first processing chamber; obtaining, as output from the first trained machine learning model, a recommended update to a first equipment constant of the first process chamber; and updating the first equipment constants of the first processing chamber in response to obtaining the output from the first trained machine learning model.

10. 10. The system of claim 9, wherein the input to the first trained machine learning model further comprises metrology data for one or more substrates processed in the processing chamber.

11. the processing device further comprising: providing metrology data of a first one or more substrates processed in a processing chamber and trace data associated with the first one or more substrates processed in the processing chamber as input to a second trained machine learning model; and obtaining the trace data associated with the one or more substrate processing procedures as output from the second trained machine learning model.

12. 12. The system of claim 11 , wherein the processing device is further for providing metrology data of a second one or more substrates processed in a processing chamber as input to the second trained machine learning model, the second one or more substrates being processed after the first one or more substrates, and wherein the metrology data of the second one or more substrates is weighted more heavily than the metrology data of the first one or more substrates when generating output by the second trained machine learning model.

13. 10. The system of claim 9, wherein the processing device is further configured to obtain, as output from the first trained machine learning model, a recommended update to a second equipment constant of the first processing chamber; and schedule the update of the second equipment constant, wherein the update of the second equipment constant is scheduled to occur after updating the first equipment constant.

14. the processing device further comprising: providing trace data of a second processing chamber and equipment constants of the second processing chamber as inputs to the first trained machine learning model; obtaining, as output from the first trained machine learning model, recommended updates to second equipment constants of the second processing chamber; 10. The system of claim 9, wherein the updating of the second equipment constant of the second processing chamber is scheduled to occur after updating the first equipment constant.

15. The system of claim 9 , wherein the trace data associated with the one or more substrate processing procedures includes an upper limit trace and a lower limit trace.

16. The upper limit trace is The trace containing the highest value among multiple traces, a trace representing a statistical upper bound on said plurality of traces; or The system of claim 15 , including one or more of the traces representing expected upper bounds for traces associated with the plurality of traces.

17. A non-transitory machine-readable storage medium storing instructions that, when executed, cause a processing device to perform operations, the operations including: As input to the first trained machine learning model, trace data associated with one or more substrate processing procedures that resulted in substrates satisfying one or more criteria; equipment constants associated with the one or more substrate processing procedures; trace data for the first processing chamber; and providing equipment constants for the first processing chamber; obtaining, as output from the first trained machine learning model, a recommended update to a first equipment constant of the first process chamber; and updating the first equipment constants of the first processing chamber in response to obtaining the output from the first trained machine learning model.

18. The operation is providing metrology data of a first one or more substrates processed in a processing chamber and trace data associated with the first one or more substrates processed in the processing chamber as input to a second trained machine learning model; and obtaining the trace data associated with the one or more substrate processing procedures as output from the second trained machine learning model.

19. 20. The non-transitory machine-readable storage medium of claim 18, wherein the operations further include providing metrology data of a second one or more substrates processed in a processing chamber as input to the second trained machine learning model, the second one or more substrates being processed after the first one or more substrates, and wherein the metrology data of the second one or more substrates is weighted more heavily than the metrology data of the first one or more substrates when generating output by the second trained machine learning model.

20. 20. The non-transitory machine-readable storage medium of claim 17, wherein the operations further include obtaining a recommended update to a second equipment constant of the first processing chamber as output from the first trained machine learning model and scheduling an update of the second equipment constant, wherein the update of the second equipment constant is scheduled to occur after updating the first equipment constant.

Images