Adjusting chamber performance by equipment constant update

Abstract

A method includes receiving, by a processing device, first trajectory data associated with a first processing chamber, wherein the first processing chamber satisfies one or more performance metrics. The method further includes generating target trajectory data based on first trajectory data associated with the first processing chamber. The method further includes receiving second trajectory data associated with a second processing chamber, wherein the second processing chamber does not satisfy one or more performance metrics. The method further includes generating a first recommended corrective action associated with the second processing chamber based on the target trajectory data and the second trajectory data, wherein the first recommended corrective action includes updating one or more equipment constants of the second processing chamber. The method further includes performing a first recommended corrective action.

Description

Technical Field

[0001] The present disclosure relates to methods associated with machine learning models for evaluating manufactured devices (e.g., semiconductor devices). More specifically, the present disclosure relates to methods for generating and utilizing equipment constant updates to improve or standardize the performance of a manufacturing chamber. Background Art

[0002] Products can be produced by performing one or more manufacturing processes using manufacturing equipment. For example, semiconductor manufacturing equipment can be used to produce substrates via semiconductor manufacturing processes. The products produced have specific properties suitable for the target application. Machine learning models are used for various process control and prediction functions associated with manufacturing equipment. Data associated with the manufacturing equipment is used to train the machine learning models. Changes can be made to process recipes, process chambers, process procedures, etc. to improve the properties of the products produced. Summary of the Invention

[0003] The following is a simplified overview of the present disclosure to provide a basic understanding of some aspects of the present disclosure. This overview is not an extensive overview of the present disclosure. It is neither intended to identify key or important elements of the present disclosure nor to describe any scope of the specific embodiments of the present disclosure or any scope of the claims. Its sole purpose is to present some concepts of the present disclosure in a simplified form as a prelude to the more detailed description that follows.

[0004] In one aspect of the present disclosure, a method includes receiving, by a processing device, first trajectory data associated with a first processing chamber, wherein the first processing chamber meets one or more performance metrics. The method further includes generating target trajectory data based on the first trajectory data associated with the first processing chamber. The method further includes receiving second trajectory data associated with a second processing chamber, wherein the second processing chamber does not meet one or more performance metrics. The method further includes generating a first recommended corrective action associated with the second processing chamber based on the target trajectory data and the second trajectory data, wherein the first recommended corrective action includes updating one or more equipment constants of the second processing chamber. The method further includes performing the first recommended corrective action.

[0005] In another aspect of the present disclosure, a system includes a memory and a processing device coupled to the memory. The processing device is configured to receive first trajectory data associated with a first processing chamber, where the first processing chamber meets one or more performance metrics. The processing device is further configured to generate target trajectory data based on the first trajectory data associated with the first processing chamber. The processing device is further configured to receive second trajectory data associated with a second processing chamber, where the second processing chamber does not meet one or more performance metrics. The processing device is further configured to generate a first recommended corrective action associated with the second processing chamber based on the target trajectory data and the second trajectory data, where the first recommended corrective action includes updating one or more equipment constants of the second processing chamber. The processing device is further configured to execute the first recommended corrective action.

[0006] In another aspect of the present disclosure, a non-transitory machine-readable storage medium stores instructions. When the instructions are executed, they cause a processing device to perform operations. The operations include receiving first trajectory data associated with a first processing chamber, where the first processing chamber meets one or more performance metrics. The operations further include generating target trajectory data based on the first trajectory data associated with the first processing chamber. The operations further include receiving second trajectory data associated with a second processing chamber, where the second processing chamber does not meet one or more performance metrics. The operations further include generating a first recommended corrective action associated with the second processing chamber based on the target trajectory data and the second trajectory data, where the first recommended corrective action includes updating one or more equipment constants of the second processing chamber. The operations further include executing the first recommended corrective action. Brief Description of the Drawings

[0008] The present disclosure is illustrated by way of example and not limitation in the figures of the accompanying drawings.

[0009] Figure 1 A block diagram illustrating an exemplary system architecture according to some embodiments.

[0010] Figure 2A A block diagram depicting an example data set generator for establishing a data set for one or more supervised models according to some embodiments.

[0011] Figure 2B A block diagram depicting an example data set generator for establishing a data set for one or more unsupervised models according to some embodiments.

[0012] Figure 3 A block diagram illustrating a system for generating output data according to some embodiments.

[0013] Figure 4AFlowchart of a method for generating a dataset for a machine learning model according to some embodiments.

[0014] Figure 4B Flowchart of a method for updating equipment constants of a processing chamber according to some embodiments.

[0015] Figure 4C Flowchart of a method for performing a calibration action associated with a processing chamber according to some embodiments.

[0016] Figure 4D Flowchart of a method for adjusting equipment constants of chambers in a chamber group according to some embodiments.

[0017] Figure 4E Flowchart of an example method for performing a calibration action associated with one or more chambers in a chamber group according to some embodiments.

[0018] Figure 5A Block diagram depicting a system for performing operations associated with updating equipment constants of a processing chamber according to some embodiments.

[0019] Figure 5B Block diagram depicting the operation of a calibration action recommendation model 530 according to some embodiments.

[0020] Figure 6 Block diagram illustrating a computer system according to some embodiments. Detailed Description

[0021] Techniques related to increasing the performance of manufacturing equipment by updating equipment constants are described herein. The manufacturing equipment is used to produce products (e.g., substrates (e.g., wafers, semiconductors)). The manufacturing equipment may include one or more manufacturing or processing chambers to separate the substrate from the environment. The properties of the produced substrate should conform to target values to facilitate a specific function. Manufacturing parameters are selected to produce a substrate that conforms to the target property values. Many manufacturing parameters (e.g., hardware parameters, processing parameters, etc.) contribute to the properties of the processed substrate.

[0022] Manufacturing parameters as used herein include process recipes and equipment constants. A process recipe includes parameters selected to produce a process result (e.g., effecting a process on a substrate characterized by one or more target properties). A process recipe may include parameters selected and / or adjusted based on product design, target output, target substrate metrics, etc. A process recipe may include parameters (e.g., process temperature, process pressure, process gases, radio frequency (RF) radiation properties, plasma properties, etc.). Equipment constants include parameters associated with the operation of manufacturing equipment. Equipment constants may include parameters that support the implementation of a process recipe. Equipment constants may not be associated with a specific process, recipe, substrate design, target property, etc. Equipment constants may be associated with a set of manufacturing equipment, processing tools, processing chambers, one or more components, etc. Equipment constants may include control settings (e.g., settings of voltage or current applied to a component to achieve a target output (e.g., as defined by a process recipe)). Equipment constants may include operating settings (e.g., settings of the operation of a component not directly related to the process recipe output (e.g., transfer robot speed, voltage applied to a component to operate the component, etc.)).

[0023] Equipment constants may include independent values (e.g., speed for the operation of a pump, acceptable pressure considering the chamber has been evacuated or emptied, etc.). Equipment constants may include tables of values (e.g., tables associating input settings (e.g., process recipe inputs) with actions (e.g., voltage applied to a component to achieve the input)). Equipment constants may include functions (e.g., functions that can be used to calculate appropriate actions for target conditions (e.g., as defined by a process recipe)). Equipment constants may include calibration tables and / or calibration constants (e.g., adjustments to standard or factory settings for a component). Equipment constants may include constants associated with one or more controllers. Equipment constants may include parameters associated with a proportional integral derivative (PID) controller (e.g., parameters that determine the effect on the controller output based on the controller input).

[0024] Equipment constants may form the basis for many operations of manufacturing equipment. Equipment constants may include parameters that control robot movement, chamber pressurization, chamber pumping, gas flow and mixing, temperature control, plasma generation, substrate fixation mechanisms, in-chamber metrology systems, and any other operations performed by the manufacturing system.

[0025] A manufacturing system can control processing conditions (e.g., conditions in a processing chamber) by specifying set points for property values, receiving data from sensors disposed within the manufacturing chamber, and adjusting the manufacturing equipment until the sensor readings match the set points. In some embodiments, the set points can be defined by a processing recipe (e.g., a processing temperature can be defined and a temperature sensor can be maintained at the defined temperature). In some embodiments, the set points can be defined by equipment constants (e.g., a target processing temperature can be defined and a table of equipment constants that correlates the target temperature (e.g., the temperature at a location not directly sensed by a temperature sensor) with the temperature readings of one or more sensors in the processing chamber can be referenced). The power supplied to one or more heaters can be adjusted to maintain the temperature at the sensor associated with the set point temperature and the equipment constant table.

[0026] Processing results can vary among groups of manufacturing equipment, tools, facilities, chambers, etc. Specific tools, chambers, etc. can produce acceptable (meeting specific conditions or requirements) products (e.g., substrates within a target property value range can be generated more frequently than by other equipment). Equipment can meet one or more conditions by frequently producing acceptable products within a time range (e.g., after preventive or corrective maintenance, after drying, after installation, etc.). Such equipment can be referred to as “golden” equipment. Here, for simplicity, the phrase “golden chamber” will be used, but golden equipment can include golden tools, golden equipment groups, golden manufacturing facilities, etc.

[0027] A golden trace can include trace data associated with one or more processing operations (e.g., operations that produce acceptable products (e.g., substrates meeting target performance thresholds, target metrology values, etc.)). The golden trace can be obtained from sensors in the golden chamber. Golden trace data can be collected during product processing in the golden chamber. By associating one or more acceptable products (e.g., substrates achieving target property values) with one or more golden chambers, the golden trace can indicate sensor data measured during processing.

[0028] In traditional systems, the operations of processing tools, chambers, facilities, etc. can be changed and / or adjusted to improve performance. The operations can be adjusted so that the production of products can meet target performance metrics (e.g., target metrology values). The operations can be adjusted to increase the likelihood of producing products that meet target performance metrics. The operations can be adjusted to increase the efficiency of the manufacturing system (e.g., in terms of materials consumed, time used, energy consumed, etc.). The operations can be adjusted to reduce the cost per acceptable product (e.g., including reducing the cost of setting up defective products, analyzing defective products, etc.).

[0029] In a conventional system, the operation of a manufacturing tool can be adjusted to achieve a closer alignment between the trace data collected from the manufacturing tool and the golden trace data. For example, the operation of a processing chamber can be adjusted to cause the trace data to be more closely aligned with the golden trace data. The operation of the manufacturing tool can be adjusted to make the manufacturing tool more similar to the golden chamber.

[0030] In a conventional system, the operation of a manufacturing tool can be adjusted by changing the processing recipe. The processing recipe can be adjusted to improve product properties. The processing recipe can be adjusted to improve the trace data (e.g., to cause the trace data to be similar to the golden trace data). The processing recipe can be adjusted to improve the manufacturing tool (e.g., to cause the performance of the processing chamber to be similar to the golden chamber).

[0031] Conventional methods have several drawbacks. Adjusting the performance of a set of manufacturing tools (e.g., processing chambers) to improve product properties is a low - efficiency process. Updates can be made (e.g., to processing recipe parameters), substrates can be processed according to the updates, and metrology operations can be performed on the substrates. The relationship between recipe input and product output may be non - linear and may not be one - to - one (e.g., one recipe ingredient may affect multiple substrate properties), etc. The performance of the manufacturing tool may also be related to aging or faulty components, which can be improved through preventive or corrective maintenance, etc. Difficulty in determining the appropriate actions to improve product performance may lead to an increase in the time to correct the tool performance. This may result in an increase in the tool's downtime, a decrease in productivity, a decrease in yield, etc. Determining corrective actions based on product performance may include an increase in the cost of performing metrology (e.g., at an independent metrology facility). Determining corrective actions based on product performance may include performing many processing runs (e.g., processing many products), increasing the consumption of energy, materials, and time, increasing the cost associated with discarding defective products, increasing the wear, aging, and / or drift of components, reducing the tool productivity compared to non - production time (e.g., reducing the green time of the chamber), etc.

[0032] There are also disadvantages to adjusting the processing recipe to a target match (e.g., more similar) of the trace data and the golden trace data. The trace data is affected by many aspects of the manufacturing process. The trace data is affected by the processing recipe. The trace data is affected by the equipment health (e.g., component aging, drift, etc. may affect the chamber performance reflected in the trace data). The trace data may be affected by equipment constants. Adjusting the processing recipe to a target match of the trace data and the golden trace data utilizes a subset of the available adjustable parameters (e.g., processing knobs) to improve the performance of the manufacturing equipment. Adjusting the processing recipe may not be as effective as other trace data matching methods. Compared with other trace data matching methods, adjusting the processing recipe may result in increased energy use, increased material use, reduced processing efficiency, etc. Compared with matching the trace data via another method, matching the trace data by adjusting the processing recipe may cause the components of the processing equipment to operate more frequently and roughly in a way that generates more pressure on the components. For example, the temperature trace can be matched by increasing the power supplied to the heater, which may create harsher conditions for one or more components of the processing chamber, may reduce the service life of one or more components of the processing chamber, may increase the drift and / or aging of one or more components of the processing chamber, may increase the overall energy use of the processing chamber, etc. Reducing the service life of one or more components of the processing chamber may increase the frequency of calibration and / or preventive maintenance, increase chamber downtime, reduce chamber green time, etc. Reducing the service life of one or more components may increase costly unplanned downtime, increase the costs associated with replacing components (including component costs, express shipping costs), etc.

[0033] Aspects of the present disclosure can address one or more of these disadvantages using conventional solutions. The equipment constants of the manufacturing equipment can be adjusted. Here, the equipment constants generally described as referring to a chamber (e.g., a golden chamber) are used to update the equipment constants of another chamber. Implementations that utilize or update the equipment constants of tools, facilities, tool groups (e.g., groups of similar tools), chamber groups (e.g., several similarly performing chambers of a tool), etc. are also applicable. The updated chamber can be part of the same tool as the golden chamber or part of a different tool. The updated chamber can be located in the same facility as the golden chamber or in a different facility.

[0034] The equipment constants are different from the processing recipe parameters. The processing recipe parameters are typically related to setpoint values of one or more properties during processing. The processing recipe parameters can include property value setpoints (e.g., target temperature, target pressure, etc.), time (e.g., the time span for maintaining the target temperature), the evolution of the setpoint over time (e.g., temperature ramp-up), etc. The equipment constants are typically settings that affect the operation of the manufacturing equipment. The equipment constants can include non-processing setpoints (e.g., transfer robot speed, gas flow rate for venting the lock, number of processing operations between performing an automatic chamber clean and / or dry operation, boundary of leak check results for generating a warning, etc.). The equipment constants can include settings relevant during processing (e.g., boundary of the pressure of the chamber pumping system for triggering a warning, gas control parameters for delivering a processing gas to the processing chamber, etc.). The equipment constants can include calibration tables (e.g., correlating setpoint values with control signals).

[0035] In some embodiments, one or more equipment constants of a gold chamber (or, as the case may be, a gold tool, a gold facility, etc.) can be applied to a processing chamber (e.g., a poorly performing chamber). In some embodiments, the equipment constants can be classified according to the risk posed by updating the equipment constants. For example, equipment constants for which a change is also unlikely to have a serious negative impact on the process (e.g., transfer robot speed, inert gas flow rate for venting the lock, etc.) can be regarded as low-risk equipment constants. Equipment constants for which a change may have a negative impact on the process (e.g., radio frequency (RF) control, plasma generation control, processing gas mixing and delivery, pressure and / or temperature control, etc.) can be regarded as high-risk. The risk of the equipment constants can be evaluated by subject matter experts, statistical models, physics-based models, machine learning models, etc. Actions for updating one or more equipment constants of the processing chamber can be taken based on the evaluated risk.

[0036] In some embodiments, one or more machine learning models can be utilized to determine corrective updates to the equipment constants. The machine learning models can be used to evaluate the impact of one or more equipment constants (e.g., impact on product performance, impact on trajectory data, impact on product variability, etc.). The machine learning models can be used to determine target equipment constants (including indications of equipment constants and equipment performance) from input data. The machine learning models can be used to determine outlier equipment constants (e.g., outlier calibration tables). The machine learning models can be used to determine the optimal time to perform maintenance (e.g., perform manual calibration) on the manufacturing equipment. The machine learning models can cause operations associated with updating the equipment constants to be performed (e.g., the machine learning models can schedule, initiate, etc., calibration operations). In some embodiments, a statistical model, a physics-based model, or another type of model can be utilized to perform one or more of these operations instead of the machine learning models.

[0037] In some embodiments, a machine learning model can be utilized to generate a golden trajectory. Trajectory data can be provided to the machine learning model as data input. Metering data of a product associated with the trajectory data can be further provided to the machine learning model as data input. Data from one or more golden chambers can be provided to the machine learning model. The machine learning model can be configured to generate one or more golden trajectories. In some embodiments, the machine learning model can generate a lower boundary and an upper boundary of the golden trajectory. In some embodiments, the machine learning model can generate a golden trajectory associated with multiple chamber sensors, multiple measurement conditions (e.g., temperature, pressure, etc.), and the like.

[0038] In some embodiments, the machine learning model can generate a recommended update for one or more equipment constants based on the golden trajectory. The machine learning model can receive one or more golden trajectories as input. The golden trajectory can be generated by another machine learning model. The golden trajectory can include an upper boundary and a lower boundary of the golden trajectory. The golden trajectory can include a trajectory associated with multiple processing conditions, multiple sensors, and the like. The machine learning model can further receive equipment constants from one or more processing chambers (e.g., chambers that are not golden chambers, chambers that do not generate golden trajectory data, underperforming chambers, etc.). The machine learning model can further receive trajectory data (e.g., trajectory data associated with the same processing chamber as the received equipment constants). The machine learning model can further receive metering data (e.g., metering data associated with the same processing chamber as the received equipment constants). The machine learning model can generate a recommended change for the equipment constants of one or more chambers as output. The machine learning model can generate a schedule for updating the equipment constants (e.g., the recommended order for updating the equipment constants) to facilitate monitoring the impact of changes in the equipment constants. The machine learning model can generate one or more recommendations for maintenance operations (e.g., calibration, cleaning, drying, etc.). The machine learning model can initiate one or more maintenance operations. The machine learning model can perform optimization operations (e.g., recommend updates to the equipment constants for improving chamber efficiency, improving chamber performance, improving energy efficiency, improving material consumption, etc.).

[0039] Aspects of the present disclosure provide technical advantages over conventional solutions. Greater processing change space can be achieved by updating equipment constants compared to updating a processing recipe. Chamber adjustments can be performed to improve uniformity between different chambers. The uniformity and / or performance of the chamber can be improved across many different processing recipes, processing operations, processing types, etc. The equipment constant updates can be based on information from various sources (e.g., subject matter experts, physics-based models, statistical models, machine learning models, optimization algorithms, etc.). The equipment constant updates can be made in conjunction with specific results (e.g., given the relationship between one or more equipment constants and one or more substrate performance metrics).

[0040] In one aspect of the present disclosure, a method includes receiving, by a processing device, first trajectory data associated with a first processing chamber, wherein the first processing chamber meets one or more performance metrics. The method further includes generating target trajectory data based on the first trajectory data associated with the first processing chamber. The method further includes receiving second trajectory data associated with a second processing chamber, wherein the second processing chamber does not meet one or more performance metrics. The method further includes generating, based on the target trajectory data and the second trajectory data, a first recommended corrective action associated with the second processing chamber, wherein the first recommended corrective action includes updating one or more equipment constants of the second processing chamber. The method further includes performing the first recommended corrective action.

[0041] In another aspect of the present disclosure, a system includes a memory and a processing device coupled to the memory. The processing device is configured to receive first trajectory data associated with a first processing chamber, wherein the first processing chamber meets one or more performance metrics. The processing device is further configured to generate target trajectory data based on the first trajectory data associated with the first processing chamber. The processing device is further configured to receive second trajectory data associated with a second processing chamber, wherein the second processing chamber does not meet one or more performance metrics. The processing device is further configured to generate, based on the target trajectory data and the second trajectory data, a first recommended corrective action associated with the second processing chamber, wherein the first recommended corrective action includes updating one or more equipment constants of the second processing chamber. The processing device is further configured to perform the first recommended corrective action.

[0042] In another aspect of the present disclosure, a non - transitory machine - readable storage medium stores instructions. When the instructions are executed, they cause a processing device to perform operations. The operations include receiving first trajectory data associated with a first processing chamber, where the first processing chamber meets one or more performance metrics. The operations further include generating target trajectory data based on the first trajectory data associated with the first processing chamber. The operations further include receiving second trajectory data associated with a second processing chamber, where the second processing chamber does not meet one or more performance metrics. The operations further include generating a first recommended corrective action associated with the second processing chamber based on the target trajectory data and the second trajectory data, where the first recommended corrective action includes updating one or more equipment constants of the second processing chamber. The operations further include executing the first recommended corrective action.

[0043] Figure 1 FIG. is a block diagram illustrating an exemplary system 100 (exemplary system architecture) according to some embodiments. System 100 includes a client device 120, manufacturing equipment 124, sensors 126, metrology equipment 128, a prediction server 112, and a data store 140. The prediction server 112 can be part of a prediction system 110. The prediction system 110 can further include server machines 170 and 180.

[0044] The sensor 126 can provide sensor data 142 associated with the manufacturing equipment 124 (e.g., associated with the production of a corresponding product (e.g., a substrate) by the manufacturing equipment 124). The sensor data 142 can be used to confirm equipment health and / or product health (e.g., product quality). The sensor data 142 can include trace data (e.g., data generated multiple times by the sensor over the duration of a process). The trace data can include values associated with the time at which a related measurement is performed. The manufacturing equipment 124 can execute runs according to a recipe or within a time period to produce a product. In some embodiments, the sensor data 142 can include values of one or more of optical sensor data, spectral data, temperature (e.g., heater temperature), pitch (SP), pressure, high-frequency radio frequency (HFRF), radio frequency (RF) matching voltage, RF matching current, RF matching capacitor position, voltage of an electrostatic chuck (ESC), actuator position, robot position, current, flow rate, power, voltage, etc. The sensor data 142 can include historical sensor data 144 and current sensor data 146. The current sensor data 146 can be associated with the product currently being processed (e.g., a substrate, a semiconductor wafer, etc.), the most recently processed product, multiple most recently processed products, etc. The current sensor data 146 can be used as an input to a trained machine learning model (e.g., to generate prediction data 168). The historical sensor data 144 can include the stored data associated with previously produced products. The historical sensor data 144 can be used to train a machine learning model (e.g., model 190). The historical sensor data 144 and / or the current sensor data 146 can include attribute data (e.g., a tag of the manufacturing equipment ID or design, sensor ID, type, and / or location, a tag of the manufacturing equipment status (e.g., current failure, service life, etc.)).

[0045] The sensor data 142 can be associated with or indicative of manufacturing parameters, e.g., hardware parameters of the manufacturing equipment 124 (e.g., hardware settings or installed components (e.g., size, type, etc.)) or processing parameters of the manufacturing equipment 124 (e.g., heater settings, gas flow rate, etc.).

[0046] Data associated with some hardware parameters and / or processing parameters can be stored, alternatively or additionally, as manufacturing parameter 150, which can include historical manufacturing parameters (e.g., associated with historical processing runs) and current manufacturing parameters. The manufacturing parameter 150 can indicate the input settings of the manufacturing apparatus (e.g., heater power, gas flow rate, etc.). The manufacturing parameter 150 can be or include a component of a processing recipe (e.g., to be executed by the manufacturing equipment 124). When the manufacturing equipment 124 executes a manufacturing process, sensor data 142 and / or manufacturing parameter 150 (e.g., equipment readings during processing of a product) can be provided. For each product (e.g., each substrate), the sensor data 142 can be different. For each product design, each recipe, etc., the manufacturing parameter 150 can be different. The manufacturing parameter 150 can be customized based on the manufacturing equipment 124 (e.g., customized for the performance of a specific processing chamber). Substrates produced by the manufacturing equipment 124 can have property values (film thickness, film strain, etc.) measured by metrology equipment 128 (e.g., measured at an independent metrology facility). The metrology data 160 can be a component of the data storage 140. The metrology data 160 can include historical metrology data 164 (e.g., metrology data associated with previously processed products).

[0047] The manufacturing parameter 150 can include hardware parameters (e.g., information indicating which components are installed in the manufacturing equipment 124, indicating component replacement, indicating component aging, indicating software version or update, etc.) and / or processing parameters (e.g., temperature, pressure, flow rate, rate, current, voltage, gas flow rate, lift speed, etc.).

[0048] In some embodiments, the metrology data 160 can be provided without using an independent metrology facility (e.g., in-situ metrology data (e.g., metrology or metrology proxy collected during processing), integrated metrology data (e.g., metrology or metrology proxy collected when the product is in the chamber or under vacuum but not during a processing operation), on-line metrology data (e.g., data collected after removing the substrate from the vacuum), etc.). The metrology data 160 can include current metrology data (e.g., metrology data associated with the current or most recently processed product), historical metrology data, etc. The historical metrology data can be used to train one or more machine learning models.

[0049] The equipment constant 152 can include settings, parameters, calibrations, etc. associated with the manufacturing equipment 124. The equipment constant can be stored in association with a processing chamber, a processing tool, a processing facility, these groups, etc. The equipment constant can be provided for training a model. The equipment constant can be provided to one or more models as an input. The equipment constant (e.g., equipment constant update) can be received from one or more models as an output.

[0050] In some embodiments, sensor data 142, metrology data 160, manufacturing parameters 150, and / or equipment constants 152 can be processed (e.g., by client device 120 and / or by prediction server 112). Processing of the data can include generating features. In some embodiments, features are patterns in sensor data 142, metrology data 160, and / or manufacturing parameters 150 (e.g., slope, width, height, peak, etc.) or combinations of values from sensor data 142, metrology data 160, equipment constants 152, and / or manufacturing parameters (e.g., power derived from voltage and current, etc.). The data can include features, and via prediction component 114, the features can be used to perform signal processing and / or to obtain prediction data 168 for performing corrective actions.

[0051] Each instance (e.g., set) of sensor data 142 can correspond to a product (e.g., a substrate), a set of manufacturing equipment, the type of substrate produced by the manufacturing equipment, etc. Each instance of metrology data 160 and manufacturing parameters 150 can similarly correspond to a product, a set of manufacturing equipment, the type of substrate produced by the manufacturing equipment, etc. The data store can further store information that correlates different groups of data types (e.g., information indicating that a set of sensor data, a set of metrology data, a set of equipment constants, and a set of manufacturing parameters are all associated with the same product, manufacturing equipment, type of substrate, etc.).

[0052] Golden trace data 162 can be or include sensor data that has been designated as golden data. Golden trace data 162 can be generated or selected by a model (e.g., selected from sensor data 142). Golden trace data 162 can be generated or selected by a machine learning model. Golden trace data 162 can include data associated with one or more measured properties, one or more sensors, etc. Golden trace data 162 can include golden trace upper and lower limits (e.g., guard bands).

[0053] The prediction data 168 can include recommended corrective actions. The prediction data 168 can include updates to the equipment constants for one or more processing chambers. The prediction data 168 can include scheduled updates. The prediction data 168 can include scheduled maintenance (e.g., scheduled recommended preventive or corrective maintenance). The prediction data 168 can include scheduled automated maintenance (e.g., component calibration, processing chamber cleaning or drying operations, etc.). In some embodiments, the prediction system 110 can use supervised machine learning to generate the prediction data 168 (e.g., the prediction data 168 includes the output from a machine learning model trained using labeled data (e.g., sensor data labeled with metrology data)). In some embodiments, the prediction system 110 can use unsupervised machine learning to generate the prediction data 168 (e.g., the prediction data 168 includes the output from a machine learning model trained using unlabeled data, and the output can include clustering results, principal component analysis, anomaly detection, etc.). In some embodiments, the prediction system 110 can use semi-supervised learning to generate the prediction data 168 (e.g., the training data can include a mixture of labeled data and unlabeled data, etc.).

[0054] The data store 140 can further store synthetic data. The synthetic data can be data associated with one or more types of data stored in the data store 140 (e.g., sensor data, manufacturing parameters, equipment constants, metrology data, etc.). The synthetic data can be data that is not generated by the manufacturing equipment or sensors, not associated with the processing of one or more substrates, etc. The synthetic data can be used to replace and / or augment the data collected by / from the manufacturing system 100. The synthetic data can be generated by a user (e.g., a subject matter expert). The synthetic data can be generated by a model (e.g., a statistical model, a machine learning model, a recurrent neural network, etc.). The synthetic data can be provided as an input to a model, provided as a training input to a model, provided as a target output of a model, etc. The synthetic data can be used to augment data types in cases where there is insufficient available data (e.g., for training a machine learning model), fill gaps in the trajectory data (e.g., bridge unsatisfactory portions of the trajectory data between other satisfactory trajectory data), etc.

[0055] The client device 120, the manufacturing equipment 124, the sensors 126, the metrology equipment 128, the prediction server 112, the data store 140, the server machine 170, and the server machine 180 can be coupled to each other via the network 130 to generate the prediction data 168 to perform corrective actions. In some embodiments, the network 130 can provide access to cloud-based services. The operations performed by the client device 120, the prediction system 110, the data store 140, etc. can be performed by a virtual cloud-based device.

[0056] In some embodiments, network 130 is a public network to provide the client device 120 with access to the prediction server 112, data store 140, and other publicly available computing devices. In some embodiments, network 130 is a private network to provide the client device 120 with access to the manufacturing equipment 124, sensors 126, metrology equipment 128, data store 140, and other privately available computing devices. Network 130 may include one or more wide area networks (WANs), local area networks (LANs), wired networks (e.g., Ethernet), wireless networks (e.g., 802.11 networks or Wi-Fi networks), cellular networks (e.g., Long Term Evolution (LTE) networks), routers, hubs, switches, server computers, cloud computing networks, and / or combinations of the foregoing.

[0057] The client device 120 may include a computing device (e.g., a personal computer (PC), laptop computer, mobile phone, smartphone, tablet computer, netbook computer, Internet-connected television (“smart TV”), Internet-connected media player (e.g., Blu-ray player), set-top box, over-the-top (OTT) streaming device, operator box, etc.). The client device 120 may include a corrective action component 122. The corrective action component 122 may receive user input indicative of the manufacturing equipment 124 (e.g., via a graphical user interface (GUI) that is displayed via the client device 120). In some embodiments, the corrective action component 122 transmits the indication to the prediction system 110, receives an output (e.g., prediction data 168) from the prediction system 110, determines a corrective action based on the output, and causes the corrective action to be implemented.

[0058] In some embodiments, the corrective action component 122 obtains sensor data 142 (e.g., current sensor data 146) associated with the manufacturing equipment 124 (e.g., from the data store 140, etc.), and provides the sensor data 142 (e.g., current sensor data 146) associated with the manufacturing equipment 124 to the prediction system 110. In some embodiments, the corrective action component 122 may obtain equipment constants 152 (e.g., an update of the equipment constants recommended by the prediction component 114) from the data store 140, and provide the equipment constants 152 to the manufacturing equipment 124 to update the equipment constants of the manufacturing equipment 124.

[0059] In some embodiments, the corrective action component 122 receives an indication of a corrective action from the prediction system 110, and causes the corrective action to be implemented. Each client device 120 may include an operating system that allows a user to generate, view, or edit one or more of the data (e.g., indications associated with the manufacturing equipment 124, corrective actions associated with the manufacturing equipment 124, etc.).

[0060] In some embodiments, the metrology data 160 (e.g., historical metrology data) corresponds to historical property data of a product (e.g., a product processed using historical manufacturing parameters associated with the historical sensor data 144 and the manufacturing parameters 150), and the prediction data 168 is associated with the predicted property data (e.g., the predicted property data of a product to be produced or that has been produced under the conditions recorded by the current sensor data 146 and / or the current manufacturing parameters). In some embodiments, the prediction data 168 is or includes the predicted metrology data (e.g., virtual metrology data, virtual synthetic microscope images) of a product to be produced or that has been produced according to the conditions recorded as the current sensor data 146, current measurement data, current metrology data, and / or current manufacturing parameters. In some embodiments, the prediction data 168 is or includes any anomalies (e.g., anomalous products, anomalous components, anomalous manufacturing equipment 124, anomalous energy usage, anomalous equipment constants, etc.) and optionally an indication of one or more causes of the anomalies. In some embodiments, the prediction data 168 is an indication of a change or drift over time in some components of the manufacturing equipment 124, sensors 126, metrology equipment 128, etc. In some embodiments, the prediction data 168 is an indication of the end of the life of components of the manufacturing equipment 124, sensors 126, metrology equipment 128, etc. In some embodiments, the prediction data 168 is an indication of the progress of a processing operation being performed (e.g., for process control).

[0061] Performing a manufacturing process that results in defective products can be expensive due to costs such as time, energy, product, components, manufacturing equipment 124, identifying the defects, and discarding the defective products. By inputting the sensor data 142 (e.g., the manufacturing parameters used or to be used to manufacture a product) into the prediction system 110, receiving the output of the prediction data 168, and performing corrective actions based on the prediction data 168, the technical advantage of the system 100 is that the costs of producing, identifying, and discarding defective products can be avoided. By updating the equipment constants of the manufacturing equipment, the equipment performance can be improved, standardized, and / or made more consistent, products that meet the target performance metrics can be produced more frequently, and the costs associated with manufacturing defective products can be avoided.

[0062] Performing a manufacturing process that results in a component failure of manufacturing equipment 124 can be expensive due to downtime, product damage, equipment damage, expedited ordering of replacement parts, etc. By inputting sensor data 142 (e.g., manufacturing parameters that are being used or will be used to manufacture a product), metrology data, measurement data, etc., receiving the output of predictive data 168, and performing a corrective action (e.g., predicted operational maintenance such as replacement, processing, cleaning, etc. of components) based on the predictive data 168, the technical advantage of system 100 is that it can avoid the costs of one or more of unexpected component failures, unscheduled downtime, productivity losses, unexpected equipment failures, product scrap, etc. Monitoring the performance of components (e.g., manufacturing equipment 124, sensors 126, metrology equipment 128, etc.) over time can provide an indication of deteriorating components. Monitoring the equipment constants 152 over time can provide an indication of deteriorating components (e.g., if the recommended equipment constants fall outside of control limits, statistical limits, guard bands, etc.).

[0063] The manufacturing parameters may be sub-optimal for producing a product and may have expensive consequences such as increased consumption of resources (e.g., energy, coolant, gas, etc.), increased amount of time to produce the product, increased component failures, increased amount of defective products, etc. By inputting an indication of the metrology into the prediction system 110, receiving the output of the predictive data 168, and performing a corrective action (e.g., based on the predictive data 168) to update the equipment constants (e.g., set optimal equipment constants) of the manufacturing equipment 124, system 100 can have the technical advantage of using improved equipment constants (e.g., processing equipment constants, non-processing equipment constants, calibration tables, etc.) to avoid the expensive consequences of sub-optimal equipment performance.

[0064] For reducing the environmental impact of a manufacturing process, the manufacturing parameters may be sub-optimal. For example, a semiconductor manufacturing process using a first set of manufacturing parameters may generate additional pollutants, waste, carbon dioxide, and / or other greenhouse gases, etc., compared to performing a process using a second set of manufacturing parameters. The manufacturing process may be less sustainable than another process with similar results (e.g., due to the combination of manufacturing parameters used). By inputting an indication of the manufacturing equipment performance into the prediction system 110, receiving the output associated with the corrective action from the prediction system 110, and formulating a corrective action, system 100 can have the technical advantage of using improved equipment constants to reduce the environmental impact of the manufacturing process.

[0065] For the production rate of one or more products, the manufacturing parameters may be sub-optimal. Utilizing a specific set or series of sets of manufacturing parameters can result in faster production, faster processing, faster processing within acceptable defect limits, etc. By inputting an indication of the manufacturing equipment performance into the prediction system 110, receiving an output associated with a corrective action from the prediction system 110, and formulating a corrective action, the system 100 can have the technical advantage of reducing the time to process a product. The system 100 can have the technical advantage of using improved equipment constants to reduce the processing time per substrate (e.g., within target defect limits and / or other performance metrics).

[0066] The corrective action can be associated with one or more of computational process control (CPC), statistical process control (SPC) (e.g., SPC on electronic components for determining the process in control, SPC for predicting the useful life of components, SPC compared with three standard deviation charts, etc.), advanced process control (APC), model-based process control, predictive operation maintenance, design optimization, updating of manufacturing parameters, updating of processing recipes, updating of equipment constants, feedback control, feedforward control, machine learning modification, etc.

[0067] In some embodiments, the corrective action includes providing a warning (e.g., a warning indicating a recommended action (e.g., scheduling maintenance or calibration); providing an alarm to stop or not perform the manufacturing process if the prediction data 168 indicates a prediction anomaly (e.g., an anomaly of a product, component, or manufacturing equipment 124), etc.). In some embodiments, the execution of the corrective action includes causing an update to one or more equipment constants. In some embodiments, the execution of the corrective action can include retraining a machine learning model associated with the manufacturing equipment 124. In some embodiments, the execution of the corrective action can include training a new machine learning model associated with the manufacturing equipment 124.

[0068] In some embodiments, the corrective action includes causing preventive operation maintenance (e.g., replacing, processing, cleaning, etc. components of the manufacturing equipment 124). In some embodiments, the corrective action includes causing design optimization (e.g., updating equipment constants, manufacturing processes, manufacturing equipment 124, etc. for an optimized product). In some embodiments, the corrective action includes updating a recipe (e.g., changing the timing for a manufacturing subsystem to enter an idle or active mode, changing setpoints of various property values, etc.). In some embodiments, the corrective action includes scheduling or performing calibration operations, cleaning operations, and / or drying operations of the processing system.

[0069] The prediction server 112, the server machine 170, and the server machine 180 may each include one or more computing devices (e.g., rack servers, router computers, server computers, personal computers, host computers, laptop computers, tablet computers, desktop computers, graphics processing units (GPUs), application-specific integrated circuits (ASICs) for accelerators (e.g., tensor processing units (TPUs)), etc.). The operations of the prediction server 112, the server machine 170, the server machine 180, the data storage 140, etc. can be performed through cloud computing services, cloud data storage services, etc.

[0070] The prediction server 112 may include a prediction component 114. In some embodiments, the prediction component 114 may receive current sensor data 146 for performing a correction action associated with the manufacturing device 124. In some embodiments, the correction action may include updating one or more equipment constants. The prediction component 114 may further receive additional data (e.g., current manufacturing parameters (e.g., received from the client device 120, retrieved from the data storage 140), metrology data 160, equipment constants 152, golden trace data 162, etc.) to generate an output (e.g., prediction data 168) for performing a correction action associated with the manufacturing equipment 124. In some embodiments, the prediction component 114 may use one or more trained machine learning models 190 to determine an output for performing a correction action based on the current data. In some embodiments, the prediction data 168 may be utilized as an input to a machine learning model. A machine learning model may receive data output by another machine learning model as an input.

[0071] The manufacturing equipment 124 may be associated with one or more machine learning models (e.g., model 190). The machine learning models associated with the manufacturing equipment 124 may perform many tasks (including process control, classification, performance prediction, process update, etc.). The model 190 may be trained using data associated with the manufacturing equipment 124 or the products processed by the manufacturing equipment 124 (e.g., sensor data 142 (e.g., collected by the sensor 126), manufacturing parameters 150 (e.g., associated with the process control of the manufacturing equipment 124), metrology data 160 (e.g., generated by the metrology equipment 128), equipment constants 152, etc.).

[0072] One type of machine learning model that can be used to perform some or all of the above tasks is an artificial neural network (e.g., a deep neural network). An artificial neural network generally includes a feature representation component having a classifier or regression layer that maps features to a desired output space. For example, a convolutional neural network (CNN) has multiple layers of convolutional filters. Pooling is performed at lower layers and can solve non-linear problems, and a multi-layer perceptron is typically appended on top to map the top-level features extracted by the convolutional layer to a decision (e.g., a classification output).

[0073] A recurrent neural network (RNN) is another type of machine learning model. Recurrent neural network models are designed to interpret a sequence of inputs where the inputs are inherently related to each other (e.g., time-series data, sequential data, etc.). The output of the perceptron of the RNN is fed back as an input into the perceptron to generate the next output.

[0074] Deep learning is a category of machine learning algorithms that use cascades of multiple non-linear processing units for feature extraction and transformation. Each successive layer uses the output of the previous layer as an input. Deep neural networks can learn in a supervised (e.g., classification) and / or unsupervised (e.g., pattern analysis) manner. Deep neural networks include a hierarchy of layers where different layers learn different levels of representations corresponding to different levels of abstraction. In deep learning, each level learns to transform its input data into a slightly more abstract and composite representation. For example, in an image recognition application, the original input can be a pixel matrix; the first representation layer can abstract the pixels and encode the edges; the second layer can combine and encode the arrangement of the edges; the third layer can encode higher-level shapes (e.g., teeth, lips, gums, etc.); and the fourth layer can recognize the scanned object. It should be noted that deep learning techniques can learn on their own which features to optimally place at which level. The "depth" in "deep learning" refers to the number of layers of data transformation. More precisely, a deep learning system has a relatively large credit assignment path (CAP) depth. The CAP is the chain of transformations from input to output. The CAP describes the potential causal relationship between the input and the output. For a feed-forward neural network, the depth of the CAP can be the depth of the network and can be the number of hidden layers plus one. For a recurrent neural network where a signal may propagate through a layer more than once, the CAP depth may be infinite.

[0075] In some embodiments, system 100 may utilize multiple machine learning models. A first machine learning model (e.g., by the prediction system 110) may be utilized to generate golden trajectory data associated with a manufacturing process, manufacturing system, product design, recipe, etc. The first machine learning model may be configured to receive metrology data and tool trajectories (e.g., trajectory sensor data) as inputs. The first machine learning model may be configured to output one or more golden trajectories. The output golden trajectories may be data measured by a golden chamber. The golden trajectories may be associated with products that meet target performance metrics. The golden trajectories may include ideal or optimal trajectories. The golden trajectories may include upper and lower boundaries, upper and lower guard bands, control limits, average trajectories, median trajectories, etc. The golden trajectories may be selected, generated, etc. for a target outcome. For example, a manufacturing process may have multiple goals (e.g., a target energy consumption, a target environmental impact, a target throughput rate, and a target performance level (e.g., a target level of defective products)). Golden trajectories may be selected to optimize one metric, balance one or more metrics, optimize one or more metrics while keeping other metrics within a target range, etc.

[0076] The process may include golden trajectory guard bands. The guard bands may represent limits on how far a trajectory may deviate before certain actions are taken. For example, if a data point is within the range defined by the guard band, if a target portion of the data point is within the guard band, if a target portion of the data point is within the target value of the guard band, etc., then the trajectory data of the process may be considered acceptable. In some embodiments, the guard bands may be generated statistically (e.g., by generating synthetic trajectories that include the target portion of the input data). In some embodiments, the guard bands may be generated by a statistical model, machine learning model, etc. In some embodiments, the guard bands may be generated based on multiple process runs, process chambers, etc. In some embodiments, the minimum and maximum data values from multiple runs (e.g., multiple golden trajectories) may define the minimum and maximum data values of the guard band. The golden trajectories used to define the guard band may be generated by a golden chamber, golden equipment, golden tool, etc.

[0077] In some embodiments, the golden trace is provided to a second machine learning model. The second machine learning model can be configured to recommend a corrective action (e.g., to the client device 120, to the user, etc.). The second machine learning model can be configured to formulate a corrective action. The second machine learning model can be configured to recommend and / or formulate a schedule for the corrective action. The second machine learning model can be configured to recommend and / or formulate an equipment constant update. The second machine learning model can receive metrology data (e.g., metrology data of a processing chamber that may include the equipment constant to be updated) as an additional input. The second machine learning model can receive equipment constants (e.g., equipment constants of the golden chamber, equipment constants of the chamber associated with the golden trace data, equipment constants of the chamber that may include the equipment constant to be updated, etc.) as an additional input. The second machine learning model can receive trace data (e.g., trace data of a chamber that includes the equipment constant to be updated) as an additional input. The second machine learning model can be configured to improve the operation of one or more processing chambers. The second machine learning model can improve the operation of one or more processing chambers by recommending and / or formulating an update to the equipment constant that improves substrate metrology (e.g., increases the likelihood that substrates processed in the processing chamber meet the target performance metric). The second machine learning model can improve the operation of one or more processing chambers by recommending and / or formulating an update to the equipment constant that improves the trace data of the processing chamber (e.g., increases the similarity between the golden trace data and the trace data of the processing chamber).

[0078] In some embodiments, the prediction component 114 receives one or more types of data, performs signal processing to decompose the data into multiple sets of current data, provides the multiple sets of data to the trained model 190 as inputs, and obtains an output indicating the prediction data 168 from the trained model 190. The input data can include sensor data 142, manufacturing parameters 150, equipment constants 152, metrology data 160, golden trace data 162, prediction data 168, etc. In some embodiments, the prediction data indicates metrology data (e.g., prediction of substrate quality). In some embodiments, the prediction data indicates the health of components and / or processing chambers. In some embodiments, the prediction data indicates the processing progress (e.g., for ending a processing operation). In some embodiments, the prediction data 168 includes golden trace data. In some embodiments, the prediction data 168 includes updated equipment constants. In some embodiments, the prediction data 168 includes a schedule for corrective actions (e.g., a schedule for updating equipment constants).

[0079] In some embodiments, the various models discussed in connection with the combined model 190 (e.g., supervised machine learning models, unsupervised machine learning models, etc.) can be combined in one model (e.g., an overall model) or can be separate models.

[0080] Data can be passed back and forth between several different models including model 190, calibration action component 122, and prediction component 114. In some embodiments, some or all of these operations can alternatively be performed by different devices (e.g., client device 120, server machine 170, server machine 180, etc.). Those of ordinary skill in the art will understand that variations in the data flow, which components perform which processes, which data is provided to which models, etc. are all within the scope of the present disclosure.

[0081] Data store 140 can be a memory (e.g., random access memory), a drive (e.g., hard disk drive, flash drive), a database system, a cloud-accessible memory system, or another type of component or device capable of storing data. Data store 140 can include multiple storage components (e.g., multiple drives or multiple databases) that can span multiple computing devices (e.g., multiple server computers). Data store 140 can store sensor data 142, manufacturing parameters 150, metrology data 160, golden trace data 162, and prediction data 168.

[0082] Sensor data 142 can include historical sensor data 144 and current sensor data 146. Sensor data can include the sensor data time trace during the duration of the manufacturing process, the association of data with physical sensors, preprocessed data (e.g., averaged and composite data), and data indicating the changing performance of sensors over time (i.e., for many manufacturing processes). Manufacturing parameters 150 and metrology data 160 can contain similar characteristics (e.g., historical metrology data and current metrology data). Historical sensor data 144, historical metrology data, and historical manufacturing parameters can be historical data (e.g., at least a portion of this data can be used to train model 190). Current sensor data 146, current metrology data, and current manufacturing parameters can be current data (at least a portion of which is input into learning model 190 after the historical data) for generating prediction data 168 (e.g., for performing calibration actions). Equipment constants 152 can include current equipment constants, historical equipment constants (e.g., for training the model), expected equipment constants (e.g., scheduled updates to the equipment constants), etc. Sensor data, manufacturing parameters, metrology data, etc. can include real (e.g., measured from the manufacturing process, measured from the produced substrate, etc.) and synthetic (e.g., generated by machine learning models, generated by subject matter experts, etc.) data.

[0083] In some embodiments, the prediction system 110 further includes server machines 170 and 180. Server machine 170 includes a dataset generator 172 capable of generating a dataset (e.g., a set of data inputs and a set of target outputs) for training, validating, and / or testing a model 190 including one or more machine learning models. Some operations of the dataset generator 172 are described below with respect to Figures 2A to 2B and Figure 4A in detail. In some embodiments, the dataset generator 172 may partition historical data (e.g., historical sensor data 144, historical manufacturing parameters, historical metrology data 164) into a training set (e.g., sixty percent of the historical data), a validation set (e.g., twenty percent of the historical data), and a test set (e.g., twenty percent of the historical data).

[0084] In some embodiments, the prediction system 110 (e.g., via the prediction component 114) generates multiple sets of features. For example, a first set of features may correspond to a first set of types of sensor data (e.g., from a first set of sensors, a first combination of values from a first set of sensors, a first pattern of values from a first set of sensors), the first set of types of sensor data corresponding to each of the datasets (e.g., the training set, the validation set, and the test set), while a second set of features may correspond to a second set of types of sensor data (e.g., from a second set of sensors different from the first set of sensors, a second combination of values different from the first combination, a second pattern different from the first pattern), the second set of types of sensor data corresponding to each of the datasets.

[0085] In some embodiments, historical data is provided to the machine learning model 190 as training data. In some embodiments, the output from another machine learning model (e.g., prediction data 168 as training data) is provided to the machine learning model 190. The type of data provided will vary depending on the intended use of the machine learning model. For example, a machine learning model may be trained by providing historical sensor data 144 as training input and corresponding metrology data 160 as the target output. In some embodiments, a large amount of data is used to train the model 190 (e.g., sensor and metrology data for hundreds of substrates may be used).

[0086] Server machine 180 includes a training engine 182, a validation engine 184, a selection engine 185, and / or a testing engine 186. The engines (e.g., training engine 182, validation engine 184, selection engine 185, and testing engine 186) may refer to hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, processing devices, etc.), software (e.g., instructions running on a processing device, a general-purpose computer system, or a dedicated machine), firmware, microcode, or a combination of the foregoing. The training engine 182 is capable of training a model 190 using one or more sets of features associated with a training set from the dataset generator 172. The training engine 182 may generate multiple trained models 190, where each trained model 190 corresponds to a different set of features of the training set (e.g., sensor data from different sets of sensors). For example, a first trained model may be trained using all features (e.g., X1 to X5), a second trained model may be trained using a first subset of features (e.g., X1, X2, X4), and a third trained model may be trained using a second subset of features that partially overlaps with the first subset of features (e.g., X1, X3, X4, and X5). The dataset generator 172 may receive the output of the trained model (e.g., prediction data 168 or equipment constants to be updated), collect the data into training, validation, and test datasets, and use the datasets to train a second model (e.g., a machine learning model configured to output prediction data, corrective actions, etc.).

[0087] The validation engine 184 is capable of validating the trained model 190 using a corresponding set of features from the validation set of the dataset generator 172. For example, a first trained machine learning model 190 trained using a first set of features of the training set may be validated using a first set of features of the validation set. The validation engine 184 may determine the accuracy of each of the trained models 190 based on the corresponding set of features of the validation set. The validation engine 184 may discard the trained models 190 that have an accuracy that does not meet a threshold accuracy. In some embodiments, the selection engine 185 is capable of selecting one or more trained models 190 that have an accuracy that meets the threshold accuracy. In some embodiments, the selection engine 185 is capable of selecting the trained model 190 that has the highest accuracy among the trained models 190.

[0088] The testing engine 186 is capable of testing the trained model 190 using a corresponding set of features of the test set from the dataset generator 172. For example, a first trained machine learning model 190 trained using a first set of features of the training set may be tested using a first set of features of the test set. The testing engine 186 may determine the trained model 190 that has the highest accuracy among all of the trained models based on the test set.

[0089] In the case of a machine learning model, model 190 may refer to the model artifact established by training engine 182 using a training set that includes data inputs and corresponding target outputs (the correct answers for the respective training inputs). In an embodiment, the training set includes synthetic microscopic images generated by synthetic data generator 174. Patterns in the data set can be found to map the data inputs to the target outputs (correct answers) and to provide a mapping for machine learning model 190 to capture these patterns. Machine learning model 190 may use one or more of support vector machine (SVM), radial basis function (RBF), clustering, supervised machine learning, semi-supervised machine learning, unsupervised machine learning, k-nearest neighbor algorithm (k-NN), linear regression, random forest, neural networks (e.g., artificial neural networks, recurrent neural networks), and the like. Synthetic data generator 174 may include one or more machine learning models (which may include one or more of the same type of models (e.g., artificial neural networks)).

[0090] In some embodiments, historical data (e.g., historical sensor data 144) may be used to train one or more machine learning models 190. In some embodiments, synthetic data 162 or a combination of historical data and synthetic data may be used to train model 190.

[0091] Prediction component 114 may provide current data to model 190 and may run model 190 on the inputs to obtain one or more outputs. For example, prediction component 114 may provide current sensor data 146 to model 190 and may run model 190 on the inputs to obtain one or more outputs. Prediction component 114 is capable of determining (e.g., extracting) prediction data 168 from the output of model 190. Prediction component 114 may determine (e.g., extract) confidence data from the output, the confidence data indicating the level of confidence that prediction data 168 is an accurate predictor of a process associated with input data of a product that is being produced or will be produced using manufacturing equipment 124 with current sensor data 146 and / or current manufacturing parameters. Prediction component 114 or correction action component 122 may use the confidence data to decide whether to cause a correction action associated with manufacturing equipment 124 based on prediction data 168.

[0092] Confidence data can include or indicate a level of confidence that the prediction data 168 is an accurate prediction for a product or component associated with at least a portion of the input data. In one example, the level of confidence is a real number between 0 and 1, inclusive, where 0 indicates no confidence that the prediction data 168 is an accurate prediction for a product processed based on the component health of the input data or a component of the manufacturing equipment 124, and 1 indicates absolute confidence that the prediction data 168 accurately predicts the nature of a product processed based on the component health of the input data or a component of the manufacturing equipment 124. In response to confidence data indicating that the level of confidence is below a threshold level for a predetermined number of instances (e.g., a percentage of instances, a frequency of instances, a total number of instances, etc.), the prediction component 114 can cause the trained model 190 to be retrained (e.g., based on current sensor data 146, current manufacturing parameters, etc.). In some embodiments, the retraining can include using historical data and / or synthetic data to generate one or more data sets (e.g., via the data set generator 172).

[0093] By way of illustration and not limitation, aspects of the present disclosure describe training one or more machine learning models 190 using historical data (e.g., historical sensor data 144, historical manufacturing parameters) and inputting current data (e.g., current sensor data 146, current manufacturing parameters, and current metrology data) into one or more trained machine learning models to determine prediction data 168. In other embodiments, heuristic models, physics-based models, or rule-based models are used to determine the prediction data 168 (e.g., without using a trained machine learning model). In some embodiments, historical and / or synthetic data can be used to train such models. In some embodiments, these models can be retrained using a combination of real historical data and synthetic data. The prediction component 114 can monitor the historical sensor data 144, historical manufacturing parameters, and metrology data 160. Any information described regarding the data inputs 210A to 210B can be monitored or otherwise used in a heuristic, physics-based, or rule-based model. Figures 2A to 2B data inputs 210A to 210B.

[0094] In some embodiments, the functions of client device 120, prediction server 112, server machines 170, and server machine 180 may be provided by a smaller number of machines. For example, in some embodiments, server machines 170 and 180 may be integrated into a single machine, and in some other embodiments, server machine 170, server machine 180, and prediction server 112 may be integrated into a single machine. In some embodiments, client device 120 and prediction server 112 may be integrated into a single machine. In some embodiments, the functions of client device 120, prediction server 112, server machines 170, server machine 180, and data store 140 may be performed by cloud-based services.

[0095] Generally, functions described in one embodiment as being performed by client device 120, prediction server 112, server machines 170, and server machine 180 may also be performed on prediction server 112 in other embodiments where appropriate. Additionally, functions attributed to specific components may be performed by different or multiple components operating together. For example, in some embodiments, prediction server 112 may determine a corrective action based on prediction data 168. In another example, client device 120 may determine prediction data 168 based on the output from a trained machine learning model.

[0096] Furthermore, the functions of specific components may be performed by different or multiple components operating together. One or more of prediction server 112, server machine 170, or server machine 180 may be accessed via a suitable application programming interface (API) as a service provided to other systems or devices.

[0097] In an embodiment, a "user" may represent a single individual. However, other embodiments of the present disclosure cover a "user" as an entity controlled by multiple users and / or automated sources. For example, a group of individual users united as an administrator group may be considered a "user".

[0098] Embodiments of the present disclosure may be applied to data quality assessment, feature enhancement, model evaluation, virtual metrology (VM), predictive maintenance (PdM), limit optimization, process control, etc.

[0099] Figures 2A to 2B A block diagram depicting example dataset generators 272A to 272B (e.g., Figure 1 dataset generator 172) according to some embodiments, to establish a dataset for training, testing, validating, etc. a model (e.g., Figure 1 model 190). Each dataset generator 272 may be Figure 1Part of the server machine 170. In some embodiments, several machine learning models associated with the manufacturing equipment 124 (e.g., within a manufacturing facility) can be trained, used, and maintained. Each machine learning model can be associated with a dataset generator 272, and multiple machine learning models can share the dataset generator 272, etc.

[0100] Figure 2A Depicts a system 200A including a dataset generator 272A for establishing a dataset for one or more supervised models (e.g., Figure 1 Model 190). A supervised model can be generated by providing training inputs and associated target outputs (e.g., correct answers) to the model. The dataset generator 272A can use historical data (e.g., historical sensor data, historical metrology data, etc.) to establish a dataset (e.g., data inputs 210A, target outputs 220A). The dataset generator 272A can be used to generate one or more datasets for a machine learning model configured to recommend corrective actions. The dataset generator 272A can be used to generate one or more datasets for a machine learning model configured to formulate corrective actions. The dataset generator 272A can be used to generate one or more datasets for a machine learning model configured to schedule updates to equipment constants for manufacturing equipment.

[0101] The dataset generator 272A can generate a dataset to train, test, and validate the model. In some embodiments, the dataset generator 272A can generate a dataset for a machine learning model. In some embodiments, the dataset generator 272A can generate a dataset for training, testing, and / or validating a machine learning model configured to schedule updates to equipment constants for manufacturing equipment. A set of target trajectory data 242A and a set of historical trajectory data 246A are provided to the machine learning model as data inputs 210A. The target trajectory data can include golden trajectory data. The historical trajectory data can include data from a processing chamber (e.g., a processing chamber exhibiting poor performance). The poor performance can include a threshold number or portion of products that do not meet a threshold quality metric, a number or portion of equipment constants that are outliers compared to other chambers, etc. The machine learning model can be configured to recommend a predicted change in the equipment constant to cause the trajectory data of the processing chamber to be more similar to the target trajectory data as the output.

[0102] In some embodiments, the dataset generator 272A can generate additional datasets to be part of the data input 210A for providing to the model. The model can be configured to recommend changes to the equipment constants based on the additional data. One or more sets of target metrology data can be provided to the machine learning model. The target metrology data can be associated with the golden trace. One or more sets of historical metrology data (e.g., metrology data of products associated with multiple sets of historical trace data) can be provided to the machine learning model. One or more sets of equipment constants (e.g., equipment constants to be updated, equipment constants associated with the processing chamber that produced the golden trace data, etc.) can be provided to the machine learning model.

[0103] The dataset generator 272A can be used to train a machine learning model to recommend and / or schedule correction actions. The machine learning model can be configured to adjust the equipment constants of one or more processing chambers. The machine learning model can be configured to adjust the equipment constants to be more closely similar to the equipment constants of the golden chamber. The machine learning model can be configured to adjust the equipment constants such that the trace data of the processing chamber can be more closely similar to the golden trace data. The machine learning model can be configured to adjust the equipment constants such that the metrology of the products produced by the processing chamber can be more closely similar to the metrology of the products produced by the golden chamber.

[0104] The machine learning model can generate a schedule for updating the equipment constants. The machine learning model can limit the number of updates to the processing chambers, tools, facilities, etc. that are executed at one time. The machine learning model can limit the number of chambers or tools to be updated at one time. The machine learning model can make the equipment constant update schedule based on the risk and effectiveness of updating the equipment constants. For example, the machine learning model can prioritize equipment constant updates with low risk (e.g., less likely to increase the production volume of defective products) and that may effectively address the defects of the processing chamber. The risk and effectiveness can be evaluated based on subject matter expertise, can be evaluated by a physics-based model, can be evaluated by a statistical or machine learning model, etc. The machine learning model can perform optimization operations associated with scheduling equipment constant updates. The machine learning model can optimize the equipment constant updates to mitigate the risk of reducing the effectiveness of the processing chamber. The machine learning model can optimize the equipment constant updates to increase the likelihood of improving the effectiveness of the processing chamber. The machine learning model can optimize the equipment constant updates to achieve other goals (e.g., reducing energy consumption, reducing material consumption, shortening the processing time, etc.).

[0105] The dataset generator 272A can be used to generate any type of machine learning model (e.g., combining Figure 1Data of the machine learning architecture under discussion). The dataset generator 272A can be used to generate data for a machine learning model for recommending equipment constant updates. The dataset generator 272A can be used to generate data for a machine learning model for scheduling equipment constant updates. The dataset generator 272A can be used to generate data for a machine learning model for formulating equipment constant updates. The dataset generator 272A can be used to generate data for a machine learning model configured to identify product anomalies and / or handle device failures. For example, the dataset generator 272A can be used to generate data for a machine learning model configured to detect outliers in equipment constants, correlations or relationships between equipment constants, trajectory data, and / or metrology data, etc. The dataset generator 272A can be used to generate data for a machine learning model configured to detect causal relationships (e.g., detecting the causes and effects of one or more metrics).

[0106] In some embodiments, the dataset generator 272A generates a dataset (e.g., a training set, a validation set, a test set) including one or more data inputs 210A (e.g., training inputs, validation inputs, test inputs). The data inputs 210A can be provided to the training engine 182, the validation engine 184, or the test engine 186. The dataset can be used to train, validate, or test a model (e.g., Figure 1 the model 190).

[0107] In some embodiments, the data input 210A can include one or more sets of data. As an example, the system 200A can generate multiple sets of sensor data, and the multiple sets of sensor data can include sensor data from one or more types of sensors, combinations of sensor data from one or more types of sensors, patterns of sensor data from one or more types of sensors, etc., one or more of which.

[0108] In some embodiments, the data input 210A can include one or more sets of data. As an example, the system 200A can generate multiple sets of historical metrology data, and the multiple sets of historical metrology data can include metrology data of the dimensional groups of the device (e.g., including the thickness of the device, but not including optical data or surface roughness, etc.), metrology data derived from one or more types of sensors, combinations of metrology data derived from one or more types of sensors, patterns of metrology data, etc., one or more of which. The multiple sets of data inputs 210A can include data describing different aspects of manufacturing (e.g., combinations of metrology data and sensor data, combinations of metrology data and manufacturing parameters, some metrology data, some manufacturing parameter data, and some combinations of sensor data, data associated with components of the manufacturing system (e.g., component quality data), etc.). The data input 210A can include measured and / or synthetic data.

[0109] In some embodiments, the dataset generator 272A may generate a first data input corresponding to the first set of target trajectory data 242A and the first set of historical trajectory data 246A to train, validate, or test the first machine learning model. The dataset generator 272A may generate a second data input corresponding to the second set of target trajectory data 242B and the second set of historical trajectory data 246B to train, validate, or test the second machine learning model.

[0110] In some embodiments, the dataset generator 272A generates a dataset (e.g., a training set, a validation set, a test set) that includes one or more data inputs 210A (e.g., training inputs, validation inputs, test inputs) and may include one or more target outputs 220A corresponding to the data inputs 210A. The dataset may also include mapping data that maps the data inputs 210A to the target outputs 220A. In some embodiments, the dataset generator 272A may generate data for training a machine learning model configured to output equipment constant updates by generating a dataset that includes output equipment constant data 268. The data inputs 210A may also be referred to as "features", "attributes", or "information". In some embodiments, the dataset generator 272A may provide the dataset to the training engine 182, the validation engine 184, or the test engine 186, where the dataset is used to train, validate, or test a machine learning model (e.g., one of the machine learning models included in the model 190, the overall machine learning model, etc.).

[0111] A system 200B that includes a dataset generator 272B (e.g., Figure 1 the dataset generator 172) establishes a dataset for one or more machine learning models (e.g., Figure 1 the model 190). The dataset generator 272B may use historical data to establish the dataset (e.g., the data inputs 210B). An example dataset generator 272B is configured to generate a dataset for a machine learning model configured to take as input data associated with a processed product and generate golden trajectory data as output. The dataset generator 272B may provide the dataset to an unsupervised machine learning model (e.g., the dataset generator 272B may provide the data inputs 210B and may not provide target outputs). The dataset generator 272B may share one or more features and / or functions with the dataset generator 272A.

[0112] The dataset generator 272B can generate a dataset to train, test, and validate a machine learning model. A set of golden chamber data 262A (e.g., metrology data of products processed by a golden processing chamber, trace data from the processing of products, etc.) is provided to the machine learning model as data input 210B. The machine learning model can include two or more individual models (e.g., the machine learning model can be an ensemble model). The machine learning model can be configured to generate output data including golden trace data. The golden trace data can include traces to be matched with processing chambers other than the golden processing chamber. The golden trace data can include upper and / or lower boundaries (e.g., an area of acceptable trace data can be defined). The golden trace data can come from various processing runs, various golden chambers, various sensors, etc. For example, trace data from different sensors can affect metrology in different ways. Different chambers can perform differently in various metrologies, and the golden trace data can reflect the improved performance of one golden chamber relative to another golden chamber in a specific area (e.g., a specific subsystem (e.g., pressure subsystem, RF subsystem, etc.)). The dataset generator 272B can generate a dataset to train an unsupervised machine learning model (e.g., a model configured to receive synthetic microscopy data as input and generate cluster data, outlier detection data, anomaly detection data, etc. as output). The model can be trained to generate output data based on the association between sensor data and metrology data (e.g., the model can be trained to identify how combined trace data from sensors is related to metrology data).

[0113] In some embodiments, the dataset generator 272B generates a dataset (e.g., a training set, a validation set, a test set) including one or more data inputs 210B (e.g., training inputs, validation inputs, test inputs). The data input 210B can also be referred to as a "feature", an "attribute", or "information". In some embodiments, the dataset generator 272B can provide the dataset to a training engine 182, a validation engine 184, or a test engine 186, where the dataset is used to train, validate, or test a machine learning model (e.g., Figure 1 the model 190). Some embodiments for generating a training set are further described with respect to Figure 4A this.

[0114] In some embodiments, the dataset generator 272B can generate a first data input corresponding to a first set of golden chamber data 244A to train, validate, or test a first machine learning model, while the dataset generator 272A can generate a second data input corresponding to a second set of golden chamber data 244B for training, validating, or testing a second machine learning model.

[0115] The data input 210B for training, validating, or testing a machine learning model may include information for a specific manufacturing chamber (e.g., for specific substrate manufacturing equipment). In some embodiments, the data input 210B may include information for a specific type of manufacturing equipment (e.g., manufacturing equipment sharing specific characteristics). The data input 210B may include data associated with a certain type of device (e.g., expected function, design, production using a specific recipe, etc.). Training a machine learning model based on the type of equipment, device, recipe, facility, etc. may allow the trained model to generate golden trajectory data applicable to multiple settings (e.g., for multiple different facilities, products, etc.).

[0116] In some embodiments, after generating a data set and using the data set to train, validate, or test a machine learning model, the model may be further trained, validated, or tested or adjusted (e.g., adjusting weights or parameters associated with the input data of the model (e.g., connection weights in a neural network)). Additional data may be utilized to perform further training, validation, testing, or adjustment (e.g., additional training data generated by manufacturing equipment after the model is initially trained).

[0117] Figure 3 FIG. is a block diagram of a system 300 for generating output data (e.g., Figure 1 predicted data 168) according to some embodiments. In some embodiments, the system (e.g., system 300) may be used in conjunction with a machine learning model configured to generate golden trajectory data (e.g., Figure 1 golden trajectory data 162). In some embodiments, the system (e.g., system 300) may be used in conjunction with a machine learning model to determine a corrective action associated with manufacturing equipment. In some embodiments, the system (e.g., system 300) may be used in conjunction with a machine learning model to determine a fault in manufacturing equipment. In some embodiments, the system (e.g., system 300) may be used in conjunction with a machine learning model to cluster or classify equipment constants for a processing tool or chamber. The system (e.g., system 300) may be used in conjunction with a machine learning model to schedule an update of equipment constants for manufacturing equipment. The system (e.g., system 300) may be used in conjunction with a machine learning model having functions different from the functions listed as being associated with a manufacturing system.

[0118] System 300 and the accompanying description relate to a machine learning model that receives data from one or more golden processing chambers and data from one or more other processing chambers as inputs and generates recommended corrective actions as outputs to improve the performance of one or more other processing chambers. Golden processing chamber data can include golden trajectory data and golden equipment constants. Other processing chamber data can include trajectory data and equipment constants. The input data can further include metrology data, additional chamber data, target performance data, etc. The output can further include recommended equipment constant updates, scheduled equipment constant updates, etc. A machine learning model with other functions can operate in conjunction with a system (similar to System 300) with appropriate changes (e.g., identification of input and output data).

[0119] At block 310, System 300 (e.g., Figure 1 components of the prediction system 110) performs data partitioning of the data to be used in training, validating, and / or testing a machine learning model (e.g., via Figure 1 the dataset generator 172 of the server machine 170). In some embodiments, the training data 364 includes golden data (e.g., golden trajectory data, golden metrology data, golden equipment constants, etc.). In some embodiments, the training data 364 includes historical data (e.g., historical metrology data, historical equipment constant data, historical trajectory data, etc.). The training data 364 can undergo data partitioning at block 310 to generate a training set 302, a validation set 304, and a test set 306. For example, the training set can be 60% of the training data, the validation set can be 20% of the training data, and the test set can be 20% of the training data.

[0120] The generation of the training set 302, the validation set 304, and the test set 306 can be customized for a specific application. For example, the training set can be 60% of the training data, the validation set can be 20% of the training data, and the test set can be 20% of the training data. System 300 can generate multiple sets of features for each of the training set, the validation set, and the test set. For example, if the training data 364 includes sensor data (including from 20 sensors (e.g., Figure 1Features of sensor data export of the sensor 126) and 10 manufacturing parameters (e.g., manufacturing parameters corresponding to the same processing run as the sensor data from 20 sensors), the sensor data can be divided into a first set of features including sensors 1 to 10 and a second set of features including sensors 11 to 20. The manufacturing parameters can also be divided into multiple groups (e.g., a first set of manufacturing parameters including parameters 1 to 5 and a second set of manufacturing parameters including parameters 6 to 10). The training input, the target output, both can be divided into multiple groups, or neither is divided into multiple groups. Multiple models can be trained on different groups of data.

[0121] At block 312, the system 300 uses the training set 302 to perform model training (e.g., via Figure 1 the training engine 182). The training of machine learning models and / or physics-based models (e.g., digital twins) can be implemented in a supervised learning manner, which includes providing a training data set including labeled inputs through the model, observing its output, defining an error (by measuring the difference between the output and the labeled value), and using techniques (e.g., gradient descent and backpropagation) to tune the weights of the model to minimize the error. In many applications, repeating this process on many labeled inputs in the training data set results in a model that can produce correct outputs when the input is different from those present in the training data set. In some embodiments, the training of machine learning models can be implemented in an unsupervised manner (e.g., no labels or classifications may be supplied during training). Unsupervised models can be configured to perform anomaly detection, result clustering, etc. In some embodiments, the generation of golden trace data can be performed by an unsupervised machine learning model (e.g., by determining the correlation between various traces and metrology metrics and by determining the range of trace data that may result in acceptable metrology and / or performance of the substrate).

[0122] For each training data item in the training data set, the training data item can be input into the model (e.g., input into a machine learning model). Then, the model can process the input training data item (e.g., multiple measured dimensions of the manufactured device, a cartoon picture of the manufactured device, etc.) to generate an output. For example, the output can include a corrective action. The output can be compared with the label of the training data item (e.g., the corrective action taken to correct a problem associated with historical data). In some embodiments, an unsupervised model can be used to recommend corrective actions (e.g., the unsupervised model can learn the relationship between the tool constants and the substrate performance and provide updates to the predicted tool constants to improve the substrate performance).

[0123] Then, the processing logic can compare the generated output (e.g., the recommended corrective action) with the label (e.g., the actual corrective action) included in the training data item. The processing logic determines an error (i.e., classification error) based on the difference between the output and the label. The processing logic adjusts one or more weights and / or values of the model based on the error.

[0124] In the case of training a neural network, an error term or delta can be determined for each node in the artificial neural network. Based on this error, the artificial neural network adjusts one or more of its parameters (weights for one or more inputs to the node) for one or more of its nodes. The parameters can be updated in a backpropagation manner such that the nodes at the highest layer are updated first, then the nodes at the next layer, and so on. The artificial neural network includes multiple layers of "neurons", where each layer receives as input the values from the neurons at the previous layer. The parameters of each neuron include the weights associated with the values received from each of the neurons at the previous layer. Thus, adjusting the parameters can include adjusting the weights assigned to each of the inputs for one or more neurons at one or more layers in the artificial neural network.

[0125] One or more operations of system 300 can be performed by a statistical model. The statistical model can utilize the input data to determine output data via one or more statistical operations. The operations of system 300 can be performed by a heuristic or rule-based model.

[0126] System 300 can use multiple sets of features of training set 302 (e.g., the first set of features of training set 302, the second set of features of training set 302, etc.) to train multiple models. For example, system 300 can train a model to generate a first trained model using the first set of features in the training set (e.g., sensor data from sensors 1 to 10, metrology measurements 1 to 10, etc.), and train a second trained model using the second set of features in the training set (e.g., sensor data from sensors 11 to 20, metrology measurements 11 to 20, etc.). In some embodiments, the first trained model and the second trained model can be combined to generate a third trained model (e.g., which can be a better predictor or synthetic data generator than the first or second trained model itself). In some embodiments, the groups of features used to compare the models can overlap (e.g., the first set of features is sensor data from sensors 1 to 15, while the second set of features is sensors 5 to 20). In some embodiments, hundreds of models can be generated, including models with various permutations of features and combinations of models.

[0127] At block 314, system 300 performs model validation using validation set 304 (e.g., via Figure 1verification engine 184). System 300 can use the corresponding feature sets of the validation set 304 to verify each of the trained models. For example, system 300 can use the first set of features in the validation set (e.g., sensor data from sensors 1 to 10 or metrology measurements 1 to 10) to verify the first trained model, and use the second set of features in the validation set (e.g., sensor data from sensors 11 to 20 or metrology measurements 11 to 20) to verify the second trained model. In some embodiments, system 300 can verify hundreds of models generated at block 312 (e.g., models with various permutations of features, combinations of models, etc.). At block 314, system 300 can determine the accuracy of each of the one or more trained models (e.g., via model verification), and can determine whether the accuracy of one or more of the trained models meets a threshold accuracy. In response to determining that the accuracy of no trained model meets the threshold accuracy, the process returns to block 312, where system 300 performs model training using different sets of features of the training set. In response to determining that the accuracy of one or more of the trained models meets the threshold accuracy, the process continues to block 316. System 300 can discard the trained models that have an accuracy below the threshold accuracy (e.g., based on the validation set).

[0128] At block 316, system 300 performs model selection (e.g., via Figure 1 selection engine 185) to determine which of the one or more trained models that meet the threshold accuracy has the highest accuracy (e.g., the selected model 308 based on the validation at block 314). In response to determining that two or more of the trained models that meet the threshold accuracy have the same accuracy, the process can return to block 312, where system 300 performs model training using a further refined training set corresponding to a further refined set of features to determine the trained model with the highest accuracy.

[0129] At block 318, system 300 performs model testing using the test set 306 (e.g., via Figure 1The test engine 186) to test the selected model 308. System 300 can use the first set of features in the test set (e.g., sensor data from sensors 1 to 10) to test the first trained model to determine that the first trained model meets the threshold accuracy (e.g., based on the first set of features of test set 306). In response to the accuracy of the selected model 308 not meeting the threshold accuracy (e.g., the selected model 308 overfits the training set 302 and / or the validation set 304 and is not applicable to other data sets (e.g., the test set 306)), the process continues to block 312, where system 300 performs model training (e.g., retraining) using a different training set corresponding to a different set of features (e.g., sensor data from different sensors). In response to determining that the accuracy of the selected model 308 meets the threshold accuracy based on the test set 306, the process continues to block 320. In at least block 312, the model can learn patterns in the training data for making predictions or generating schedules for updating equipment constants, while in block 318, system 300 applies the model to the remaining data (e.g., the test set 306) to test the predictions.

[0130] At block 320, system 300 uses the trained model (e.g., the selected model 308) to receive current data 322 (e.g., current trajectory data associated with the most recently processed substrate, current equipment constants of the processing chamber, etc.) and determine (e.g., extract) equipment constant data 324 from the output of the trained model (e.g., Figure 1 the predicted data 168). Calibration actions associated with Figure 1 the manufacturing equipment 124 can be performed in view of the equipment constant data 324. In some embodiments, the current data 322 can correspond to the same type of features in the historical data used to train the machine learning model. In some embodiments, the current data 322 corresponds to a subset of the types of features in the historical data used to train the selected model 308 (e.g., the machine learning model can be trained using multiple metrology measurements and is configured to generate an output based on a subset of the metrology measurements).

[0131] In some embodiments, the performance of the machine learning model trained, validated, and tested by system 300 may deteriorate. For example, the manufacturing system associated with the trained machine learning model may experience gradual changes or sudden changes. Changes in the manufacturing system may cause a decline in the performance of the trained machine learning model. A new model can be generated to replace the machine learning model with declining performance. The new model can be generated by changing the old model through retraining, generating a new model, etc.

[0132] In some embodiments, one or more of acts 310 to 320 may occur in various orders and / or in conjunction with other acts not presented and described herein. In some embodiments, one or more of acts 310 to 320 may not be performed. For example, in some embodiments, one or more of the data partitioning of block 310, the model verification of block 314, the model selection of block 316, or the model testing of block 318 may not be performed.

[0133] Figure 3 Depicts a system configured to train, validate, test, and use one or more machine learning models. The machine learning models are configured to receive data as input (e.g., setpoints provided to manufacturing equipment, sensor data, metrology data, etc.) and provide data as output (e.g., prediction data, corrective action data, classification data, etc.). The input and / or output data may be processed, feature extracted, and formatted for the convenience of the model, ease of interpretation, etc. The blocks of system 300 for partitioning, training, validating, selecting, testing, and using may similarly be performed using different types of data to train a second model. Retraining may also be performed using current data 322 and / or additional training data 346.

[0134] Figures 4A to 4C Flowcharts of methods 400A to 400C related to training and utilizing models according to certain embodiments. Methods 400A to 400C may include training and utilizing machine learning models, statistical models, rule-based models, heuristic models, physics-based models, etc. Methods 400A to 400C may be associated with recommending and / or implementing corrective actions. Methods 400A to 400C may be associated with updating equipment constants of one or more processing chambers. Methods 400A to 400C may be performed by processing logic, which may include hardware (e.g., circuits, dedicated logic, programmable logic, microcode, processing devices, etc.), software (e.g., instructions running on a processing device, a general computer system, or a dedicated machine), firmware, microcode, or a combination of the above. In some embodiments, methods 400A to 400C may be partially performed by prediction system 110. Method 400A may be partially performed by prediction system 110 (e.g., Figure 1 server machine 170 of Figures 2A to 2BThe dataset generators 272A to 272B) execute. According to an embodiment of the present disclosure, the prediction system 110 may use method 400A to generate a dataset for training, validating, or testing at least one of the machine learning models. Methods 400B to 400C may be executed by the prediction server 112 (e.g., the prediction component 114) and / or the server machine 180 (e.g., the training, validation, and testing operations may be executed by the server machine 180). In some embodiments, the non-transitory machine-readable storage medium stores instructions that, when executed by a processing device (e.g., the processing device of the prediction system 110, the server machine 180, the prediction server 112, etc.), cause the processing device to execute one or more of methods 400A to 400C.

[0135] For simplicity of illustration, methods 400A to 400C are depicted and described as a series of operations. However, the operations according to the present disclosure may occur in various orders and / or simultaneously, and in conjunction with other operations not presented and described herein. In addition, not all of the illustrated operations may be performed to implement methods 400A to 400C according to the disclosed subject matter. Further, those skilled in the art will understand that methods 400A to 400C may alternatively be represented as a series of interrelated states via a state diagram or events.

[0136] Figure 4A A flowchart of method 400A for generating a dataset for a machine learning model according to some embodiments. Referring to Figure 4A In some embodiments, at block 401, the processing logic for implementing method 400A initializes the training set T to an empty set.

[0137] At block 402, the processing logic generates a first data input (e.g., a first training input, a first validation input), which may include one or more of sensors, manufacturing parameters, metrology data, etc. In some embodiments, the first data input may include a first set of features for the type of data, while the second data input may include a second set of features for the type of data (e.g., regarding Figure 3 as described). The input data may include historical data.

[0138] In some embodiments, at block 403, the processing logic optionally generates a first target output for one or more of the data inputs (e.g., the first data input). In some embodiments, the input includes one or more golden traces, and the target output includes a recommended update for the tool constant. In some embodiments, the input further includes additional data (e.g., metrology data, processing chamber tool constants, and / or processing chamber trace data), and the target output includes a recommended schedule for the tool constant update. In some embodiments, no target output is generated (e.g., an unsupervised machine learning model can group or find correlations in the input data rather than provide a target output).

[0139] At block 404, the processing logic optionally generates mapping data indicating an input / output mapping. The input / output mapping (or mapping data) can refer to the data input (e.g., one or more of the data inputs described herein), the target output for the data input, and the association between the data input and the target output. In some embodiments, for example, in the situation associated with a machine learning model that does not provide a target output, block 404 may not be performed.

[0140] At block 405, in some embodiments, the processing logic adds the mapping data generated at block 404 to the data set T.

[0141] At block 406, based on whether the data set T is sufficient for training, validating, and / or testing at least one of the machine learning models (e.g., Figure 1 model 190), the processing logic branches. If the data set T is sufficient, the execution proceeds to block 407; otherwise the execution continues back to block 402. It should be noted that in some embodiments, the sufficiency of the data set T can be determined simply based on the number of inputs in the data set that map to an output in some embodiments, while in some other embodiments, in addition to or instead of the number of inputs, the sufficiency of the data set T can be determined based on one or more other criteria (e.g., a measure of the diversity of the data examples, accuracy, etc.).

[0142] At block 407, processing logic provides a data set T (e.g., provided to server machine 180) to train, validate, and / or test machine learning model 190. In some embodiments, data set T is a training set and is provided to the training engine 182 of server machine 180 to perform training. In some embodiments, data set T is a validation set and is provided to the validation engine 184 of server machine 180 to perform validation. In some embodiments, data set T is a test set and is provided to the test engine 186 of server machine 180 to perform testing. In the case of a neural network, for example, input values of a given input / output mapping (e.g., numerical values associated with data input 210A) are input into the neural network, and output values of the input / output mapping (e.g., numerical values associated with target output 220A) are stored in the output nodes of the neural network. Then, the connection weights in the neural network are adjusted according to a learning algorithm (e.g., backpropagation, etc.), and the procedure is repeated for other input / output mappings in data set T. After block 407, the model (e.g., model 190) can be trained using the training engine 182 of server machine 180, validated using the validation engine 184 of server machine 180, or tested using the test engine 186 of server machine 180, at least one of which. The trained model can be implemented by the prediction component 114 (of prediction server 112) to generate prediction data 168 for performing signal processing, generate golden trace data, or perform calibration actions associated with manufacturing equipment 124.

[0143] Figure 4B FIG. 400B is a flow chart of a method 400B for updating equipment constants of a processing chamber according to some embodiments. At block 410, data is provided as input to a first trained machine learning model. The data provided includes trace data. The trace data can be golden trace data. The trace data can be associated with a substrate, a substrate processing procedure, etc. The trace data can be associated with a substrate processing procedure that causes the substrate to meet one or more criteria. The trace data can be associated with a substrate processing procedure that causes the substrate to meet one or more performance thresholds. The data provided as input includes golden equipment constants. The data provided as input includes the trace data of a first processing chamber. The data provided as input includes the equipment constants of a first processing chamber.

[0144] In some embodiments, additional input data may be provided to the first trained machine learning model. Metrology data may be provided to the first trained machine learning model. The metrology data may include golden metrology data, metrology data associated with golden track data, metrology data associated with the first processing chamber, and the like. The first trained machine learning model may be configured to recommend adjustments to the manufacturing equipment to increase the acceptable metrology data or the similarity between the golden metrology data and the current metrology data. The first trained machine learning model may be configured to adjust the equipment constants to increase the similarity between the substrates processed by one or more golden processing chambers and the substrates processed by the first processing chamber. The first trained machine learning model may be configured to adjust the equipment constants to increase the similarity between the track data from the first processing chamber and the golden track data. In some embodiments, the first trained machine learning model is configured to adjust the equipment constants of the first processing chamber for the track data within the limits defined by the golden track data. For example, the golden track data may define a golden track upper limit and a golden track lower limit, and the machine learning model may recommend an update of the equipment constants to increase the likelihood that the first processing chamber generates track data within the golden track limits.

[0145] In some embodiments, the golden track data may be provided to the first trained machine learning model by a second trained machine learning model. The second trained machine learning model may be configured to generate one or more sets of golden track data. The golden track data may include data associated with a single substrate. The golden track data may include data associated with multiple substrates. The golden track data may include data from a single chamber (e.g., a single golden chamber). The golden track data may include data from multiple processing chambers.

[0146] In some embodiments, metrology data can be provided as input to a second trained machine learning model. The metrology data can be associated with acceptable products (e.g., products that meet one or more performance value thresholds). The metrology data can be associated with a golden processing chamber (e.g., metrology of substrates processed by the golden chamber). Trajectory data can be further provided as input to the second trained machine learning model. The input trajectory data can be trajectory data associated with the metrology data (e.g., trajectory sensor data collected during processing of substrates associated with the input metrology data). The input trajectory data can be trajectory data of one or more golden chambers. The second machine learning model can be configured to generate a golden trajectory according to one or more criteria. The golden trajectory data can be associated with one or more substrates that meet one or more criteria, one or more performance thresholds, etc. For example, the second machine learning model can be configured to select golden trajectory data based on the likelihood of the process indicated by the trajectory data that produces substrates meeting one or more performance thresholds. The second machine learning model can generate a mapping between the trajectory data and the metrology values. The second trained machine learning model can generate a mapping between the trajectory data and the substrate performance. The second trained machine learning model can utilize the mapping to generate upper and lower golden trajectory data.

[0147] The golden trajectory data can be selected to correspond to the upper and lower limits of the trajectory data values corresponding to the target likelihood of substrates meeting one or more performance thresholds. The upper golden trajectory can include the highest trajectory values of multiple input trajectory data (e.g., the highest trajectory values corresponding to acceptable end products). The upper limit golden trajectory can include the statistical upper bounds of multiple trajectories (e.g., based on quartile or standard deviation analysis). The lower limit trajectory data can be selected in a manner similar to that of the upper limit trajectory data. The upper and lower limits can generate a guard band for the golden trajectory data. The second trained machine learning model can sort multiple trajectory data (e.g., organize multiple trajectory data from lowest to highest). The second trained machine learning model can base the order on the mean, median, the difference between each time step and the mean of multiple trajectories at that time step, or another metric. The second trained machine learning model can be configured to generate golden trajectory data according to one or more configuration settings. For example, the second trained machine learning model can be configured to be sensitive to specific substrate defects and can generate upper and lower limits of the golden trajectory data to avoid generating substrates including the target defects.

[0148] In some embodiments, after a training operation, additional data can be provided to a second trained machine learning model. The additional data (e.g., data associated with substrates processed after the initial training operation) can be used to retrain the second trained machine learning model. Input data can be provided to the second trained machine learning model, and the second trained machine learning model can adjust one or more weights or biases (e.g., retrain) based on the input data. The second trained machine learning model can assign a higher weight to newer data (e.g., data associated with most recently produced substrates) relative to less new data. The second trained machine learning model can assign a higher weight to newer data relative to less new data.

[0149] At block 412, the processing logic obtains a recommended update of a first equipment constant of a first processing chamber from the first trained machine learning model as an output. The processing logic can obtain multiple recommended updates of multiple equipment constants of the first processing chamber. The processing logic can further obtain one or more recommended updates of equipment constants of a second processing chamber. The processing logic can receive a schedule of the recommended updates (e.g., the processing logic can receive the order to update the equipment constants).

[0150] At block 414, the processing logic updates the first equipment constant of the first processing chamber in response to obtaining the output from the first trained machine learning model. Updating the equipment constant can include changing a value associated with the operation of the first processing chamber. Updating the equipment constant can include scheduling maintenance (e.g., a calibration operation). Updating the equipment constant can include initiating a maintenance operation (e.g., a calibration operation).

[0151] Figure 4C A flowchart of method 400C for performing a correction action associated with a processing chamber according to some embodiments. At block 420, a processing device (e.g., processing logic) receives first trajectory data associated with a first processing chamber. The first processing chamber meets one or more performance metrics. The first processing chamber can be a golden chamber. The first trajectory data can be or include golden trajectory data. The first trajectory data and / or the first processing chamber can be associated with one or more processing operations that meet one or more conditions. The first trajectory data and / or the first processing chamber can be associated with one or more processing operations that result in the substrate meeting the performance metrics.

[0152] The processing logic can further receive first metrology data of the first substrate. The first substrate can be associated with first trajectory data. The first substrate can meet one or more performance metrics. The first substrate can be part of a set of substrates having acceptable properties (e.g., the first substrate can be associated with designating the first processing chamber as a gold chamber). The processing logic can further receive a first set of equipment constants associated with the first processing chamber.

[0153] At block 422, the processing logic generates target trajectory data based on the first trajectory data associated with the first processing chamber. The target trajectory data can be or include gold trajectory data. The target trajectory data can include upper and lower limits. The target trajectory data can include a guard band. The target trajectory data can be generated based on multiple sets of trajectory data. The target trajectory data can be generated based on data from multiple processing chambers. The target trajectory data can be generated based on data associated with multiple substrates. The target trajectory data can contain and / or define a range of trajectory data values. The target trajectory data can contain and / or define a range of trajectory data values that meet one or more performance metrics. The target trajectory data can contain and / or define a range of trajectory data values that may be associated with a manufacturing process that meets one or more performance metrics.

[0154] At block 424, the processing logic receives second trajectory data associated with a second processing chamber. The second processing chamber does not meet one or more performance metrics. The second processing chamber may not be operating at a target level (e.g., target levels of energy efficiency, time efficiency, environmental impact, target profitability, etc.). The processing logic can further receive second metrology data of a second substrate associated with the second trajectory data. The processing logic can further receive a second set of equipment constants associated with the second processing chamber. The processing logic can further receive third trajectory data associated with a third processing chamber. The third trajectory data and the third processing chamber can share one or more characteristics with the second trajectory data and the second processing chamber. The third processing chamber may not meet the same set of performance metrics that the second processing chamber does not meet. The third processing chamber may not meet a different set of performance metrics than the performance metrics that the second processing chamber does not meet.

[0155] At block 426, the processing logic generates a first recommended correction action associated with a second processing chamber. The first recommended correction action is generated based on target trajectory data and second trajectory data. The first recommended correction action includes updating one or more equipment constants of the second processing chamber. The first recommended correction action can be generated in response to second trajectory data that is different from the target trajectory data. The first recommended correction action can be generated in response to second trajectory data that differs from the target trajectory data by a target value, a target ratio, etc. The first recommended correction action can be generated in response to second trajectory data that includes values outside an acceptable range (e.g., values of a target quantity). The acceptable range can be defined by the target trajectory data, can include the target trajectory data, can be based on the target trajectory data, etc.

[0156] The generation of the first recommended correction action can be further performed in view of metrology data (e.g., metrology data of a first substrate and a second substrate). The generation of the first recommended correction action can be further performed in view of equipment constants (e.g., a first set of equipment constants and a second set of equipment constants). The generation of the first recommended correction action can be performed as part of generating a plurality of recommended correction actions. The plurality of actions can include actions for one or more optimization goals (e.g., minimizing environmental impact, maximizing processing throughput, etc.). The plurality of actions can include updates of a plurality of equipment constants. The plurality of actions can include actions for a plurality of processing chambers. Generating the first recommended correction action can include generating a schedule for implementing two or more recommended correction actions. The correction actions can be scheduled according to the updates. The correction actions can be scheduled such that one or more manufacturing processes occur between the updates (e.g., to monitor the difference in processing chamber performance due to the first update before performing the second update). The correction actions can be scheduled such that at least one substrate is processed by the processing chamber between the correction actions associated with the processing chamber. The correction actions can be scheduled such that at least one substrate is processed by the processing chamber between the updates of the equipment constants of the processing chamber. The plurality of correction actions can include correction actions for a plurality of processing chambers (e.g., one or more correction actions associated with a second processing chamber and one or more correction actions associated with a third processing chamber). The correction actions can be used to cause a group of processing chambers to be consistent, to perform according to one or more performance metrics, etc.

[0157] The generation of the first recommended correction action can be performed by a trained model. The generation of the first recommended correction action can be performed by a plurality of models, an overall model, etc. Generating the first recommended correction action can include operations performed by one or more statistical models, one or more rule-based models, one or more heuristic models, one or more machine learning models, etc.

[0158] Generating a first recommended corrective action (e.g., as part of a plurality of recommended corrective actions) can include providing target trajectory data and second trajectory data to a trained model (e.g., a trained machine learning model, an overall model, etc.). Generating a first recommended corrective action can further include receiving an output from the trained model. The output can indicate one or more recommended corrective actions (e.g., including the first recommended corrective action). Generating a first recommended corrective action can further include scheduling the execution of the first recommended corrective action.

[0159] At block 428, the processing logic executes the first recommended corrective action.

[0160] Figure 4D A flowchart of a method for adjusting equipment constants of chambers in a chamber group according to some embodiments. At block 430, the processing device receives data indicating the performance of a plurality of processing chambers. The plurality of processing chambers can be a chamber group. The plurality of processing chambers can be the processing chambers of one or more processing tools. The plurality of processing chambers can be included in one or more facilities, manufacturing facilities, etc. The processing device can be, for example, a central server associated with a manufacturing facility.

[0161] At block 432, the processing logic provides data indicating the performance of a plurality of processing chambers to the model. The model can be a trained machine learning model. The model can be a statistical model, a rule-based model, a heuristic model, a physics-based model, etc. The model can be an overall model (e.g., can include one or more individual models, one or more trained machine learning models, various types of models, etc.). The model can recommend corrective actions. The model can recommend equipment constant updates. The model can recommend a schedule for performing equipment constant updates and / or corrective actions. The model can update a previous schedule for performing corrective actions.

[0162] Data indicating the performance of a plurality of processing chambers can include trajectory data. Data indicating the performance of a plurality of processing chambers can include metrology data. Data indicating the performance of a plurality of processing chambers can include metrology data of substrates manufactured in the processing chambers of the plurality of processing chambers.

[0163] Data indicative of the performance of multiple processing chambers can include data associated with each of the multiple chambers. One or more chambers can be determined to perform in a satisfactory manner. One or more chambers can be determined to meet one or more performance metrics. One or more chambers can be determined to meet one or more performance metric thresholds. Performance metrics can include trajectory data metrics, metrology metrics, energy usage metrics, environmental impact metrics, etc. One or more chambers that meet the performance metrics can be designated as gold chambers. One or more chambers that meet the performance metrics can be designated as gold chambers associated with those metrics. For example, a chamber that meets the performance metrics associated with a gas flow system can be designated as a gold chamber for gas flow system metrics. Trajectory data from one or more chambers that meet the performance metrics can be designated as gold trajectory data. Trajectory data from one or more chambers that meet the performance metrics can be used to generate gold trajectory data. Gold trajectory data can be associated with a standard of performance (e.g., a level of performance that a chamber is to conform to). The standard of performance can include target metrology data, a range of target metrology data, etc. The standard of performance can include target trajectory data, gold trajectory data, a range of trajectory data, etc. Data associated with one or more chambers that meet the performance metrics can be used to generate one or more standards of performance. If chamber performance meets one or more standards of performance, the operation of the chamber can be considered acceptable. In response to chamber performance not meeting the standards of performance, the chamber can have corrective actions associated with its execution. If the performance of a processing chamber does not conform to the standards associated with a gold chamber, the processing chamber can trigger a corrective action. If data indicative of the performance of a processing chamber (e.g., trajectory data, metrology data) does not meet one or more performance standards, the processing chamber can trigger a recommended corrective action. If the performance of a processing chamber is different from a performance standard, different from the performance of a target chamber, different from the performance of a gold chamber, etc., a corrective action can be recommended and / or executed in association with the processing chamber. The corrective action can be directed at one or more differences between the performance of the processing chamber and a performance metric, a performance standard, the performance of another processing chamber, etc. The corrective action can be directed at reducing one or more differences between the performance of two processing chambers, between a performance metric of a processing chamber and a performance standard, etc.

[0164] At block 434, the processing logic receives an output from the model. The output includes a first recommended tool constant update associated with a first processing chamber of the multiple processing chambers. The output further includes a second recommended tool constant update associated with a second processing chamber of the multiple processing chambers.

[0165] At block 436, the processing logic updates a first equipment constant of a first processing chamber. The processing logic further updates a second equipment constant of a second processing chamber. The update of the first and second equipment constants is performed in view of a first recommended equipment constant update and a second recommended equipment constant update. In some embodiments, for different processing chambers, the first and second equipment constants can be the same constant. For example, the same calibration table associated with a particular system, subsystem, or chamber component can be updated for two processing chambers. The updates can be the same or different for the processing chambers. In some embodiments, the first and second equipment constants can be associated with different equipment constants (e.g., constants associated with different operations, properties, systems, and / or components of the processing chamber).

[0166] Figure 4E FIG. 400E is a flow diagram of an example method 400E for performing calibration actions associated with one or more chambers in a group of chambers, according to some embodiments. Figure 4E And the associated description is meant to be illustrative, providing additional clarity of example applications associated with the present disclosure, and not limiting.

[0167] At block 440, a processing operation is performed on a plurality of substrates using a plurality of processing chambers. The processing chambers can be a group of chambers. The processing operation can be one or more processing steps, can include a plurality of sub-operations, etc. The processing operation can include processing performed in one or more processing chambers, one or more types of processing chambers, etc. In some embodiments, the target processing operation can include processing performed between introducing a substrate into a chamber and removing the substrate from the chamber. The processing operation can be a small part of the total processing of manufacturing a substrate.

[0168] At block 442, data indicating the performance of the plurality of processing chambers is received. The data can include trace data. The data can include metrology data. The data can include recipe data. The data can include equipment constants. The data can be used to identify one or more chambers that meet a performance metric (e.g., a metrology metric or a trace data metric of a product). The data can be used to identify one or more golden chambers associated with a target processing operation. The data can be used to generate golden trace data. The data can be analyzed to standardize the performance of the group of chambers, improve the performance of the group of chambers, adjust the performance targets and / or metrics of the chambers, etc. The data can be analyzed to determine chamber outliers. The data can be analyzed to determine that one or more chambers are outliers in terms of performance, equipment constants, etc.

[0169] At block 444, differences between data associated with various processing chambers are compared. For example, differences in trajectory data, metrology data, tool constant data, etc. can be considered. The impact of the differences in the data can be determined. Differences in metrology data may be related to differences in trajectory data. Differences in metrology and / or trajectory data can be correlated and / or mitigated by differences in tool constants or recipes. Determining the correlation between tool constants, recipes, trajectory data, and metrology data can be performed by a model. Determining the correlation between the data can be based on subject matter expertise. Determining the correlation between the data can be performed by a trained machine learning model. For example, one chamber can produce substrates having one or more properties indicative of non-ideal performance of the pressure system, while another chamber can produce substrates having properties indicative of non-ideal performance of the radio frequency (RF) system. Associated tool constants can be updated to vary / improve the performance of the chambers in the plurality of processing chambers.

[0170] At block 446, corrective actions can be recommended and / or executed. The corrective actions can be associated with one or more of the plurality of processing chambers. The corrective actions can include updating the tool constants of one or more of the processing chambers. The tool constants can be updated to reduce the differences between the chamber performances of the plurality of chambers in a group of chambers. The tool constants can be updated to reduce the differences between the metrology data of the processed substrates associated with different chambers. The tool constants can be updated to reduce the differences between the trajectory data associated with different chambers. For different chambers, the tool constants can be updated to different values (e.g., based on the performance of the chamber). Due to minor differences between chambers (e.g., component aging, manufacturing differences of components (e.g., within manufacturing tolerances), etc.), for different chambers, the tool constants can be updated to different values. The tool constant updates can be scheduled (e.g., the lowest risk changes can be made first, the most likely effective changes can be made first, the changes can be spaced out over time to allow substrates to be processed and data associated with those substrates to be analyzed between tool constant updates, etc.). The corrective actions can be directed at outliers. The corrective actions can be directed at chambers that are outliers in terms of performance, tool constants, etc.

[0171] Figure 5AA block diagram depicting a system 500A for performing operations associated with updating equipment constants of a processing chamber according to some embodiments. System 500A is an example system, and other systems including different data streams will be considered to be within the scope of the present disclosure. System 500A includes two trained machine learning models. System 500A includes a golden trajectory generation model 506 and a corrective action recommendation model 510. In some embodiments, the operation of one or both of these models can be performed by a physics-based model, a statistical model, a rule-based model, etc. The operation of the models of system 500A can be performed by more or fewer models. For example, the golden trajectory generation model 506 and the corrective action recommendation model 510 can be combined into a single monolithic model.

[0172] Golden chamber trajectory data 504 and golden chamber metrology data 502 are provided to the golden trajectory generation model 506. The golden chamber trajectory data 504 and the golden chamber metrology data 502 can be associated with one or more golden chambers. A golden chamber can be a chamber that produces acceptable products (e.g., products that meet performance thresholds). A golden chamber can be a chamber that meets performance thresholds over a time period, and the golden data from the chamber can be from the period when the chamber meets performance thresholds. A golden chamber can be a chamber that has a target likelihood of meeting performance thresholds (e.g., a target proportion of substrates processed by the chamber meets performance thresholds). The golden chamber trajectory data 504 can include trajectory data from a golden chamber, trajectory data from a golden chamber when the golden chamber is processing products that meet performance thresholds, etc. The golden chamber metrology data 502 can include metrology data of products processed by the golden chamber. The golden chamber metrology data 502 can include metrology data of products that meet performance thresholds. The golden chamber metrology data 502 and the golden chamber trajectory data 504 can be associated with the same set of products. The golden trajectory generation model 506 can be or include a machine learning model. The golden trajectory generation model 506 can be or include a physics-based model. The golden trajectory generation model 506 can be or include a heuristic model. The golden trajectory generation model 506 can be or include a rule-based model. The golden trajectory generation model 506 can be or include a statistical model.

[0173] The golden trajectory generation model 506 can be configured to generate golden trajectory data 508. The golden trajectory data 508 can include data associated with one or more processed products. The golden trajectory data 508 can include data associated with one or more sensors of the manufacturing equipment. The golden trajectory data 508 can include a single trajectory of a sensor (e.g., the golden trace data 508 can indicate the "ideal" or "optimal" trajectory from the provided golden chamber trajectory data 504). The golden trajectory data 508 can include multiple trajectories of a sensor (e.g., the golden trajectory data 508 can include an upper golden trajectory and a lower golden trajectory). Trajectory data that stays within the boundaries of the upper and lower golden trajectory data (e.g., from other processing chambers) may not be considered abnormal. The upper and lower golden trajectories can define upper and lower limits, can define a guard band, etc. The golden trajectory data 508 can include data from multiple processing runs, multiple processing chambers, etc. Selecting a trajectory for the golden trajectory data 508 can include extracting the correlation between the trajectory data and the metrology data (e.g., mapping the impact of the trajectory data on the metrology data). Selecting a trajectory for the golden trajectory data 508 can include selecting trajectory data associated with a product that presents an acceptable metrology metric of the type associated with the selected trajectory data. The golden trajectory data can be measurement data or synthetic data. Synthetic golden trajectory data can be generated by a subject matter expert. Synthetic golden trajectory data can be generated by a model. Synthetic golden trajectory data can be generated by a machine learning model (e.g., a regression neural network). Synthetic golden trajectory data can be generated by a statistical or heuristic model. Synthetic golden trajectory data can be generated by a physics-based model. Synthetic golden trajectory data can be generated by a digital twin model (e.g., a virtual representation of a physical equipment (e.g., a manufacturing chamber)).

[0174] The golden trajectory data 508 is provided to the corrective action recommendation model 510. The corrective action recommendation model 510 can recommend a corrective action, can schedule the execution of the corrective action, can cause the execution of the corrective action, etc. The corrective action recommendation model 510 can generate data including the recommended corrective action 520 as an output. The corrective action recommendation model 510 can recommend an update to one or more equipment constants of one or more processing chambers in a group of processing chambers.

[0175] The calibration action recommendation model 510 can receive further inputs. The model can receive a set of trajectory data 516. The model can receive the golden chamber equipment constants 518. The model can receive a set of equipment constants 514. The model can receive a set of metrology data 512. The model can receive the golden chamber metrology data 502. The set of metrology data 512 and the golden chamber metrology data 502 can optionally be provided to the calibration action recommendation model 510. In some embodiments, the calibration action recommendation model 510 can be configured to recommend calibration actions to increase the similarity between the processed products and the products represented in the golden metrology data. In some embodiments, the calibration action recommendation model 510 can be configured to recommend calibration actions to increase the similarity between the trajectory data of one or more chambers in the processing chamber set and the golden trajectory data.

[0176] Figure 5B FIG. is a block diagram depicting the operation of a calibration action recommendation model 530 according to some embodiments. The calibration action recommendation model 530 can be Figure 5A the calibration action recommendation model 510. The calibration action recommendation model 530 can be a single model, a set of models, an integrated model, etc. The calibration action recommendation model 530 can include one or more machine learning models, heuristic models, rule-based models, statistical models, etc. In some embodiments, some operations of the calibration action recommendation model 530 can be performed by a user and / or a subject matter expert.

[0177] The calibration action recommendation model 530 includes a trajectory-to-metrology correlation 532. The trajectory-to-metrology correlation 532 can include one or more models. The trajectory-to-metrology correlation 532 can identify the relationship between the metrology output of a manufacturing process and the trajectory data. The trajectory-to-metrology correlation 532 can identify the causal relationship between the trajectory sensor data and the metrology data. The trajectory-to-metrology correlation 532 can predict the metrology data based on the trajectory data. Similar models can be applied to other metrics (e.g., correlating trajectory data with environmental impact, energy usage, throughput, etc.).

[0178] The calibration action recommendation model 530 includes an equipment-constant-to-trajectory correlation 534. The parameter-to-trajectory correlation can include correlating manufacturing parameters with the trajectory data. The parameter-to-trajectory correlation 534 can include determining the causal relationship between the manufacturing parameters and the trajectory data. The parameter-to-trajectory correlation 534 can include determining the impact of recipes, setpoints, equipment constants, equipment components, etc. on the trajectory data. Similar models can be applied to other metrics (e.g., correlating parameter data with environmental impact, energy usage, throughput, etc.).

[0179] The calibration action recommendation model 530 includes a chamber differentiation 536. The chamber differentiation 536 can include determining differences between chambers, tools, multiple sets of processing equipment, etc. The chamber differentiation 536 can determine how different chambers vary in response to changed parameters, equipment constants, trajectory data, etc. For example, the chamber differentiation 536 can allow the calibration action recommendation model 530 to compensate for differences in installed components, variations within the manufacturing tolerances of components of the manufacturing equipment, differences when the chamber ages, etc.

[0180] The calibration action recommendation model 530 includes a parameter update schedule 538. The parameter update schedule 538 can perform operations for determining the location, timing, conditions, etc. for performing parameter updates. The parameter update schedule 538 can include scheduling updates of equipment constants. Updating equipment constants (e.g., as opposed to updating recipes) can allow the same recipe to be executed on multiple chambers, where chamber differences are compensated for by applying equipment constant updates on a chamber-by-chamber basis. The parameter update schedule 538 can include determining the risk of parameter change (e.g., the likelihood that a parameter change has a negative impact on processing performance). The parameter update schedule 538 can include determining the effectiveness of a parameter change (e.g., the likelihood that a parameter change has an expected impact on trajectory data, metrology data, or another output metric). The parameter update schedule 538 can include scheduling conditional updates (e.g., some updates can be scheduled to be conditional on trajectory data or another output metric within a time period). The parameter update schedule 538 can include selecting different updates for different chambers. After multiple processing runs, data can be consulted to determine the effectiveness of various updates, schedule additional updates, etc.

[0181] The calibration action recommendation model 530 can include further components. The calibration action recommendation model 530 can include fewer components. The calibration action recommendation model 530 can be configured to recommend calibration actions. The calibration action recommendation model 530 can be configured to recommend and / or implement parameter updates. The calibration action recommendation model 530 can be configured to recommend and / or implement equipment constant updates. The calibration action recommendation model 530 can be configured to perform chamber matching, cluster matching, and / or processing optimization routines.

[0182] Figure 6FIG. is a block diagram illustrating a computer system 600 in accordance with some embodiments. In some embodiments, the computer system 600 may be connected (e.g., via a network (e.g., local area network (LAN), internal network, external network, or the Internet)) to other computer systems. The computer system 600 may operate in the capacity of a server or a client computer in a client-server environment, or as a peer computer in a peer-to-peer or distributed network environment. The computer system 600 may be provided by a personal computer (PC), a tablet computer, a set-top box (STB), a personal digital assistant (PDA), a cellular phone, a network device, a server, a network router, a switch or bridge, or any device capable of executing a set of instructions (sequentially or otherwise) that specify actions to be taken by the device. Further, the term "computer" shall include any collection of computers that individually or jointly execute a set (or multiple sets) of instructions to perform any one or more of the methods described herein.

[0183] In a further aspect, the computer system 600 may include a processing device 602, a volatile memory 604 (e.g., random access memory (RAM)), a non-volatile memory 606 (e.g., read-only memory (ROM) or electronically erasable programmable ROM (EEPROM)), and a data storage device 618 that may communicate with each other via a bus 608.

[0184] The processing device 602 may be provided by one or more processors (e.g., a general-purpose processor such as, for example, a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a microprocessor implementing other types of instruction sets, or a microprocessor implementing a combination of types of instruction sets) or a special-purpose processor such as, for example, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), or a network processor.

[0185] The computer system 600 may further include a network interface device 622 (e.g., coupled to a network 674). The computer system 600 may also include a video display unit 610 (e.g., an LCD), an alphanumeric input device 612 (e.g., a keyboard), a cursor control device 614 (e.g., a mouse), and a signal generation device 620.

[0186] In some embodiments, the data storage device 618 may include a non-transitory computer-readable storage medium 624 (e.g., a non-transitory machine-readable medium) on which instructions 626 for encoding any one or more of the methods or functions described herein may be stored, the instructions 626 including for encoding Figure 1components (e.g., the prediction component 114, the corrective action component 122, the model 190, etc.) and instructions for implementing the methods described herein.

[0187] During execution of instructions 626 by computer system 600, the instructions 626 may also reside completely or partially within volatile memory 604 and / or within processing device 602, and thus, volatile memory 604 and processing device 602 may also constitute machine-readable storage media.

[0188] Although computer-readable storage medium 624 is shown as a single medium in the illustrative examples, the term "computer-readable storage medium" shall include a single medium or multiple media (e.g., a centralized or distributed database, and / or associated caches and servers) for storing one or more sets of executable instructions. The term "computer-readable storage medium" shall also include any tangible medium that is capable of storing or encoding a set of instructions for execution by a computer that cause the computer to perform any one or more of the methods described herein. The term "computer-readable storage medium" shall include, but is not limited to, solid-state memory, optical media, and magnetic media.

[0189] The methods, components, and features described herein may be implemented by discrete hardware components or may be integrated into the functionality of other hardware components (e.g., ASICs, FPGAs, DSPs, or similar devices). Additionally, the methods, components, and features may be implemented by firmware modules or functional circuitry within a hardware device. Additionally, the methods, components, and features may be implemented using any combination of hardware devices and computer program components or using a computer program.

[0190] Unless otherwise specifically stated, terms (e.g., "receive", "execute", "provide", "obtain", "cause", "access", "determine", "increase", "use", "train", "decrease", "generate", "correct", etc.) refer to actions and processes performed or implemented by a computer system that manipulate and transform data represented as a physical (electronic) quantity in a computer system register and memory into other data similarly represented as a physical quantity in a computer system register or memory or other such information storage, transmission, or display device. Additionally, the terms "first", "second", "third", "fourth", etc. used herein mean markers for distinguishing different elements and may not have an order meaning according to their numerical designations.

[0191] The examples described herein also relate to an apparatus for performing the methods described herein. This apparatus may be specifically constructed to perform the methods described herein or may include a general-purpose computer system selectively programmed by a computer program stored in a computer system. Such a computer program may be stored in a tangible computer-readable storage medium.

[0192] The methods and illustrative examples described herein are not inherently related to any particular computer or other apparatus. Various general-purpose systems may be used in accordance with the teachings described herein, or it may prove convenient to construct more specialized apparatus to perform the methods and / or each of their individual functions, routines, subroutines, or operations described herein. Examples of the structures of such systems are set forth in the above description.

[0193] The foregoing description is intended to be illustrative and not restrictive. Although the present disclosure has been described with reference to particular illustrative examples and embodiments, it will be recognized that the present disclosure is not limited to the examples and embodiments described. The scope of the present disclosure should be determined with reference to the following claims and the full scope of equivalents to which the claims are entitled.

Claims

1. A method, the method comprising: Receiving, by a processing device, first trajectory data associated with a first processing chamber, wherein the first processing chamber meets one or more performance metrics; Generating target trajectory data based on the first trajectory data associated with the first processing chamber; Receiving second trajectory data associated with a second processing chamber, wherein the second processing chamber does not meet the one or more performance metrics; Generating a first recommended corrective action associated with the second processing chamber based on the target trajectory data and the second trajectory data, wherein the first recommended corrective action includes updating one or more equipment constants of the second processing chamber; and Performing the first recommended corrective action.

2. The method of claim 1, the method further comprising: Receiving first metrology data of a first substrate associated with the first trajectory data; And Receiving second metrology data of a second substrate associated with the second trajectory data, wherein the first recommended corrective action is further based on the first metrology data and the second metrology data.

3. The method of claim 1, the method further comprising: Receiving a first set of equipment constants associated with the first processing chamber; And Receiving a second set of equipment constants associated with the second processing chamber, wherein the first recommended corrective action is further based on the first set of equipment constants and the second set of equipment constants.

4. The method of claim 1, the method further comprising generating a plurality of recommended corrective actions, wherein the plurality of recommended corrective actions includes the first recommended corrective action, and wherein generating the plurality of recommended corrective actions includes generating a schedule for implementing at least two of the plurality of recommended corrective actions.

5. The method of claim 1, the method further comprising: Receiving third trajectory data associated with a third processing chamber, wherein the third processing chamber does not meet the one or more performance metrics; Generating a plurality of recommended corrective actions, wherein the plurality of recommended corrective actions includes the first recommended corrective action and a second recommended corrective action, and wherein the second recommended corrective action is associated with the third processing chamber; and Performing the second recommended corrective action.

6. The method of claim 1, wherein the performance of the first recommended corrective action and the performance of the second recommended corrective action are scheduled such that at least one substrate is processed by the second processing chamber between the performance of the first recommended corrective action and the performance of the second recommended corrective action.

7. The method of claim 1, wherein generating the first recommended corrective action includes: Providing the target trajectory data and the second trajectory data to a trained model; Receiving an output from the trained model indicating the first recommended corrective action; And Scheduling the performance of the first recommended corrective action.

8. The method of claim 7, wherein the trained model includes a trained machine learning model.

9. The method of claim 1, wherein the target trajectory data includes a range of trajectory data values that meet one or more performance metrics.

10. A system includes a memory and a processing device coupled to the memory, wherein the processing device is configured to: Receive first trajectory data associated with a first processing chamber, wherein the first processing chamber meets one or more performance metrics; Generate target trajectory data based on the first trajectory data associated with the first processing chamber; Receive second trajectory data associated with a second processing chamber, wherein the second processing chamber does not meet the one or more performance metrics; Generate a first recommended corrective action associated with the second processing chamber based on the target trajectory data and the second trajectory data, wherein the first recommended corrective action includes updating one or more equipment constants of the second processing chamber; and Execute the first recommended corrective action.

11. The system of claim 10, wherein the processing device is further configured to: Receive first metrology data of a first substrate associated with the first trajectory data; and Receive second metrology data of a second substrate associated with the second trajectory data, wherein the first recommended corrective action is further based on the first metrology data and the second metrology data.

12. The system of claim 10, wherein the processing device is further configured to: Receive a first set of equipment constants associated with the first processing chamber; and Receive a second set of equipment constants associated with the second processing chamber, wherein the first recommended corrective action is further based on the first set of equipment constants and the second set of equipment constants.

13. The system of claim 10, wherein the processing device is further configured to generate a plurality of recommended corrective actions, wherein the plurality of recommended corrective actions includes the first recommended corrective action, and wherein generating the plurality of recommended corrective actions includes generating a schedule for implementing at least two of the plurality of recommended corrective actions.

14. The system of claim 10, wherein the processing device is further configured to: Receive third trajectory data associated with a third processing chamber, wherein the third processing chamber does not meet the one or more performance metrics; Generate a plurality of recommended corrective actions, wherein the plurality of recommended corrective actions includes the first recommended corrective action and a second recommended corrective action, and wherein the second recommended corrective action is associated with the third processing chamber; and Execute the second recommended corrective action.

15. The system of claim 10, wherein the target trajectory data includes a range of trajectory data values that meet one or more performance metrics.

16. A non-transitory machine-readable storage medium for storing instructions that, when executed, cause a processing device to perform operations, the operations including: Receive first trajectory data associated with a first processing chamber, wherein the first processing chamber meets one or more performance metrics; Generate target trajectory data based on the first trajectory data associated with the first processing chamber; Receive second trajectory data associated with a second processing chamber, wherein the second processing chamber does not meet the one or more performance metrics; Generate a first recommended correction action associated with the second processing chamber based on the target trajectory data and the second trajectory data, where the first recommended correction action includes updating one or more equipment constants of the second processing chamber; and Execute the first recommended correction action.

17. The non-transitory machine-readable storage medium according to claim 16, wherein the operation further includes: Receiving first metrology data of a first substrate associated with the first trajectory data; and Receiving second metrology data of a second substrate associated with the second trajectory data, wherein the first recommended correction action is further based on the first metrology data and the second metrology data.

18. The non-transitory machine-readable storage medium according to claim 16, wherein the operation further includes: Receiving a first set of equipment constants associated with the first processing chamber; and Receiving a second set of equipment constants associated with the second processing chamber, wherein the first recommended correction action is further based on the first set of equipment constants and the second set of equipment constants.

19. The non-transitory machine-readable storage medium according to claim 16, wherein the operation further includes generating a plurality of recommended correction actions, wherein the plurality of recommended correction actions includes the first recommended correction action, and wherein generating the plurality of recommended correction actions includes generating a schedule for implementing at least two of the plurality of recommended correction actions.

20. The non-transitory machine-readable storage medium according to claim 16, wherein generating the first recommended correction action includes: Providing the target trajectory data and the second trajectory data to a trained model; Receiving an output from the trained model indicating the first recommended correction action; and Scheduling the execution of the first recommended correction action.

Images