Calibrating the Processing Chamber

Abstract

The method includes receiving sensor data from a sensor associated with processing a substrate through a processing chamber of a substrate processing apparatus. The sensor data includes a first subset received from one or more first sensors and a second subset received from one or more second sensors, where the first subset is mapped to the second subset. The method further includes identifying model input data and model output data. The model output data is output from the physics-based model based on the model input data. The method further includes training the machine learning model with data inputs including the first subset and the model input data and target output data including the second subset and the model output data to adjust calibration parameters of the machine learning model. The calibration parameters are used by the physics-based model to perform corrective actions associated with the processing chamber.

Description

[Technical field]

[0001] The present disclosure relates to calibration, and more particularly, to calibration of a processing chamber. [Background technology]

[0002] The manufacturing equipment produces a product, e.g., a substrate processing equipment produces substrates. The sensor is used to provide sensor data associated with the manufacturing equipment. A parameter of the manufacturing equipment is selected based on the sensor data. Summary of the Invention

[0003] The following is a simplified summary of the disclosure to provide a basic understanding of some aspects of the disclosure. This summary is not an extensive overview of the disclosure. It is not intended to identify key or critical elements of the disclosure, nor to delineate the scope or claims of particular implementations of the disclosure. Its sole purpose is to present some concepts of the disclosure in a simplified form as a prelude to the more detailed description presented later.

[0004] In one aspect of the disclosure, a method includes receiving sensor data associated with processing a substrate through a processing chamber of a substrate processing apparatus from a plurality of sensors. The sensor data includes a first subset received from one or more first sensors and a second subset received from one or more second sensors, where the first subset is mapped to the second subset. The method further includes identifying model input data and model output data. The model output data is output from the physics-based model based on the model input data. The method further includes training a machine learning model with data inputs including the first subset and the model input data and target output data including the second subset and the model output data to adjust one or more calibration parameters of the machine learning model. The one or more calibration parameters are used by the physics-based model to perform one or more corrective actions associated with the processing chamber.

[0005] In another aspect of the disclosure, a method includes identifying one or more calibration parameters tuned by training a machine learning model. The machine learning model is trained with data inputs including a first subset of sensor data and first model input data, and target output data including a second subset of sensor data and first model output data. The sensor data is received from a plurality of sensors. The sensor data is associated with processing of a substrate through a processing chamber of the substrate processing apparatus, and the first model output data is output from a physics-based model based on the first model input data. The method further includes identifying second model input data. The method further includes receiving second model output data from the physics-based model in response to providing the second model input data and the calibration parameters as inputs to the physics-based model. One or more corrective actions associated with the processing chamber are performed based on the second model output data.

[0006] In another aspect of the disclosure, a non-transitory machine-readable storage medium stores instructions that, when executed, cause a processing device to perform operations including receiving sensor data from a plurality of sensors associated with processing a substrate through a processing chamber of a substrate processing apparatus. The sensor data includes a first subset received from one or more first sensors and a second subset received from one or more second sensors, where the first subset is mapped to the second subset. The processing device further identifies model input data and model output data. The model output data is output from the physics-based model based on the model input data. The processing device further trains the machine learning model using data inputs including the first subset and the model input data and target output data including the second subset and the model output data to adjust one or more calibration parameters of the machine learning model. The one or more calibration parameters are used by the physics-based model to perform one or more corrective actions associated with the processing chamber.

[0007] The present disclosure is illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings. [Brief description of the drawings]

[0008] [Figure 1A] FIG. 1 is a block diagram illustrating an example system (example system architecture) in accordance with certain embodiments. [Figure 1B] FIG. 2 is a block diagram illustrating data flow associated with adjusting calibration parameter data, in accordance with certain embodiments. [Diagram 2] FIG. 1 is a block diagram of an example dataset generator used to create a dataset for a machine learning model, in accordance with certain embodiments. [Diagram 3] FIG. 1 is a block diagram illustrating a system for generating calibration parameter data in accordance with certain embodiments. [Figure 4A] 1 is a flow diagram of a method associated with generating calibration parameter data for triggering corrective action, according to certain embodiments. [Figure 4B] 1 is a flow diagram of a method associated with generating calibration parameter data for triggering corrective action, according to certain embodiments. [Figure 4C] 1 is a flow diagram of a method associated with generating calibration parameter data for triggering corrective action, according to certain embodiments. [Diagram 5] FIG. 1 is a block diagram illustrating a computer system in accordance with certain embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Described herein are techniques directed to calibration of process chambers (e.g., methods for calibrating a process chamber digital twin).

[0010] Manufacturing equipment is used to produce products. For example, substrate processing equipment produces substrates (e.g., semiconductors, wafers, etc.). The substrate processing equipment has manufacturing parameters, such as hardware parameters and / or process parameters, that are used during substrate processing. For example, manufacturing parameters of a particular temperature and pressure in a processing chamber may be used during substrate processing. The processed (or partially processed) substrate has characteristic data (e.g., thickness data, roughness data). The manufacturing parameters of the substrate processing equipment are adjusted to try to produce a substrate (e.g., a good substrate) having characteristic data that matches the threshold characteristic data.

[0011] Traditionally, a model is created to attempt to mirror the processing chamber. The model is manually calibrated to attempt to determine manufacturing parameters to produce substrates having characteristic data that match the threshold characteristic data. Manual calibration requires a lot of time, trial and error, and is wasteful of materials. Given the many different manufacturing parameters, the many combinations of different values ​​of the manufacturing parameters, and time and material limitations, non-ideal manufacturing parameters are often determined by manual calibration. The use of non-ideal manufacturing parameters results in defective substrates, reduced yield, and inefficient processing.

[0012] The model may be created based on inferring a relationship between a first manufacturing parameter and a second manufacturing parameter. For example, the model may be created based on inferring between a temperature and a resulting pressure estimate. Sensor data from a sensor associated with the process chamber may then be used to update the estimate of the relationship between the manufacturing parameters. For example, measured temperature data from a temperature sensor and measured pressure data from a pressure sensor may be used to update the estimate of the relationship between the temperature data and the resulting pressure data. A manufacturing parameter may result from many other manufacturing parameters, some of which may not be measured. Conventional models that are updated based on sensor data are not fully updated because there are unmeasured manufacturing parameters that affect the manufacturing parameters of the model. Non-ideal models lead to the use of non-ideal manufacturing parameters, which results in defective substrates, reduced yields, and inefficient processing.

[0013] Traditionally, models are created and updated offline (when the processing chamber is not in use) and then used throughout the life of the processing chamber. As the processing chamber is used repeatedly, the relationships between different manufacturing parameters change (e.g., drift) due to material accumulation, aging of parts, changes during cleaning procedures, etc. For example, the relationship between temperature and pressure in a processing chamber drifts over time. Traditional models become increasingly inaccurate over time due to drift. As the models become more and more inefficient, more substrates are defective, yields are further reduced, and processing becomes more inefficient.

[0014] Traditionally, the same model may be used for different processing chambers. Because processing chambers are produced to tolerances, each processing chamber operates slightly differently. Because each processing chamber ages differently, each processing chamber experiences different types of drift. Traditional models used for many different processing chambers result in different levels of precision for each processing chamber, leading to inconsistencies in the substrates produced by the processing chamber, lower yields, and reduced efficiency.

[0015] The disclosed methods, devices, and systems provide solutions to the deficiencies of conventional systems. The disclosed methods provide for calibration of a process chamber (e.g., calibration of a process chamber digital twin).

[0016] The sensor data is received from a sensor. The sensor may be associated with the process chamber (e.g., disposed within the process chamber and providing sensor data for the process chamber, etc.). For example, the sensor may provide temperature data, pressure data, flow data, etc. during operation of the process chamber (e.g., processing of a substrate).

[0017] The sensor data includes different subsets of sensor data received from different sensors. A first subset of sensor data is received from one or more first sensors, and a second subset of sensor data is received from one or more second sensors, and the first subset is mapped to the second subset. In some embodiments, the first subset includes chamber pressure data, backside pressure data, heater temperature data, and gas flow data. Additionally, the first subset can include chamber component spacing data. The second subset can include chamber component temperature data.

[0018] Model input data and model output data are identified. The model input data is provided to a physics-based model, and the physics-based model generates (e.g., predicts) model output data based on the input model input data. In some embodiments, the model output data includes component temperature data associated with one or more components of the process chamber.

[0019] The first subset of sensor data and the model input data may correspond to one or more first types of data (e.g., the same first type of data). The second subset of sensor data and the model output data may correspond to one or more second types of data different from the first type of data (e.g., the same type of data). For example, the first subset of sensor data and the model input data may both include spacing, chamber pressure, backside pressure, heater temperature, and / or gas flow rate. The second subset of sensor data and the model output data may both include a temperature of the component.

[0020] The machine learning model can be trained with data inputs including a first subset of the sensor data and model input data, and target output data including a second subset of the sensor data and model output data to adjust calibration parameters of the machine learning model. The calibration parameters can include unmeasured or unmeasurable parameters, such as thermal contact resistance, thermal contact conductance, etc. After the machine learning model adjusts the calibration parameters, the calibration parameters can be used by the physics-based model to perform corrective actions associated with the process chamber and generate more accurate predictive variables.

[0021] The model input data and the adjusted calibration parameters can be input into the physics-based model to generate updated model output data. The adjusted calibration parameters can be provided to the physics-based model to calibrate the physics-based model and provide a more accurate digital twin of the processing chamber. In some embodiments, the physics-based model is a digital twin model of the processing chamber. The updated model output data is more accurate than the initial model output data generated by the physics-based model using an initial prediction of the calibration parameters (i.e., the unadjusted calibration parameters). The updated model output data can be used to perform a corrective action associated with the processing chamber. For example, the corrective action can include one or more of providing an alert, interrupting operation of the processing chamber, or updating manufacturing parameters of the processing chamber.

[0022] Aspects of the present disclosure provide technical advantages over conventional systems. Adjusting the calibration parameters of the physics-based model with the help of machine learning models can save time compared to manual calibration of conventional models. Manual calibration may no longer be necessary when using aspects of the present disclosure. Calibration parameters that may be difficult, impractical, or impossible to calculate manually using conventional methods can be more easily and accurately predicted using the methods of the present disclosure, resulting in more accurate and precise models compared to models of conventional methods. Furthermore, using the present disclosure, models can be created and updated online (as the processing chamber is in use), as opposed to conventional models that are created and updated offline. The present disclosure also accounts for drift in the processing chambers, leading to more accurate models that increase consistency between chambers, which in turn reduces defective substrates and increases yield, unlike traditional approaches. In addition, the methods of the present disclosure can be used to iterate the calibration parameters, generating increasingly accurate physics-based models over time, as compared to conventional methods. Furthermore, the methods of the present disclosure may be able to correct the physics-based models for drift in chamber characteristics and for differences between individual chambers. The physics-based models of the present disclosure therefore provide greater confidence in implementing corrective actions on the processing chamber, leading to more consistent and precise substrate manufacturing as compared to conventional methods.

[0023] 1A is a block diagram illustrating an example system 100A (an example system architecture) according to some embodiments. System 100 includes client devices 120, manufacturing equipment 124, sensors 126, a prediction server 112, and a data store 140. Prediction server 112 may be part of a prediction system 110. Prediction system 110 may further include server machines 170 and 180.

[0024] The sensors 126 can provide sensor data 142 associated with the manufacturing equipment 124 (e.g., associated with the manufacturing equipment 124 producing a corresponding product, such as a wafer, a substrate, a semiconductor, and / or a display). The sensor data 142 can be used, for example, for equipment health and / or product health (e.g., product quality). The manufacturing equipment 124 can run to produce a product according to a recipe or over a period of time. In some embodiments, the sensor data 142 can include one or more values ​​of temperature (e.g., heater temperature, component temperature), spacing (SP), pressure (e.g., chamber pressure, backside pressure), high frequency radio frequency (HFRF), electrostatic chuck (ESC) voltage, current, flow rate (e.g., gas flow rate, flow rate of one or more gases), power, voltage, etc. The sensor data 142 may include one or more subsets 144 of the sensor data 142 (e.g., a first subset 144 of the sensor data 142 from a first subset of the sensors 126 and a second subset 144 of the sensor data 142 from a second subset of the sensors 126). The sensor data 142 may be provided while the manufacturing equipment 124 is performing a manufacturing process (e.g., equipment readings as the product is processed). The sensor data 142 may vary from product to product (e.g., substrate to substrate). In some embodiments, the sensor data 142 is obtained from an experimental run of a process operation in a processing chamber.

[0025] In some embodiments, the sensor data 142 may be processed (e.g., by the client device 120 and / or by the prediction server 112). Processing the sensor data 142 may include generating features. In some embodiments, the features are patterns in the sensor data 142 (e.g., slope, width, height, peak, etc.) or combinations of values ​​from the sensor data 142 (e.g., power derived from voltage and current, etc.). The sensor data 142 may include features that may be used by the prediction system 110 to perform processing of the signal and / or to obtain model input data 154, model output data 156, and / or calibration parameter data 162 for performing corrective actions.

[0026] Each instance (e.g., set) of sensor data 142 may correspond to a product (e.g., board), a set of manufacturing equipment, a type of board produced by the manufacturing equipment, combinations thereof, etc. Data store 140 may further store information relating sets of different data types, e.g., information indicating that the sets of sensor data 142 and / or sets of model data are all associated with the same product, manufacturing equipment, type of board, etc.

[0027] In some embodiments, the predictive system 110 can generate the calibration parameter data 162 using supervised machine learning (e.g., a model 190B trained based on inputs including the first subset 144 of the sensor data 142 and the model input data 154, and a target output including the model output data 156 and the second subset 144 of the sensor data 142 to adjust the calibration parameter data 162). In some embodiments, the calibration parameter data 162 is used to calibrate a physics-based model (e.g., model 190A) of a processing chamber of the manufacturing tool 124. In some embodiments, the calibrated physics-based model is a digital twin model of the processing chamber of the manufacturing tool 124 that is used to perform corrective actions (e.g., update manufacturing parameters of the processing chamber). As used herein, a digital twin is a digital replica of a physical asset, such as a manufactured part (e.g., a manufacturing tool 124, a processing chamber, a substrate processing system, etc.). The digital twin includes properties of the physical asset at each stage of the manufacturing process, including, but not limited to, coordinate dimensions, weight properties, material properties (e.g., density, surface roughness), electrical properties (e.g., conductivity), optical properties (e.g., reflectivity), manufacturing parameters (e.g., hardware parameters, process parameters), etc.

[0028] The client devices 120, the manufacturing equipment 124, the sensors 126, the prediction server 112, the data store 140, the server machine 170, and the server machine 180 may be coupled to each other via a network 130 to generate the model output data 156 and / or the calibration parameter data 162, and optionally to perform corrective actions.

[0029] In some embodiments, the network 130 is a public network that provides the client device 120 with access to the prediction server 112, the data store 140, and / or other publicly available computing devices. In some embodiments, the network 130 is a private network that provides the client device 120 with access to the manufacturing equipment 124, the sensors 126, the data store 140, and / or other privately available computing devices. In some embodiments, the network 130 is a cloud-based network capable of performing cloud-based functions (e.g., providing cloud service functions to one or more devices in the system). The network 130 may include one or more wide area networks (WANs), local area networks (LANs), wired networks (e.g., Ethernet networks), wireless networks (e.g., 802.11 networks or Wi-Fi networks), cellular networks (e.g., Long Term Evolution (LTE) networks), routers, hubs, switches, server computers, cloud computing networks, and / or combinations thereof.

[0030] The client device 120 may include computing devices such as a personal computer (PC), a laptop, a mobile phone, a smartphone, a tablet computer, a netbook computer, a network-connected television ("smart TV"), a network-connected media player (e.g., Blu-ray player), a set-top box, an over-the-top (OTT) streaming device, an operator box, a cloud server, a cloud-based system (e.g., a cloud service device, a cloud network device), etc. The client device 120 may include a corrective action component 122. The corrective action component 122 may receive user input of instructions associated with the manufacturing equipment 124 (e.g., via a graphical user interface (GUI) displayed via the client device 120). In some embodiments, the corrective action component 122 sends instructions to the prediction system 110, receives output (e.g., model output data 156) from the prediction system 110, determines a corrective action based on the output, and causes the corrective action to be implemented.

[0031] In some embodiments, the prediction system 110 may further include a calibration component 116. The calibration component 116 may use the sensor data 142 and the model data 152 (e.g., associated with the physics-based model 190A) to train the machine learning model 190B. In some embodiments, the calibration component 116 may provide the model input data 154 to the physics-based model 190A to generate the model output data 156. The calibration component 116 may receive the model data 152 associated with the physics-based model 190A and input at least a portion of the model data 152 to the machine learning model 190B. The calibration component 116 may receive the calibration parameter data 162 from the model 190B. In some embodiments, the calibration component 116 provides the calibration parameter data 162 and the model input data 154 to the model 190A and receives the model output data 156 from the model 190A. The calibration component 116 provides the model output data 156 to the client device 120, which, in consideration of the model output data 156, triggers a corrective action via the corrective action component 122. In some embodiments, the corrective action component 122 obtains (e.g., from the data store 140, etc.) sensor data 142 associated with the manufacturing equipment 124 (e.g., the subset 144 of the sensor data 142) and provides the sensor data 142 associated with the manufacturing equipment 124 (e.g., the subset 144 of the sensor data 142) to the predictive system 110.

[0032] In some embodiments, the corrective action component 122 stores the sensor data 142 in the data store 140, and the prediction server 112 retrieves the sensor data 142 from the data store 140. In some embodiments, the prediction server 112 can store the output of the trained machine learning model 190B (e.g., model output data 156) in the data store 140, and the client device 120 can retrieve the output from the data store 140. In some embodiments, the corrective action component 122 receives corrective action instructions from the prediction system 110 and causes the corrective action to be implemented. In some embodiments, the corrective action includes updating manufacturing parameters (e.g., hardware parameters, process parameters) of the manufacturing equipment 124 based on the model output data 156. The client device 120 can include an operating system that enables a user to one or more of create, view, or edit data (e.g., instructions associated with the manufacturing equipment 124, corrective actions associated with the manufacturing equipment 124, etc.).

[0033] In some embodiments, the calibration parameter data 162 is associated with one or more values ​​of the manufacturing equipment 124 that are not measured or cannot be measured (e.g., calibration parameters of the manufacturing equipment 124). In some embodiments, the calibration parameters (e.g., the calibration parameter data 162) include one or more of a thermal contact conductance or a thermal contact resistance of junctions (e.g., where the components contact) of individual components of the manufacturing equipment 124. The interfaces and / or junctions between two solid objects of the manufacturing equipment 124 generate a thermal contact resistance (and a thermal contact conductance) for heat flow between the two solid objects. In some embodiments, the calibration parameter data 162 is a predicted calibration parameter value (e.g., a predicted thermal contact resistance or a predicted thermal contact conductance) of one or more components of the manufacturing equipment 124 based on the sensor data 142 (e.g., the current state of the manufacturing equipment 124) and / or the model data 152 (e.g., the predicted state of the manufacturing equipment 124). In some embodiments, thermal contact resistance or thermal contact conductance exists between one or more components of the processing chamber. For example, each pair of components in contact with each other has an associated thermal contact conductance (e.g., the time rate of steady-state heat flow between the components induced by a unit temperature difference) or thermal contact resistance (e.g., the ratio of the temperature drop across the interface to the average heat flow). Some such thermal relationships may include the thermal contact conductance or thermal contact resistance of the contacts (e.g., interfaces) between the showerhead and the lid of the processing chamber, between the heater and the shaft of the processing chamber, between the blocker plate and the lid of the processing chamber, between the blocker plate and the chamber sidewall of the processing chamber, etc. In some embodiments, the calibration parameter data 162 includes an indication of change or drift over time in any component of the manufacturing tool 124, the sensor 126, etc. In some embodiments, the calibration parameter data 162 reflects the process conditions of the manufacturing tool 124 (e.g., dependent on the temperature, pressure, etc. of the substrate processing operation). In some embodiments, the calibration parameter data 162 is used to calibrate a physics-based model of the processing chamber (eg, model 190A).

[0034] Running a manufacturing process that results in a defective product can be costly in terms of time, energy, products, components, manufacturing equipment 124, costs of identifying and discarding the defective product, etc. By inputting sensor data 142 (e.g., manufacturing parameters being used to manufacture the product) and model data 152, adjusting calibration parameter data 162, and performing corrective actions (e.g., calibrating a physics-based model of the processing chamber) based on the calibration parameter data 162, system 100 can have the technical advantage of avoiding the costs of producing, identifying, and discarding defective products by utilizing an accurate and calibrated physics-based model of the processing chamber.

[0035] The manufacturing parameters may not be optimal for producing the product, which may result in costly consequences such as increased consumption of resources (e.g., energy, coolant, gas, etc.), increased time to produce the product, increased component failures, increased amount of defective products, etc. By inputting the sensor data 142 and the model data 152 into the trained machine learning model 190B, adjusting the calibration parameter data 162, and taking corrective action (e.g., based on the calibration parameter data 162) to update the manufacturing parameters (e.g., calibrating the physics-based model 190A of the processing chamber to update the model data 152), the system 100 may have the technical advantage of avoiding the costly consequences of non-optimal manufacturing parameters by utilizing an accurate, calibrated physics-based model of the processing chamber using optimal manufacturing parameters (e.g., hardware parameters, process parameters, optimal design).

[0036] The corrective action may be associated with one or more of computational process control (CPC), statistical process control (SPC) (e.g., SPC of electronic components to determine the process under control, SPC to predict the useful life of a component, SPC for comparison to a 3 sigma graph, etc.), advanced process control (APC), model-based process control, preventive operational maintenance, design optimization, manufacturing parameter updates, manufacturing recipe updates, feedback control, machine learning corrections, model calibration (e.g., calibration of physics-based models), and the like.

[0037] In some embodiments, the corrective action includes providing a warning (e.g., an alarm to stop and not run the manufacturing process). In some embodiments, the corrective action includes providing feedback control (e.g., modifying manufacturing parameters in response to the calibration parameter data 162 and / or the model output data 156). In some embodiments, the corrective action includes providing machine learning (e.g., modifying one or more manufacturing parameters based on the calibration parameter data 162). In some embodiments, performing the corrective action includes causing an update of one or more manufacturing parameters. In some embodiments, the corrective action includes calibrating a physics-based model based on the calibration parameter data 162.

[0038] The manufacturing parameters may include hardware parameters (e.g., replacing a component, using a specific component, replacing a processing chip, updating firmware, etc.) and / or process parameters (e.g., temperature, pressure, flow rate, speed, current, voltage, gas flow, lift speed, etc.). In some embodiments, the corrective action includes triggering preventative operational maintenance (e.g., replacing, treating, cleaning, etc., of a manufacturing equipment 124 component). In some embodiments, the corrective action includes triggering design optimization (e.g., updating manufacturing parameters, manufacturing process, manufacturing equipment 124, etc., for an optimized product). In some embodiments, the corrective action includes updating a recipe (e.g., putting the manufacturing equipment 124 into an idle mode, sleep mode, warm-up mode, etc.). In some embodiments, the corrective action includes updating a physics-based model.

[0039] Prediction server 112, server machine 170, and server machine 180 may each include one or more computing devices such as a rack mount server, a router computer, a server computer, a personal computer, a mainframe computer, a laptop computer, a tablet computer, a desktop computer, a graphics processing unit (GPU), an accelerator application specific integrated circuit (ASIC) (e.g., a tensor processing unit (TPU)), etc. Prediction server 112, server machine 170, and server machine 180 may each include a cloud server or a server capable of performing one or more cloud-based functions.

[0040] The prediction server 112 can include a calibration component 116. In some embodiments, the calibration component 116 receives the sensor data 142 (e.g., a subset 144 of the sensor data 142) and the model data 152 (e.g., the model input data 154 and the model output data 156) and generates outputs (e.g., the calibration parameter data 162 and / or the model output data 156) for performing corrective actions associated with the manufacturing equipment 124. In some embodiments, the calibration component 116 can use a trained machine learning model 190B to determine the outputs for performing the corrective actions. In some embodiments, the calibration component 116 can use a physics-based model (e.g., the model 190A) to determine at least a portion of the outputs for performing the corrective actions.

[0041] In some embodiments, the sensor data 142 and the model data 152 are provided to train the machine learning model 190B. The machine learning model 190B is trained to adjust the calibration parameter data 162 used in the physics-based model of the process chamber. In some embodiments, the machine learning model 190B is trained based on inputs (e.g., the sensor data 142) of spacing data, chamber pressure data, heater temperature data, and / or chamber flow data from sensors 126 disposed at locations within the process chamber. In some embodiments, one or more sensors 126 are not disposed in close proximity to the substrate. For example, temperature sensors may be disposed at various locations near gas intakes, exhaust conduits, substrate supports, chamber walls, etc. The model 190B adjusts the calibration parameter data 162. In some embodiments, the calibration parameter data 162 includes thermal contact resistance or thermal contact conductance data at contact points between components of the process chamber.

[0042] Combining machine learning models with physics-based modeling provides technical advantages over other techniques. The physics-based model provides a map of manufacturing parameters (e.g., of a process chamber). Due to manufacturing tolerance ranges, component aging, etc., traditional physics-based models may have inaccuracies (e.g., may not accurately model the process chamber). For example, a heater may provide slightly less or more energy than expected, a gas flow regulator may not precisely tolerate a selected flow rate, contact between surfaces in the chamber may be somewhat less than ideal, etc. Such variations may be unknown to the user and may not be captured by traditional physics-based models. By using the machine learning model 190B to adjust the calibration parameter data 162 (e.g., implementing a machine learning model that performs virtual measurements of unmeasurable values, such as the calibration parameter data 162 including thermal contact resistance and / or thermal contact conductance values), the physics-based model 190A of the present disclosure becomes more accurate than traditional models. The machine learning model 190B is trained for various input parameters (e.g., sensor data 142, model data 152) spanning a region of the parameter space. Variations in the actual conditions in the process chamber (e.g., due to inaccuracies in the initial physics-based model) within the range of the parameter space in which the machine learning model was trained to operate can be accounted for via interpolation. In addition, settings somewhat outside the region of the parameter space used to train the machine learning model can be accounted for via extrapolation. In this manner, unexpected variations in the operating conditions of the process chamber can be accounted for by using the calibration parameter data 162 provided by the prediction system 110.

[0043] In some embodiments, the calibration component 116 may receive the sensor data 142 (e.g., a subset 144 of the sensor data 142) and perform pre-processing, such as extracting patterns in the data or combining the data into new composite data. In some embodiments, the calibration component 116 may receive the model data 152 (e.g., model input data 154 and model output data 156) associated with a physics-based model. The calibration component 116 may then provide the sensor data 142 and the model data 152 to train the machine learning model 190B. The machine learning model 190B may be trained using a data input including a first subset 144 of the sensor data 142 (e.g., a subset 144A of the sensor data 142 of FIG. 1B) and the model input data 154, and a target output data including a second subset 144 of the sensor data 142 (e.g., a subset 144B of the sensor data 142 of FIG. 1B) and the model output data 156. In some embodiments, the first subset 144A of the sensor data 142 and the model input data 154 correspond to one or more first types of data. In some embodiments, the second subset 144B of the sensor data 142 and the model output data 156 correspond to one or more second types of data. In some embodiments, the one or more second types of data are different from the one or more first types of data. The calibration component 116 can receive one or more calibration parameter values ​​(e.g., calibration parameter data 162) associated with the physics-based model (e.g., model 190A) of the process chamber from the trained machine learning model 190B. The calibration component 116 can then trigger a corrective action. The corrective action can include sending an alert to the client device 120. The corrective action can also include updating a manufacturing parameter of the manufacturing tool 124. The corrective action can also include updating the physics-based model (e.g., model 190A) based on the calibration parameter data 162 (e.g., calibration parameter values).

[0044] The data store 140 may be a memory (e.g., random access memory), a drive (e.g., hard drive, flash drive), a database system, or another type of component or device capable of storing data. The data store 140 may be in situ (e.g., a local computer) or remote (e.g., a cloud-based system). The data store 140 may include multiple storage components (e.g., multiple drives or multiple databases) that may span multiple computing devices (e.g., multiple server computers). The data store 140 may store the sensor data 142, the model data 152, and the calibration parameter data 162.

[0045] The sensor data 142 can include one or more subsets of the sensor data (e.g., subset 144 of the sensor data 142). The sensor data can include trace data over the duration of a manufacturing process, associations of data with physical sensors, pre-processed data (e.g., averages and composite data), and data indicative of sensor performance over time (i.e., many manufacturing processes).

[0046] The model data 152 may include features similar to the features of the sensor data 142 .

[0047] The calibration parameter data 162 may indicate one or more calibration parameter values ​​(e.g., calibration parameters) of the physically-based model 190A of the processing chamber of the manufacturing tool 124. The calibration parameter data 162 may also include predictions of the calibration parameter values.

[0048] In some embodiments, the prediction system 110 further includes a server machine 170 and a server machine 180. The server machine 170 includes a dataset generator 172 that can generate a dataset (e.g., a set of data inputs and a set of target outputs) for training, validating, and / or testing the machine learning model 190B. Some operations of the dataset generator 172 are described in more detail below with respect to FIG. 2 and FIG. 4A. In some embodiments, the dataset generator 172 can divide the sensor data (e.g., a subset 144 of the sensor data 142) and the model data (e.g., the model input data 154 and the model output data 156) into a training set (e.g., 60 percent of the data), a validation set (e.g., 20 percent of the data), and a test set (e.g., 20 percent of the data). In some embodiments, the prediction system 110 (e.g., via the calibration component 116) generates a plurality of feature sets. For example, a first feature set may correspond to a first type of sensor dataset (e.g., from a first sensor set, a first combination of values ​​from the first sensor set, a first pattern of values ​​from the first sensor set) corresponding to each dataset (e.g., training set, validation set, and test set), and a second feature set may correspond to a second type of sensor dataset (e.g., from a second sensor set different from the first sensor set, a second combination of values ​​different from the first combination, a second pattern different from the first pattern) corresponding to each dataset.

[0049] The server machine 180 includes a training engine 182, a validation engine 184, a selection engine 185, and / or a test engine 186. The engines (e.g., the training engine 182, the validation engine 184, the selection engine 185, and the test engine 186) may refer to hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, processing devices, etc.), software (e.g., instructions executed on a processing device, a general-purpose computer system, or a dedicated machine), firmware, microcode, or a combination thereof. The training engine 182 may be capable of training a machine learning model 190B using one or more feature sets associated with a training set from the dataset generator 172. The training engine 182 may generate multiple trained machine learning models 190B, each trained machine learning model 190B corresponding to a different feature set of the training set (e.g., sensor data from a different set of sensors). For example, a first trained machine learning model may have been trained using all features (e.g., X1-X5), a second trained machine learning model may have been trained using a first subset of features (e.g., X1, X2, X4), and a third trained machine learning model may have been trained using a second subset of features (e.g., X1, X3, X4, and X5) that may partially overlap with the first subset of features. Dataset generator 172 may receive the output of a trained machine learning model (e.g., model 190B), collect the data into training, validation, and testing datasets, and use the datasets to train a second machine learning model.

[0050] The validation engine 184 may be capable of validating the trained machine learning models 190B using the corresponding feature sets of the validation set from the dataset generator 172. For example, a first trained machine learning model 190B trained using a first feature set of the training set may be validated using a first feature set of the validation set. The validation engine 184 may determine the accuracy of each of the trained machine learning models 190B based on the corresponding feature sets of the validation set. The validation engine 184 may discard trained machine learning models 190B having an accuracy that does not meet a threshold accuracy. In some embodiments, the selection engine 185 may be capable of selecting one or more trained machine learning models 190B having an accuracy that meets the threshold accuracy. In some embodiments, the selection engine 185 may be capable of selecting the trained machine learning model 190B having the highest accuracy among the trained machine learning models 190B.

[0051] The testing engine 186 may be capable of testing the trained machine learning model 190B using a corresponding feature set of a test set from the dataset generator 172. For example, a first trained machine learning model 190B trained using a first feature set of the training set may be tested using a first feature set of the test set. The testing engine 186 may determine the trained machine learning model 190B that has the highest accuracy among all of the trained machine learning models based on the test set.

[0052] The machine learning model 190B may refer to a model artifact created by the training engine 182 using a training set that includes data inputs and corresponding target outputs (ground truth for each training input). Patterns can be found in the data set that map the data inputs to the target outputs (ground truth), and the machine learning model 190B is provided with mappings that capture these patterns. In some embodiments, the machine learning model 190B reconstructs calibration parameter values ​​associated with a physics-based model of the process chamber. The machine learning model 190B can use one or more of the following: support vector machine (SVM), radial basis function (RBF), clustering, supervised machine learning, semi-supervised machine learning, unsupervised machine learning, k-nearest neighbor algorithm (k-NN), linear regression, random forest, neural network (e.g., artificial neural network), and the like.

[0053] The calibration component 116 can provide model input data 154 to the model 190A and receive model output data 156 from the model 190A. The calibration component 116 can provide the sensor data 142 and the model data 152 to train the machine learning model 190B and adjust the calibration parameter data 162. The calibration component 116 can provide the model input data 154 and the calibration parameter data 162 to the model 190A and receive more accurate model output data 156 from the model 190A.

[0054] For purposes of illustration and not limitation, aspects of the disclosure describe training one or more machine learning models 190B using sensor data (e.g., sensor data 142) and model data (e.g., model data 152) to adjust calibration parameter data 162. In other implementations, a heuristic or rule-based model is used to adjust the calibration parameter data 162 (e.g., by specifying values ​​for parameters that control the sparsity of the fit, without using a trained machine learning model, etc.). Any of the information described with respect to data input 210 in FIG. 2 may be supervised or otherwise used in a heuristic or rule-based model.

[0055] In some embodiments, the functionality of client device 120, prediction server 112, server machine 170, and server machine 180 may be provided by a fewer number of machines. For example, in some embodiments, server machines 170 and 180 may be combined into a single machine, while in some other embodiments, server machine 170, server machine 180, and prediction server 112 may be combined into a single machine. In some embodiments, client device 120 and prediction server 112 may be combined into a single machine.

[0056] In general, functions described in one embodiment as being performed by client device 120, prediction server 112, server machine 170, and server machine 180 may also be performed on prediction server 112 in other embodiments, where appropriate. In addition, functions attributed to particular components may be performed by different components or multiple components operating together. For example, in some embodiments, prediction server 112 may determine corrective actions based on calibration parameter data 162. In another example, client device 120 may determine calibration parameter data 162 based on a trained machine learning model (e.g., model 190B).

[0057] In addition, the functionality of a particular component may be performed by different components or multiple components working together. One or more of prediction server 112, server machine 170, or server machine 180 may be accessed as a service offered to other systems or devices via an appropriate application programming interface (API).

[0058] In embodiments, a "user" may be represented as a single individual. However, other embodiments of the present disclosure encompass a "user" that is an entity controlled by multiple users and / or automated sources. For example, a set of individual users aggregated as a group of administrators may be considered a "user."

[0059] Embodiments of the present disclosure may be applied to data quality assessment, feature enhancement, model evaluation, virtual metrology (VM), predictive maintenance (PdM), marginal optimization, and the like.

[0060] Although embodiments of the present disclosure are described with respect to generating calibration parameter data 162 to perform corrective actions at a manufacturing facility (e.g., a semiconductor manufacturing facility), the embodiments may also be applied to improved data processing in general by utilizing virtual measurement and calibration parameter data 162 based on physical information.

[0061] FIG. 1B is a block diagram illustrating a data flow 100B for adjusting calibration parameter data 162 (e.g., to calibrate a physics-based model of a processing chamber) according to certain embodiments. In some embodiments, model 190A receives model input data 154. In some embodiments, model 190A is a physics-based model of a processing chamber. Model input data 154 may be input parameters of the physics-based model. In some embodiments, model input data 154 is for the entire operating space of the processing chamber. For example, model input data 154 may include a set of parameters covering a full range of processing chamber operation, such as a full range of spacing parameters, a full range of chamber pressure parameters, a full range of backside pressure parameters, a full range of heater temperature parameters, and / or a full range of gas flow parameters. Model input data 154 may include a substantial portion of a combination of input parameters associated with the processing chamber. In some embodiments, model input data 154 includes manufacturing parameters used in a substrate processing recipe.

[0062] In response to receiving the inputs, model 190A generates model output data 156. In some embodiments, model output data 156 is associated with the process chamber. For example, in some embodiments, model output data 156 includes one or more modeled temperature values ​​of one or more components of the process chamber based on model input data 154. In some embodiments, model output data 156 includes a set of modeled temperature values ​​for each set of model input data 154 (e.g., input parameters) input to model 190A.

[0063] Initially, the model output data 156 from the physics-based model 190A may be inaccurate. In some embodiments, the physics-based model 190A uses unadjusted (e.g., or unfurther adjusted) calibration parameter data 162 (e.g., one or more calibration parameter values) to generate the model output data 156. The accuracy of the physics-based model 190A is improved by adjusting the calibration parameter data 162 (e.g., one or more calibration parameters).

[0064] The model input data 154 and the subset 144A of the sensor data 142 are provided as training inputs, and the model output data 156 and the subset 144B of the sensor data 142 are provided as target outputs to the model 190B (e.g., to an untrained machine learning model, to a trained machine learning model to be further trained). The model 190B is trained to adjust the calibration parameter data 162 (e.g., to generate one or more adjusted calibration parameters). In some embodiments, the model input data 154 and the model output data 156 are provided to the untrained machine learning model by the calibration component 116 (see FIG. 1A). In addition, a first subset of the sensor data (e.g., the subset 144A of the sensor data 142) and a second subset of the sensor data (e.g., the subset 144B of the sensor data 142) are provided to the untrained machine learning model (e.g., by the calibration component 116).

[0065] In some embodiments, after the machine learning model 190B is trained, the calibration parameter data 162 (e.g., one or more adjusted calibration parameters) are retrieved from the trained machine learning model 190B. The calibration parameter data 162 (e.g., the adjusted calibration parameters) and the model input data 154 are provided as inputs to the physics-based model 190A. In some embodiments, the physics-based model 190A is updated based on the adjusted calibration parameters. The accuracy of the physics-based model 190A is improved based on the calibration parameter data 162. Providing the adjusted calibration parameters to the physics-based model (e.g., model 190A) enables more accurate modeling of the process chamber. In some embodiments, providing the adjusted calibration parameters to the physics-based model generates a more accurate digital twin of the process chamber.

[0066] 2 is a block diagram of an example dataset generator 272 (e.g., dataset generator 172 of FIG. 1A ) used to create a dataset for a machine learning model (e.g., model 190B of FIG. 1A ), according to certain embodiments. The dataset generator 272 may be part of the server machine 170 of FIG. 1A . In some embodiments, the system 100 of FIG. 1A includes multiple machine learning models. In such cases, each model may have a separate dataset generator, or the models may share a dataset generator.

[0067] 2, a system 200 including a dataset generator 272 (e.g., dataset generator 172 of FIG. 1A) creates a dataset for a machine learning model (e.g., model 190B of FIG. 1A). The dataset generator 272 can create the dataset using sensor data and model data of a physics-based model (e.g., model 190A of FIG. 1A). In some embodiments, the dataset generator 272 creates training inputs by selecting a subset of model input data and model output data of the physics-based model (e.g., a subset of model data 152 of FIG. 1A). For example, an output of the physics-based model can be a predicted temperature value of a component in a process chamber.

[0068] The dataset generator 272 can form a training set based on the sensor data and the model input data. In some embodiments, the data input 210 includes actual data (e.g., sensor data) and simulated data (e.g., representing sensor data that may be received from a sensor). The data input can include a subset 244A-L of the sensor data 242 and a set of model input data 254A-Z. The dataset generator 272 also generates a target output 220 for training the machine learning model. The target output includes a subset 244M-Z of the sensor data 242 and a set of model output data 256A-Z output from a physics-based model of the process chamber. In some implementations, the target output 220 includes a set of data outputs (e.g., model output data) from the physics-based model that indicate predictions of the physics-based model based on one or more inputs (e.g., model input data) to the model. The set of data output from the physics-based model can include predicted component temperature values ​​for each component of the process chamber of interest. The data input 210 and the target output 220 are provided to a machine learning model (eg, model 190B in FIG. 1A).

[0069] 2, in some embodiments, the dataset generator 272 generates a dataset (e.g., a training set, a validation set, a test set) that includes one or more data inputs 210 (e.g., training inputs, validation inputs, test inputs) and may include one or more target outputs 220 corresponding to the data inputs 210. The dataset may also include mapping data that maps the data inputs 210 to the target outputs 220. The data inputs 210 may also be referred to as "features," "attributes," or "information." In some embodiments, the dataset generator 272 may provide a dataset to the training engine 182, validation engine 184, or test engine 186 of FIG. 1A, where the dataset is used to train, validate, or test the machine learning model 190B of FIG. 1A. Some embodiments of generating a training set may be further described with respect to FIG. 4A.

[0070] In some embodiments, the dataset generator 272 may generate a first data input corresponding to a first subset 244A of the sensor data 242 and a first set of model input data 254A for training, validating, or testing a first machine learning model, and the dataset generator 272 may generate a second data input corresponding to a second subset 244B of the sensor data 242 and a second set of model input data 254B for training, validating, or testing a second machine learning model.

[0071] In some embodiments, the dataset generator 272 can perform operations on one or more of the data inputs 210 and the target outputs 220. The dataset generator 272 can extract patterns from the data (gradient, curvature, etc.), combine the data (average, feature generation, etc.), or separate simulated sensors into groups to train separate models.

[0072] The data input 210 and the target output 220 for training, validating, or testing a machine learning model can include information about a particular processing chamber (e.g., a particular substrate processing chamber). The data input 210 and the target output 220 can include information about a particular processing chamber design (e.g., to be used for all processing chambers of that design).

[0073] In some embodiments, the information used to train the machine learning model may be from a particular type of manufacturing equipment in a manufacturing facility having particular characteristics (e.g., manufacturing equipment 124 in FIG. 1A ), allowing the trained machine learning model to determine outcomes for a particular group of manufacturing equipment 124 based on sensor data and / or model data associated with one or more components that share the characteristics of the particular group. In some embodiments, the information used to train the machine learning model may be for components from more than one manufacturing facility, allowing the trained machine learning model to determine outcomes for components based on input from one manufacturing facility.

[0074] In some embodiments, after generating the dataset and using the dataset to train, validate, or test the machine learning model, the machine learning model may be further trained, validated, or tested, or tuned.

[0075] 3 is a block diagram illustrating a system 300 for generating calibration parameter data 362 (e.g., calibration parameter data 162 of FIG. 1A) according to certain embodiments. System 300 can be used to adjust calibration parameter data 362 and perform corrective actions using input from sensors and modeling techniques (e.g., physics-based modeling and / or machine learning modeling).

[0076] 3, in block 310, the system 300 (e.g., a component of the forecasting system 110 of FIG. 1A) performs data splitting of the historical data 360 (e.g., via the dataset generator 172 of the server machine 170 of FIG. 1A) to generate a training set 302, a validation set 304, and a test set 306. For example, the training set may be 60% of the simulated data, the validation set may be 20% of the simulated data, and the test set may be 20% of the simulated data. The historical data 360 may include sensor data 342 (e.g., sensor data 142 of FIG. 1A), model input data 354 (e.g., model input data 154 of FIG. 1A), and model output data 356 (e.g., model output data 156 of FIG. 1A).

[0077] In block 312, the system 300 performs model training (e.g., via the training engine 182 of FIG. 1A) using the training set 302. The system 300 can train multiple models using multiple feature sets of the training set 302 (e.g., a first feature set including a group of simulated sensors of the training set 302, a second feature set including a different group of simulated sensors of the training set 302, etc.). For example, the system 300 can train machine learning models to generate a first trained machine learning model using the first feature set of the training set and generate a second trained machine learning model using the second feature set of the training set (e.g., data different from the data used to train the first machine learning model). In some embodiments, the first trained machine learning model and the second trained machine learning model can be combined to generate a third trained machine learning model (e.g., which may be a better predictor than the first or second trained machine learning models alone). In some embodiments, the feature sets used to compare models may overlap (e.g., one model may be trained using simulated sensors 1-15 and a second model may be trained using simulated sensors 10-20). In some embodiments, hundreds of models may be generated, including models with various feature permutations and combinations of models.

[0078] In block 314, the system 300 performs model validation using the validation set 304 (e.g., via the validation engine 184 of FIG. 1A). The system 300 can validate each of the trained models using a corresponding feature set of the validation set 304. For example, the validation set 304 can use the same subset of simulated sensors used in the training set 302, but for different input conditions. In some embodiments, the system 300 can validate hundreds of models (e.g., models with various permutations of features, combinations of models, etc.) generated in block 312. In block 314, the system 300 can determine the accuracy of each of the one or more trained models (e.g., via model validation) and can determine whether one or more of the trained models have an accuracy that meets the threshold accuracy. In response to determining that none of the trained models have an accuracy that meets the threshold accuracy, flow returns to block 312 and the system 300 performs training of the models using a different feature set of the training set. In response to determining that one or more of the trained models have an accuracy that meets the threshold accuracy, flow proceeds to block 316. The system 300 may discard trained machine learning models that have an accuracy below the threshold accuracy (e.g., based on a validation set).

[0079] In block 316, the system 300 performs model selection (e.g., via the selection engine 185 of FIG. 1A ) to determine which of the one or more trained models that meet the threshold accuracy has the highest accuracy (e.g., the selected model 308 based on the validation of block 314). In response to determining that two or more of the trained models that meet the threshold accuracy have the same accuracy, flow can return to block 312, where the system 300 performs model training using a further refined training set corresponding to the further refined feature set to determine the trained model with the highest accuracy.

[0080] At block 318, the system 300 performs model testing (e.g., via the test engine 186 of FIG. 1A) using the test set 306 to test the selected model 308. The system 300 may test the first trained machine learning model using a first feature set of the test set (e.g., simulated sensors 1-15) to determine (e.g., based on the first feature set of the test set 306) that the first trained machine learning model meets a threshold accuracy. In response to the accuracy of the selected model 308 not meeting the threshold accuracy (e.g., the selected model 308 is overfitted to the training set 302 and / or the validation set 304 and is not applicable to other data sets, such as the test set 306), flow proceeds to block 312, where the system 30A performs training (e.g., retraining) of the model using a different training set, which may correspond to a different feature set or a reorganization of the substrates divided into the training set, validation set, and test set. In response to determining that the selected model 308 has an accuracy that meets the threshold accuracy based on the test set 306, flow proceeds to block 320. At least in block 312, the model may learn patterns in the simulated sensor data to make predictions, and in block 318, the system 300 may apply the model to the remaining data (e.g., the test set 306) to test the predictions.

[0081] In block 320, the system 300 determines calibration parameter data 362 from the trained model 320 (e.g., the selected model 308). Training the model 320 adjusts the calibration parameter data 362. The adjusted calibration parameter data 362 may be used to perform an action (e.g., perform corrective action in connection with the manufacturing equipment 124 of FIG. 1A, provide and alert the client device 120 of FIG. 1A, update a physics-based model, etc.).

[0082] The calibration parameter data 362 and the model input data 354 are provided to a physics-based model (e.g., model 190A of FIG. 1A ) to generate model output data 156. The current data 346 may include the sensor data 342, the model input data 354, and updated model output data 356 generated based on the model input data 354 and the adjusted calibration parameter data 362.

[0083] In some embodiments, retraining of the machine learning model is done by supplying additional data (e.g., current data 346) to further train the model. The current data 346 may be provided in block 312. The current data 346 may differ from the historical data 360 originally used to train the model by incorporating input parameters that were not part of the original training, input parameters outside the parameter space covered by the original training, or may be updated to reflect chamber specific knowledge (e.g., differences from an ideal chamber due to manufacturing tolerance ranges, aged components, etc.). The selected model 308 may be retrained based on this data. In some embodiments, the selected model 308 is retrained or further trained until the change in the calibration parameter data 362 responsive to the training of the selected model 308 meets a threshold change (e.g., changes less than a threshold amount).

[0084] In some embodiments, one or more of acts 310-320 may be performed in various orders and / or with other acts not shown and described herein. In some embodiments, one or more of acts 310-320 may not be performed. For example, in some embodiments, one or more of data partitioning of block 310, model validation of block 314, model selection of block 316, or model testing of block 318 may not be performed.

[0085] 4A-4C are flow diagrams of methods 400A-C associated with generating calibration parameter data for triggering corrective actions, according to certain embodiments. Methods 400A-C may be performed by processing logic, which may include hardware (e.g., circuits, dedicated logic, programmable logic, microcode, processing device, etc.), software (e.g., instructions executed on a processing device, general-purpose computer system, or dedicated machine), firmware, microcode, or combinations thereof. In some embodiments, methods 400A-C may be performed in part by prediction system 110 of FIG. 1A. Method 400A may be performed in part by prediction system 110 (e.g., server machine 170 and dataset generator 172 of FIG. 1A, dataset generator 272 of FIG. 2). Prediction system 110 may use method 400A to generate a dataset for at least one of training, validating, or testing a machine learning model, according to an embodiment of the present disclosure. Method 400B may be performed by server machine 180 (e.g., training engine 182, etc.). Method 400C may be performed by prediction server 112 (e.g., calibration component 116). In some embodiments, a non-transitory storage medium stores instructions that, when executed by a processing device (e.g., prediction system 110, server machine 180, prediction server 112, etc.), cause the processing device to perform one or more of methods 400A-C.

[0086] For ease of explanation, methods 400A-C are shown and described as a series of operations. However, operations in accordance with the present disclosure may occur in various orders and / or simultaneously, as well as with other operations not shown and described herein. Moreover, not all illustrated operations may be performed to implement methods 400A-C in accordance with the disclosed subject matter. In addition, those skilled in the art will understand and appreciate that methods 400A-C may alternatively be represented as a series of interrelated states via a state diagram or events.

[0087] FIG. 4A is a flow diagram of a method 400A for generating a dataset for a machine learning model for generating calibration parameter data (eg, calibration parameter data 162 of FIG. 1A) in accordance with certain embodiments.

[0088] Referring to FIG. 4A, in some embodiments, at block 401, processing logic performing method 400A initializes a training set T to an empty set.

[0089] At block 402, the processing logic generates a first data input (e.g., a first training input, a first validation input) that may include a first subset of sensor data (e.g., subset 144A of sensor data 142 of FIG. 1B) and model input data (e.g., set of model input data 154 of FIG. 1B). In some embodiments, the first data input may include a first set of features for a type of data and the second data input may include a second set of features for a type of data (e.g., as described with respect to FIG. 3).

[0090] At block 403, the processing logic generates a first target output for one or more of the data inputs (e.g., the first data input). In some embodiments, the first target output includes a second subset of the sensor data (e.g., subset 244B of sensor data 142 of FIG. 1B) and model output data (e.g., model output data 156 of FIG. 1B). The model output data may be an output of a physics-based model (e.g., model 190A of FIG. 1B) based on inputs of model input data 154.

[0091] At block 404, processing logic optionally generates mapping data indicative of an input-output mapping, which may refer to a data input (e.g., one or more of the data inputs described herein), a target output for the data input, and an association between the data input and the target output.

[0092] At block 405, processing logic, in some embodiments, adds the mapping data generated at block 404 to the dataset T.

[0093] At block 406, the processing logic branches based on whether the dataset T is sufficient for at least one of training, validation, and / or testing of the machine learning model 190 of FIG. 1A. If so, execution proceeds to block 407; if not, execution continues back to block 402. Note that while in some embodiments the sufficiency of the dataset T may be determined simply based on the number of inputs in the dataset that, in some embodiments, were mapped to outputs, in some other implementations the sufficiency of the dataset T may be determined based on one or more other criteria in addition to or instead of the number of inputs (e.g., a measure of diversity of the data examples, accuracy, etc.).

[0094] At block 407, the processing logic provides the dataset T (e.g., to the server machine 180 of FIG. 1A ) to train, validate, and / or test the machine learning model 190. In some embodiments, the dataset T is a training set and is provided to the training engine 182 of the server machine 180 to perform training. In some embodiments, the dataset T is a validation set and is provided to the validation engine 184 of the server machine 180 to perform validation. In some embodiments, the dataset T is a test set and is provided to the testing engine 186 of the server machine 180 to perform testing. After block 407, the machine learning model (e.g., the machine learning model 190B) can be at least one of trained using the training engine 182 of the server machine 180, validated using the validation engine 184 of the server machine 180, or tested using the testing engine 186 of the server machine 180. The trained machine learning model can be implemented by the calibration component 116 (of the prediction server 112) to adjust the calibration parameter data 162 to perform corrective actions associated with the manufacturing equipment 124.

[0095] FIG. 4B is a method 400B for training a machine learning model (eg, model 190B of FIG. 1A) to adjust calibration parameter data and perform corrective actions, according to certain embodiments.

[0096] 4B, in block 410 of method 400B, processing logic receives sensor data from one or more sensors of a processing chamber of a manufacturing tool (e.g., sensor 126 of manufacturing tool 124). The sensor data may include sensor values ​​associated with the processing chamber during a manufacturing process for a particular arrangement of manufacturing parameters (e.g., recipe parameters, process parameters, hardware settings, etc.). For example, the sensor data may be collected during an experimental run of a process operation in the processing chamber. The sensor data may include a first subset and a second subset, and the first subset is mapped to the second subset. In some embodiments, the first subset of the sensor data includes one or more of interval data, chamber pressure data, heater temperature data, or chamber flow data. In some embodiments, the second subset includes component temperature data associated with one or more components of the processing chamber.

[0097] At block 412, the processing logic identifies model input data and model output data, and the model output data is output from the physics-based model. In some embodiments, the model input data and the model output data (e.g., model input data 154 and model output data 156 of FIG. 1A) are associated with a physics-based model of the processing chamber (e.g., model 190A of FIG. 1A). In some embodiments, the model input data is a set of inputs to the physics-based model that represents the operating space of the processing chamber. For example, the model input data can substantially represent a range of input parameter values ​​(e.g., temperature, pressure, etc.) of the processing chamber. As another example, the model input data can represent a design of experiments (DOE) that corresponds to the physics-based model, where the variables of the DOE are one or more input parameters of the physics-based model. In some embodiments, the model output data is a set of outputs of the physics-based model, where each output corresponds to an input. For example, the model output data is a database of results of running a DOE that corresponds to the physics-based model. In some embodiments, the model output data is output from the physics-based model of the processing chamber. The model output data is mapped to the model input data. In some embodiments, the physics-based model first utilizes calibration parameters that have not yet been adjusted. Adjusting the calibration parameters improves the accuracy of the physics-based model. In some embodiments, the calibration parameters are one or more of a predicted thermal contact resistance value or a predicted thermal contact conductance value between corresponding components of the processing chamber.

[0098] At block 414, the processing logic trains the machine learning model. The machine learning model is trained using data inputs including a first subset of the sensor data and model input data. The machine learning model is trained using a second subset of the sensor data and target output data including model output data. Training the machine learning model is to adjust one or more calibration parameters (e.g., calibration parameter data 162 of FIG. 1A-1B) associated with the physics-based model. Prior to training the machine learning model, a range of values ​​can be assigned to the one or more calibration parameters. In some embodiments, adjusting the one or more calibration parameters includes adjusting corresponding values ​​of the one or more calibration parameters within a range of values. In some embodiments, a user (e.g., a human user) assigns a range of values ​​associated with the calibration parameters. In some embodiments, the processing device assigns a range of values ​​associated with the calibration parameters. In some embodiments, the calibration parameters are included in the calibration parameter data 162 of FIG. 1A. In some embodiments, the adjusted calibration parameters are determined based on an algorithm of the machine learning model. The adjusted calibration parameters are provided to the physics-based model. In some embodiments, the physics-based model is updated based on the adjusted calibration parameters. By updating the physics-based model based on the adjusted calibration parameters, the physics-based model becomes a more accurate representation of the processing chamber and can better predict the behavior of the processing chamber.

[0099] In block 416, the machine learning model may be retrained or further trained using the data inputs including the first subset of the sensor data and the updated model input data, and the target output data including the second subset of the sensor data and the updated model output data, to further adjust the calibration parameters. In some embodiments, the updated model input data and the updated model output data are associated with an updated physics-based model (e.g., an updated physics-based model based on the adjusted calibration parameters). By retraining the machine learning model, the calibration parameters can be further adjusted. In some embodiments, the further adjusted calibration parameters are provided to the physics-based model. The physics-based model is further updated based on the further adjusted calibration parameters. By further updating the physics-based model, the physics-based model becomes a more accurate representation of the processing chamber and can better predict the behavior of the processing chamber.

[0100] FIG. 4C is a method 400C for using calibration parameter data conditioned via a trained machine learning model (eg, model 190 of FIG. 1A) according to certain embodiments.

[0101] 4C, at block 420 of method 400C, processing logic identifies adjusted calibration parameters by training a machine learning model (e.g., trained via block 414 of FIG. 4B). The adjusted calibration parameters and model input data are provided to a physics-based model to output updated model output data (e.g., the physics-based model is updated based on the adjusted calibration parameters). In some embodiments, the machine learning model is trained according to the method described in connection with FIG. 4B.

[0102] At block 422, the processing logic identifies second model input data. In some embodiments, the second model input data represents a set of input parameters (e.g., one or more of a spacing parameter, a chamber pressure parameter, a backside pressure parameter, a heater temperature parameter, a gas flow parameter, etc.) to be input to the physics-based model. In some embodiments, the second model input data is input to the updated physics-based model to calculate one or more outputs. In some embodiments, the second model input data is provided by a user (e.g., a human user). In some embodiments, the second model input data is provided to the physics-based model after the physics-based model is updated with the adjusted calibration parameters.

[0103] At block 424, the processing logic receives second model output data from the physics-based model. The receipt of the second model output data is in response to providing the second model input data and the adjusted calibration parameters as inputs to the physics-based model. In some embodiments, the physics-based model is updated based on the adjusted calibration parameters in response to providing the calibration parameters. In some embodiments, the second model output data is generated by the physics-based model after the physics-based model is updated based on the adjusted calibration parameters.

[0104] At block 426, processing logic triggers one or more corrective actions based on the second model output data. In some embodiments, the one or more corrective actions are associated with the process chamber. In some embodiments, the corrective actions include providing an alert, interrupting a manufacturing device, updating manufacturing parameters, etc.

[0105] FIG. 5 is a block diagram illustrating a computer system 500, according to certain embodiments. In some embodiments, the computer system 500 may be connected to other computer systems (e.g., via a network such as a local area network (LAN), an intranet, an extranet, or the Internet). The computer system 500 may operate in the capacity of a server or a client computer in a client-server environment, or as a peer computer in a peer-to-peer or distributed network environment. The computer system 500 may be provided by a personal computer (PC), a tablet PC, a set-top box (STB), a personal digital assistant (PDA), a mobile phone, a web appliance, a server, a network router, a switch or bridge, or any device capable of executing a set of instructions (sequential or otherwise) that specify actions to be taken by the device. Furthermore, the term "computer" is intended to include any collection of computers that individually or jointly execute a set (or sets) of instructions to perform any one or more of the methods described herein. The computer system 500 may include one or more of the client device 120, the prediction server 112, the server machine 170, the server machine 180, and the like.

[0106] In a further aspect, the computer system 500 may include a processing device 502, a volatile memory 504 (e.g., random access memory (RAM)), a non-volatile memory 506 (e.g., read only memory (ROM) or electrically erasable programmable ROM (EEPROM)), and a data storage device 518, which may communicate with each other via a bus 508.

[0107] The processing device 502 may be provided by one or more processors, such as a general-purpose processor (e.g., a Complex Instruction Set Computing (CISC) microprocessor, a Reduced Instruction Set Computing (RISC) microprocessor, a Very Long Instruction Word (VLIW) microprocessor, a microprocessor implementing other types of instruction sets, or a combination of multiple types of instruction sets, etc.) or a special-purpose processor (e.g., an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), a Digital Signal Processor (DSP), or a network processor, etc.).

[0108] Computer system 500 may further include a network interface device 522 (e.g., coupled to a network 574). Computer system 500 may also include a video display unit 510 (e.g., LCD), an alphanumeric input device 512 (e.g., keyboard), a cursor control device 514 (e.g., mouse), and a signal generating device 520.

[0109] In some implementations, the data storage device 518 may include a non-transitory computer-readable storage medium 524 (e.g., a non-transitory machine-readable storage medium) (e.g., a non-transitory machine-readable storage medium that stores instructions) that may store instructions 526 encoding any one or more of the methods or functions described herein, including instructions for encoding the components of FIG. 1A (e.g., calibration component 116, models 190A-B, etc.) and implementing the methods described herein.

[0110] The instructions 526 may also reside, completely or partially, within the volatile memory 504 and / or within the processing device 502 during execution thereof by the computer system 500, and thus the volatile memory 504 and the processing device 502 may also constitute machine-readable storage media.

[0111] Although the computer-readable storage medium 524 is shown as a single medium in the illustrative example, the term "computer-readable storage medium" is intended to include a single medium or multiple media (e.g., a centralized or distributed database and / or associated caches and servers) that store one or more sets of executable instructions. The term "computer-readable storage medium" is also intended to include any tangible medium capable of storing or encoding a set of instructions for execution by a computer that causes the computer to perform any one or more of the methods described herein. The term "computer-readable storage medium" is intended to include, but is not limited to, solid-state memory, optical media, and magnetic media.

[0112] The methods, components, and features described herein may be implemented by discrete hardware components or integrated into the functionality of other hardware components, such as ASICs, FPGAs, DSPs, or similar devices. Additionally, the methods, components, and features may be implemented by firmware modules or functional circuits within a hardware device. Additionally, the methods, components, and features may be implemented in any combination of hardware devices and computer program components, or in a computer program.

[0113] Unless otherwise specified, terms such as "receive," "perform," "provide," "obtain," "cause," "access," "determine," "add," "use," "identify," "train," "interrupt," "update," and the like refer to actions and processes performed or implemented by a computer system that manipulate and transform data represented as physical (electronic) quantities in computer system registers and memory into other data similarly represented as physical quantities in computer system memory or registers or other such information storage, transmission or display devices. Also, terms such as "first," "second," "third," "fourth," and the like, as used herein, are intended as labels to distinguish different elements and may not have a meaning that denotes an ordering by their numerical designation.

[0114] The examples described herein also relate to an apparatus for carrying out the methods described herein. This apparatus may be specially constructed to carry out the methods described herein, or may comprise a general-purpose computer system selectively programmed by a computer program stored in the computer system. Such a computer program may be stored in a computer-readable tangible storage medium.

[0115] The methods and illustrative embodiments described herein are not inherently related to any particular computer or other apparatus. Various general-purpose systems may be used in accordance with the teachings described herein, or it may prove convenient to construct more specialized apparatus to perform the methods described herein and / or each of their individual functions, routines, subroutines, or operations. Examples of structures for a variety of these systems are set forth in the description above.

[0116] The above description is intended to be illustrative and not limiting. Although the present disclosure has been described with reference to certain illustrative examples and implementations, it will be recognized that the present disclosure is not limited to the described examples and implementations. The scope of the present disclosure should be determined with reference to the following claims, along with the full scope of equivalents to which such claims are entitled.

Claims

1. receiving sensor data from a plurality of sensors associated with processing of a substrate through a processing chamber of a substrate processing apparatus, the sensor data including a first subset received from one or more first sensors and a second subset received from one or more second sensors, the first subset being mapped to the second subset; identifying model input data and model output data, the model output data being output from a physics-based model based on the model input data; training a machine learning model using data inputs comprising the first subset and the model input data and target output data comprising the second subset and the model output data to adjust one or more calibration parameters of the machine learning model, the one or more calibration parameters being used by the physics-based model to perform one or more corrective actions associated with the process chamber; A method comprising:

2. the first subset of sensor data includes one or more of interval data, chamber pressure data, heater temperature data, or chamber flow data; the second subset of sensor data includes component temperature data associated with one or more components of the processing chamber; The method of claim 1.

3. 2. The method of claim 1 , wherein the first subset of sensor data and the model input data correspond to one or more first types of data, and the second subset of sensor data and the model output data correspond to one or more second types of data that are different from the one or more first types of data.

4. The method of claim 1 , wherein the calibration parameters include one or more of a predicted thermal contact resistance value or a predicted thermal contact conductance value between corresponding components of the processing chamber.

5. 10. The method of claim 1, wherein the physics-based model comprises a digital twin model used to update process parameters of the process chamber.

6. 2. The method of claim 1 , wherein in response to the one or more calibration parameters being adjusted, the model input data and the one or more calibration parameters are input into the physics-based model to generate updated model output data, and the updated model output data is used to perform the one or more corrective actions.

7. The one or more corrective actions: Providing warnings; interrupting operation of the processing chamber; or updating manufacturing parameters of the processing chamber; The method of claim 1 , comprising one or more of:

8. 2. The method of claim 1 , further comprising: prior to training the machine learning model, assigning a range of values ​​to the one or more calibration parameters, and wherein adjusting the one or more calibration parameters comprises adjusting a corresponding value of the one or more calibration parameters within the range of values.

9. identifying one or more calibration parameters tuned by training a machine learning model, the machine learning model being trained with data inputs comprising a first subset of sensor data and first model input data, and target output data comprising a second subset of the sensor data and first model output data, the sensor data being received from a plurality of sensors, the sensor data being associated with processing of a substrate through a processing chamber of a substrate processing apparatus, and the first model output data being output from a physics-based model based on the first model input data; Identifying second model input data; receiving second model output data from the physics-based model in response to providing the second model input data and the calibration parameters as inputs to the physics-based model, where one or more corrective actions associated with the process chamber are performed based on the second model output data; and A method comprising:

10. the first subset of sensor data includes one or more of interval data, chamber pressure data, heater temperature data, or chamber flow data; the second subset of sensor data includes component temperature data associated with one or more components of the processing chamber; 10. The method of claim 9.

11. 10. The method of claim 9, wherein the first subset of sensor data and the first model input data correspond to one or more first types of data, and the second subset of sensor data and the first model output data correspond to one or more second types of data different from the one or more first types of data.

12. 10. The method of claim 9, wherein the calibration parameters include one or more predicted thermal contact resistance or conductance values ​​between corresponding components of the processing chamber.

13. 10. The method of claim 9, wherein the physics-based model comprises a digital twin model used to update process parameters of the process chamber.

14. 10. The method of claim 9, wherein in response to the one or more calibration parameters being adjusted, the first model input data and the one or more calibration parameters are input into the physics-based model to generate updated model output data, and the one or more corrective actions are performed based on the updated model output data.

15. The one or more corrective actions Providing warnings; interrupting operation of the processing chamber; or updating manufacturing parameters of the processing chamber; 10. The method of claim 9, comprising one or more of:

16. A non-transitory machine-readable storage medium storing instructions that, when executed, cause a processing device to: receiving sensor data from a plurality of sensors associated with processing of a substrate through a processing chamber of a substrate processing apparatus, the sensor data including a first subset received from one or more first sensors and a second subset received from one or more second sensors, the first subset being mapped to the second subset; identifying model input data and model output data, the model output data being output from a physics-based model based on the model input data; training a machine learning model using data inputs comprising the first subset and the model input data and target output data comprising the second subset and the model output data to adjust one or more calibration parameters of the machine learning model, the one or more calibration parameters being used by the physics-based model to perform one or more corrective actions associated with the process chamber; A non-transitory machine-readable storage medium that causes operations including

17. 17. The non-transitory machine-readable storage medium of claim 16, wherein the first subset of sensor data and the model input data correspond to one or more first types of data, and the second subset of sensor data and the model output data correspond to one or more second types of data that are different from the one or more first types of data.

18. 20. The non-transitory machine-readable storage medium of claim 16, wherein the physics-based model comprises a digital twin model used to update processing parameters of the processing chamber.

19. 17. The non-transitory machine-readable storage medium of claim 16, wherein in response to the one or more calibration parameters being adjusted, the model input data and the one or more calibration parameters are input to the physics-based model to generate updated model output data, and the one or more corrective actions are performed based on the updated model output data.

20. 20. The non-transitory machine-readable storage medium of claim 16, wherein the processing device further assigns a range of values ​​to the one or more calibration parameters prior to training the machine learning model, and wherein adjusting the one or more calibration parameters comprises adjusting a corresponding value of the one or more calibration parameters within the range of values.

Images