Processing Chamber Calibration

Abstract

The method includes receiving sensor data from a sensor associated with processing a substrate through a processing chamber of a substrate processing apparatus. The sensor data includes a first subset received from one or more first sensors and a second subset received from one or more second sensors, where the first subset is mapped to the second subset. The method further includes identifying model input data and model output data. The model output data is output from the physics-based model based on the model input data. The method further includes training the machine learning model with data inputs including the first subset and the model input data and target output data including the second subset and the model output data to adjust calibration parameters of the machine learning model. The calibration parameters are used by the physics-based model to perform corrective actions associated with the processing chamber.

Description

[Technical Field]

[0001] FIELD OF THE DISCLOSURE The present disclosure relates to calibration, and more particularly to calibration of processing chambers. [Background technology]

[0002] The manufacturing equipment produces a product, for example, a substrate processing equipment produces substrates. The sensors are used to provide sensor data associated with the manufacturing equipment. Parameters of the manufacturing equipment are selected based on the sensor data. Summary of the Invention

[0003] The following is a simplified summary of the disclosure to provide a basic understanding of some aspects of the disclosure. This summary is not an extensive overview of the disclosure. It is not intended to identify key or critical elements of the disclosure, nor is it intended to delineate the scope or claims of particular implementations of the disclosure. Its sole purpose is to present some concepts of the disclosure in a simplified form as a prelude to the more detailed description that is presented later.

[0004] In one aspect of the present disclosure, a method includes receiving sensor data associated with processing a substrate through a processing chamber of a substrate processing apparatus from a plurality of sensors. The sensor data includes a first subset received from one or more first sensors and a second subset received from one or more second sensors, where the first subset is mapped to the second subset. The method further includes identifying model input data and model output data. The model output data is output from a physics-based model based on the model input data. The method further includes training a machine learning model using data inputs including the first subset and the model input data and target output data including the second subset and the model output data to adjust one or more calibration parameters of the machine learning model. The one or more calibration parameters are used by the physics-based model to perform one or more corrective actions associated with the processing chamber.

[0005] In another aspect of the present disclosure, a method includes identifying one or more adjusted calibration parameters by training a machine learning model. The machine learning model is trained using data inputs including a first subset of sensor data and first model input data, and target output data including a second subset of sensor data and first model output data. The sensor data is received from a plurality of sensors. The sensor data is associated with processing of a substrate through a processing chamber of a substrate processing apparatus, and the first model output data is output from a physics-based model based on the first model input data. The method further includes identifying second model input data. The method further includes receiving second model output data from the physics-based model in response to providing the second model input data and the calibration parameters as inputs to the physics-based model. One or more corrective actions associated with the processing chamber are performed based on the second model output data.

[0006] In another aspect of the present disclosure, a non-transitory machine-readable storage medium stores instructions that, when executed, cause a processing device to perform operations including receiving sensor data associated with processing a substrate through a processing chamber of a substrate processing apparatus from a plurality of sensors. The sensor data includes a first subset received from one or more first sensors and a second subset received from one or more second sensors, where the first subset is mapped to the second subset. The processing device further identifies model input data and model output data. The model output data is output from the physics-based model based on the model input data. The processing device further trains the machine learning model using data inputs including the first subset and the model input data and target output data including the second subset and the model output data to adjust one or more calibration parameters of the machine learning model. The one or more calibration parameters are used by the physics-based model to perform one or more corrective actions associated with the processing chamber.

[0007] The present disclosure is illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings. [Brief explanation of the drawings]

[0008] [Figure 1A] FIG. 1 is a block diagram illustrating an example system (example system architecture) according to certain embodiments. [Figure 1B] FIG. 2 is a block diagram illustrating data flow associated with adjusting calibration parameter data, according to certain embodiments. [Figure 2] FIG. 1 is a block diagram of an exemplary dataset generator used to create a dataset for a machine learning model, according to certain embodiments. [Figure 3] FIG. 1 is a block diagram illustrating a system for generating calibration parameter data, in accordance with certain embodiments. [Figure 4A] 1 is a flow diagram of a method associated with generating calibration parameter data for triggering corrective action, according to certain embodiments. [Figure 4B] 1 is a flow diagram of a method associated with generating calibration parameter data for triggering corrective action, according to certain embodiments. [Figure 4C] 1 is a flow diagram of a method associated with generating calibration parameter data for triggering corrective action, according to certain embodiments. [Figure 5] FIG. 1 is a block diagram illustrating a computer system, in accordance with certain embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0009] Described herein are techniques directed to process chamber calibration (e.g., methods for calibrating a process chamber digital twin).

[0010] Manufacturing equipment is used to produce products. For example, substrate processing equipment produces substrates (e.g., semiconductors, wafers, etc.). The substrate processing equipment has manufacturing parameters, such as hardware parameters and / or process parameters, used during substrate processing. For example, manufacturing parameters of a particular temperature and pressure in a processing chamber may be used during substrate processing. Processed (or partially processed) substrates have characteristic data (e.g., thickness data, roughness data). The manufacturing parameters of the substrate processing equipment are adjusted to produce substrates (e.g., good substrates) having characteristic data that match threshold characteristic data.

[0011] Traditionally, models are created to attempt to mirror the processing chamber. The models are manually calibrated to attempt to determine manufacturing parameters to produce substrates with characteristic data that match the threshold characteristic data. Manual calibration requires a lot of time, trial and error, and is wasteful of materials. Given the many different manufacturing parameters, the many combinations of different values ​​of the manufacturing parameters, and time and material limitations, non-ideal manufacturing parameters are often determined through manual calibration. The use of non-ideal manufacturing parameters results in defective substrates, reduced yield, and inefficient processing.

[0012] The model can be created based on an estimate of the relationship between a first manufacturing parameter and a second manufacturing parameter. For example, the model can be created based on an estimate between temperature and a resulting estimate of pressure. Sensor data from sensors associated with the processing chamber can then be used to update the estimate of the relationship between the manufacturing parameters. For example, measured temperature data from a temperature sensor and measured pressure data from a pressure sensor can be used to update the estimate of the relationship between the temperature data and the resulting pressure data. A manufacturing parameter can result from many other manufacturing parameters, some of which may not be measured. Traditional models updated based on sensor data are not fully updated because there are unmeasured manufacturing parameters that affect the manufacturing parameters of the model. Non-ideal models can lead to the use of non-ideal manufacturing parameters, resulting in defective substrates, reduced yields, and inefficient processing.

[0013] Traditionally, models are created and updated offline (when the processing chamber is not in use), and then the model is used throughout the life of the processing chamber. As processing chambers are used repeatedly, the relationships between different manufacturing parameters change (e.g., drift) due to material accumulation, aging of parts, changes during cleaning procedures, etc. For example, the relationship between temperature and pressure in a processing chamber drifts over time. Due to drift, traditional models become increasingly inaccurate over time. As models become increasingly inefficient, more substrates are defective, further reducing yields and making the process more inefficient.

[0014] Traditionally, the same model may be used for different processing chambers. Because processing chambers are manufactured to tolerances, each processing chamber operates slightly differently. Different processing chambers age differently, resulting in different types of drift for each processing chamber. Traditional models used for many different processing chambers result in different levels of precision for each processing chamber, leading to inconsistencies in the substrates produced by each processing chamber, lower yields, and reduced efficiency.

[0015] The disclosed methods, devices, and systems provide solutions to the deficiencies of conventional systems. The disclosed methods provide for calibration of a process chamber (e.g., calibration of a process chamber digital twin).

[0016] The sensor data is received from a sensor. The sensor may be associated with the processing chamber (e.g., disposed within the processing chamber and providing sensor data for the processing chamber, etc.). For example, the sensor may provide temperature data, pressure data, flow data, etc. during operation of the processing chamber (e.g., processing of a substrate).

[0017] The sensor data includes different subsets of sensor data received from different sensors. A first subset of sensor data is received from one or more first sensors, and a second subset of sensor data is received from one or more second sensors, and the first subset is mapped to the second subset. In some embodiments, the first subset includes chamber pressure data, backside pressure data, heater temperature data, and gas flow data. Further, the first subset can include chamber component spacing data. The second subset can include chamber component temperature data.

[0018] Model input data and model output data are identified. The model input data is provided to a physics-based model, which generates (e.g., predicts) model output data based on the input model input data. In some embodiments, the model output data includes component temperature data associated with one or more components of the processing chamber.

[0019] The first subset of sensor data and the model input data may correspond to one or more first types of data (e.g., the same first type of data). The second subset of sensor data and the model output data may correspond to one or more second types of data different from the first type of data (e.g., the same type of data). For example, the first subset of sensor data and the model input data may both include spacing, chamber pressure, backside pressure, heater temperature, and / or gas flow rate. The second subset of sensor data and the model output data may both include a temperature of the component.

[0020] The machine learning model can be trained using data inputs including a first subset of the sensor data and model input data, and target output data including a second subset of the sensor data and model output data to adjust calibration parameters of the machine learning model. The calibration parameters may include unmeasured or unmeasurable parameters, such as thermal contact resistance and thermal contact conductance. After the machine learning model adjusts the calibration parameters, the calibration parameters can be used by the physics-based model to perform corrective actions associated with the process chamber and generate more accurate predictors.

[0021] The model input data and the adjusted calibration parameters can be input into the physics-based model to generate updated model output data. Providing the adjusted calibration parameters to the physics-based model can calibrate the physics-based model to provide a more accurate digital twin of the processing chamber. In some embodiments, the physics-based model is a digital twin model of the processing chamber. The updated model output data is more accurate than the initial model output data generated by the physics-based model using the initial predictions of the calibration parameters (i.e., the unadjusted calibration parameters). The updated model output data can be used to perform corrective actions associated with the processing chamber. For example, the corrective actions can include one or more of providing a warning, interrupting operation of the processing chamber, or updating manufacturing parameters of the processing chamber.

[0022] Aspects of the present disclosure provide technical advantages over conventional systems. Adjusting the calibration parameters of a physics-based model with the help of a machine learning model can save time compared to conventional manual calibration of the model. Using aspects of the present disclosure, manual calibration may no longer be necessary. Calibration parameters that may have been difficult, impractical, or impossible to calculate manually using conventional methods can be more easily and accurately predicted using the methods of the present disclosure, resulting in more accurate and precise models compared to models from conventional methods. Furthermore, using the present disclosure, models can be created and updated online (while the processing chamber is in use), as opposed to conventional models that are created and updated offline. The present disclosure also accounts for process chamber drift, resulting in more accurate models that increase consistency between chambers and therefore reduce defective substrates and increase yield, unlike traditional approaches. Additionally, the methods of the present disclosure can iterate calibration parameters, generating increasingly accurate physics-based models over time compared to conventional methods. Furthermore, the methods of the present disclosure may be able to correct physics-based models for drift in chamber characteristics and for differences between individual chambers. Thus, the physics-based model of the present disclosure provides greater confidence in taking corrective actions on the processing chamber, leading to more consistent and precise substrate manufacturing compared to conventional methods.

[0023] 1A is a block diagram illustrating an example system 100A (an example system architecture), according to some embodiments. System 100 includes client devices 120, manufacturing equipment 124, sensors 126, a prediction server 112, and a data store 140. Prediction server 112 may be part of a prediction system 110. Prediction system 110 may further include server machines 170 and 180.

[0024] The sensors 126 can provide sensor data 142 associated with the manufacturing equipment 124 (e.g., associated with the manufacturing equipment 124 producing a corresponding product, such as a wafer, a substrate, a semiconductor, and / or a display). The sensor data 142 can be used, for example, for equipment health and / or product health (e.g., product quality). The manufacturing equipment 124 can run to produce a product according to a recipe or over a period of time. In some embodiments, the sensor data 142 can include one or more values ​​of temperature (e.g., heater temperature, component temperature), spacing (SP), pressure (e.g., chamber pressure, backside pressure), high frequency radio frequency (HFRF), electrostatic chuck (ESC) voltage, current, flow rate (e.g., gas flow rate, flow rate of one or more gases), power, voltage, etc. The sensor data 142 may include one or more subsets 144 of sensor data 142 (e.g., a first subset 144 of sensor data 142 from a first subset of sensors 126 and a second subset 144 of sensor data 142 from a second subset of sensors 126). The sensor data 142 may be provided while the manufacturing equipment 124 is performing a manufacturing process (e.g., equipment readings as the product is processed). The sensor data 142 may vary from product to product (e.g., from substrate to substrate). In some embodiments, the sensor data 142 is obtained from an experimental run of a process operation in a processing chamber.

[0025] In some embodiments, the sensor data 142 may be processed (e.g., by the client device 120 and / or by the prediction server 112). Processing the sensor data 142 may include generating features. In some embodiments, the features are patterns in the sensor data 142 (e.g., slope, width, height, peak, etc.) or combinations of values ​​from the sensor data 142 (e.g., power derived from voltage and current, etc.). The sensor data 142 may include features that may be used by the prediction system 110 to perform signal processing and / or to obtain model input data 154, model output data 156, and / or calibration parameter data 162 for performing corrective actions.

[0026] Each instance (e.g., set) of sensor data 142 may correspond to a product (e.g., a substrate), a set of manufacturing equipment, a type of substrate produced by the manufacturing equipment, a combination thereof, etc. Data store 140 may further store information relating sets of different data types, e.g., information indicating that the sets of sensor data 142 and / or sets of model data are all associated with the same product, manufacturing equipment, type of substrate, etc.

[0027] In some embodiments, the predictive system 110 can generate the calibration parameter data 162 using supervised machine learning (e.g., a model 190B trained based on inputs including a first subset 144 of sensor data 142 and model input data 154, and a target output including model output data 156 and a second subset 144 of sensor data 142, to adjust the calibration parameter data 162). In some embodiments, the calibration parameter data 162 is used to calibrate a physics-based model (e.g., model 190A) of a processing chamber of the manufacturing tool 124. In some embodiments, the calibrated physics-based model is a digital twin model of the processing chamber of the manufacturing tool 124, which is used to perform corrective actions (e.g., update manufacturing parameters of the processing chamber). As used herein, a digital twin is a digital replica of a physical asset, such as a manufactured part (e.g., a manufacturing tool 124, a processing chamber, a substrate processing system, etc.). The digital twin includes properties of the physical asset at each stage of the manufacturing process, including, but not limited to, coordinate dimensions, weight properties, material properties (e.g., density, surface roughness), electrical properties (e.g., conductivity), optical properties (e.g., reflectivity), manufacturing parameters (e.g., hardware parameters, process parameters), etc.

[0028] Client device 120, manufacturing equipment 124, sensors 126, prediction server 112, data store 140, server machine 170, and server machine 180 may be coupled to each other via network 130 to generate model output data 156 and / or calibration parameter data 162, and optionally to perform corrective actions.

[0029] In some embodiments, network 130 is a public network that provides client device 120 with access to prediction server 112, data store 140, and / or other publicly available computing devices. In some embodiments, network 130 is a private network that provides client device 120 with access to manufacturing equipment 124, sensors 126, data store 140, and / or other privately available computing devices. In some embodiments, network 130 is a cloud-based network capable of performing cloud-based functions (e.g., providing cloud service functionality to one or more devices in the system). Network 130 may include one or more wide area networks (WANs), local area networks (LANs), wired networks (e.g., Ethernet networks), wireless networks (e.g., 802.11 networks or Wi-Fi networks), cellular networks (e.g., Long Term Evolution (LTE) networks), routers, hubs, switches, server computers, cloud computing networks, and / or combinations thereof.

[0030] The client device 120 may include computing devices such as a personal computer (PC), a laptop, a mobile phone, a smartphone, a tablet computer, a netbook computer, a network-connected television ("smart TV"), a network-connected media player (e.g., a Blu-ray player), a set-top box, an over-the-top (OTT) streaming device, an operator box, a cloud server, a cloud-based system (e.g., a cloud service device, a cloud network device), etc. The client device 120 may include a corrective action component 122. The corrective action component 122 may receive user input of instructions associated with the manufacturing equipment 124 (e.g., via a graphical user interface (GUI) displayed via the client device 120). In some embodiments, the corrective action component 122 sends instructions to the prediction system 110, receives output (e.g., model output data 156) from the prediction system 110, determines corrective actions based on the output, and causes the corrective actions to be implemented.

[0031] In some embodiments, the prediction system 110 may further include a calibration component 116. The calibration component 116 may use the sensor data 142 and model data 152 (e.g., associated with the physics-based model 190A) to train the machine learning model 190B. In some embodiments, the calibration component 116 provides model input data 154 to the physics-based model 190A to generate model output data 156. The calibration component 116 may receive the model data 152 associated with the physics-based model 190A and input at least a portion of the model data 152 into the machine learning model 190B. The calibration component 116 may receive calibration parameter data 162 from the model 190B. In some embodiments, the calibration component 116 provides the calibration parameter data 162 and the model input data 154 to the model 190A and receives the model output data 156 from the model 190A. The calibration component 116 provides the model output data 156 to the client device 120, which, in consideration of the model output data 156, triggers a corrective action via the corrective action component 122. In some embodiments, the corrective action component 122 obtains (e.g., from the data store 140, etc.) sensor data 142 associated with the manufacturing equipment 124 (e.g., a subset 144 of the sensor data 142) and provides the sensor data 142 associated with the manufacturing equipment 124 (e.g., a subset 144 of the sensor data 142) to the predictive system 110.

[0032] In some embodiments, the corrective action component 122 stores the sensor data 142 in the data store 140, and the prediction server 112 retrieves the sensor data 142 from the data store 140. In some embodiments, the prediction server 112 can store the output of the trained machine learning model 190B (e.g., model output data 156) in the data store 140, and the client device 120 can retrieve the output from the data store 140. In some embodiments, the corrective action component 122 receives corrective action instructions from the prediction system 110 and causes the corrective action to be implemented. In some embodiments, the corrective action includes updating manufacturing parameters (e.g., hardware parameters, process parameters) of the manufacturing equipment 124 based on the model output data 156. The client device 120 can include an operating system that enables a user to one or more of create, view, or edit data (e.g., instructions associated with the manufacturing equipment 124, corrective actions associated with the manufacturing equipment 124, etc.).

[0033] In some embodiments, calibration parameter data 162 is associated with one or more values ​​of manufacturing equipment 124 that are not measured or cannot be measured (e.g., calibration parameters of manufacturing equipment 124). In some embodiments, calibration parameters (e.g., calibration parameter data 162) include one or more of thermal contact conductance or thermal contact resistance of junctions (e.g., where components contact) of individual components of manufacturing equipment 124. Interfaces and / or junctions between two solid objects of manufacturing equipment 124 generate thermal contact resistances (and thermal contact conductances) for heat flow between the two solid objects. In some embodiments, calibration parameter data 162 is predicted calibration parameter values ​​(e.g., predicted thermal contact resistances or predicted thermal contact conductances) of one or more components of manufacturing equipment 124 based on sensor data 142 (e.g., current state of manufacturing equipment 124) and / or model data 152 (e.g., predicted state of manufacturing equipment 124). In some embodiments, thermal contact resistance or thermal contact conductance exists between one or more components of the processing chamber. For example, each pair of components in contact with each other has an associated thermal contact conductance (e.g., the time rate of steady-state heat flow between the components induced by a unit temperature difference) or thermal contact resistance (e.g., the ratio of the temperature drop across the interface to the average heat flow). Some such thermal relationships may include the thermal contact conductance or thermal contact resistance of a contact point (e.g., an interface) between the showerhead and the processing chamber lid, between the heater and the processing chamber shaft, between the blocker plate and the processing chamber lid, between the blocker plate and the processing chamber sidewall, etc. In some embodiments, the calibration parameter data 162 includes an indication of change or drift over time in any component of the manufacturing tool 124, the sensor 126, etc. In some embodiments, the calibration parameter data 162 reflects the process conditions of the manufacturing tool 124 (e.g., dependent on the temperature, pressure, etc. of the substrate processing operation). In some embodiments, the calibration parameter data 162 is used to calibrate a physics-based model of the processing chamber (eg, model 190A).

[0034] Running a manufacturing process that results in a defective product can be costly in terms of time, energy, product, components, manufacturing equipment 124, the cost of identifying and discarding the defective product, etc. By inputting sensor data 142 (e.g., manufacturing parameters used to manufacture the product) and model data 152, adjusting calibration parameter data 162, and performing corrective actions (e.g., calibrating a physics-based model of the processing chamber) based on the calibration parameter data 162, system 100 can have the technical advantage of avoiding the costs of producing, identifying, and discarding defective product by utilizing an accurate and calibrated physics-based model of the processing chamber.

[0035] The manufacturing parameters may not be optimal for producing the product, which may result in costly consequences such as increased consumption of resources (e.g., energy, coolant, gas, etc.), increased time to produce the product, increased component failures, increased amount of defective products, etc. By inputting the sensor data 142 and the model data 152 into the trained machine learning model 190B, adjusting the calibration parameter data 162, and taking corrective action (e.g., based on the calibration parameter data 162) to update the manufacturing parameters (e.g., calibrating the physics-based model 190A of the processing chamber to update the model data 152), the system 100 may have the technical advantage of avoiding the costly consequences of sub-optimal manufacturing parameters by utilizing an accurate, calibrated physics-based model of the processing chamber using optimal manufacturing parameters (e.g., hardware parameters, process parameters, optimal design).

[0036] The corrective action may be associated with one or more of computational process control (CPC), statistical process control (SPC) (e.g., SPC of electronic components to determine the process under control, SPC to predict the useful life of a component, SPC for comparison with 3 sigma graphs, etc.), advanced process control (APC), model-based process control, preventative operational maintenance, design optimization, manufacturing parameter updates, manufacturing recipe updates, feedback control, machine learning corrections, model calibration (e.g., physics-based model calibration), etc.

[0037] In some embodiments, the corrective action includes providing a warning (e.g., an alarm to stop or not run the manufacturing process). In some embodiments, the corrective action includes providing feedback control (e.g., modifying manufacturing parameters in response to the calibration parameter data 162 and / or the model output data 156). In some embodiments, the corrective action includes providing machine learning (e.g., modifying one or more manufacturing parameters based on the calibration parameter data 162). In some embodiments, performing the corrective action includes causing an update of one or more manufacturing parameters. In some embodiments, the corrective action includes calibrating a physics-based model based on the calibration parameter data 162.

[0038] The manufacturing parameters may include hardware parameters (e.g., replacing a component, using a specific component, replacing a processing chip, updating firmware, etc.) and / or process parameters (e.g., temperature, pressure, flow rate, speed, current, voltage, gas flow, lift speed, etc.). In some embodiments, the corrective action includes triggering preventative operational maintenance (e.g., replacing, treating, cleaning, etc., of a manufacturing tool 124 component). In some embodiments, the corrective action includes triggering design optimization (e.g., updating manufacturing parameters, manufacturing process, manufacturing tool 124, etc., for an optimized product). In some embodiments, the corrective action includes updating a recipe (e.g., placing the manufacturing tool 124 in an idle mode, sleep mode, warm-up mode, etc.). In some embodiments, the corrective action includes updating a physics-based model.

[0039] Prediction server 112, server machine 170, and server machine 180 may each include one or more computing devices such as a rack-mounted server, a router computer, a server computer, a personal computer, a mainframe computer, a laptop computer, a tablet computer, a desktop computer, a graphics processing unit (GPU), an accelerator application-specific integrated circuit (ASIC) (e.g., a tensor processing unit (TPU)), etc. Prediction server 112, server machine 170, and server machine 180 may each include a cloud server or server capable of performing one or more cloud-based functions.

[0040] The prediction server 112 may include a calibration component 116. In some embodiments, the calibration component 116 receives the sensor data 142 (e.g., a subset 144 of the sensor data 142) and the model data 152 (e.g., the model input data 154 and the model output data 156) and generates outputs (e.g., the calibration parameter data 162 and / or the model output data 156) for performing corrective actions associated with the manufacturing equipment 124. In some embodiments, the calibration component 116 may use a trained machine learning model 190B to determine the outputs for performing the corrective actions. In some embodiments, the calibration component 116 may use a physics-based model (e.g., the model 190A) to determine at least a portion of the outputs for performing the corrective actions.

[0041] In some embodiments, the sensor data 142 and the model data 152 are provided to train a machine learning model 190B. The machine learning model 190B is trained to adjust the calibration parameter data 162 used in the physics-based model of the processing chamber. In some embodiments, the machine learning model 190B is trained based on inputs (e.g., the sensor data 142) of spacing data, chamber pressure data, heater temperature data, and / or chamber flow data from sensors 126 disposed at locations within the processing chamber. In some embodiments, one or more sensors 126 are not disposed in close proximity to the substrate. For example, temperature sensors may be disposed at various locations near gas intakes, exhaust conduits, substrate supports, chamber walls, etc. The model 190B adjusts the calibration parameter data 162. In some embodiments, the calibration parameter data 162 includes thermal contact resistance or thermal contact conductance data at contact points between components of the processing chamber.

[0042] Combining machine learning models with physics-based modeling offers technical advantages over other techniques. The physics-based model provides a map of manufacturing parameters (e.g., of a processing chamber). Due to manufacturing tolerance ranges, component aging, etc., traditional physics-based models may have inaccuracies (e.g., may not accurately model the processing chamber). For example, a heater may provide slightly less or more energy than expected, a gas flow regulator may not precisely allow the selected flow rate, contact between surfaces within the chamber may be slightly less than ideal, etc. Such variations may be unknown to the user and may not be captured by traditional physics-based models. By using machine learning model 190B to adjust calibration parameter data 162 (e.g., by implementing a machine learning model that performs virtual measurements of unmeasurable values, such as calibration parameter data 162 including thermal contact resistance and / or thermal contact conductance values), the physics-based model 190A of the present disclosure is more accurate than traditional models. The machine learning model 190B is trained for various input parameters (e.g., sensor data 142, model data 152) spanning a region of parameter space. Variations in actual conditions in the processing chamber within the parameter space in which the machine learning model was trained to operate (e.g., due to inaccuracies in the initial physics-based model) can be accounted for via interpolation. In addition, settings somewhat outside the region of parameter space used to train the machine learning model can be accounted for via extrapolation. In this manner, unexpected variations in the processing chamber operating conditions can be accounted for by using the calibration parameter data 162 provided by the prediction system 110.

[0043] In some embodiments, calibration component 116 receives sensor data 142 (e.g., subset 144 of sensor data 142) and may perform preprocessing, such as extracting patterns in the data or combining the data into new composite data. In some embodiments, calibration component 116 receives model data 152 (e.g., model input data 154 and model output data 156) associated with a physics-based model. Calibration component 116 may then provide sensor data 142 and model data 152 to train machine learning model 190B. Machine learning model 190B may be trained using data input including first subset 144 of sensor data 142 (e.g., subset 144A of sensor data 142 in FIG. 1B ) and model input data 154, and target output data including second subset 144 of sensor data 142 (e.g., subset 144B of sensor data 142 in FIG. 1B ) and model output data 156. In some embodiments, the first subset 144A of the sensor data 142 and the model input data 154 correspond to one or more first types of data. In some embodiments, the second subset 144B of the sensor data 142 and the model output data 156 correspond to one or more second types of data. In some embodiments, the one or more second types of data are different from the one or more first types of data. The calibration component 116 can receive one or more calibration parameter values ​​(e.g., calibration parameter data 162) associated with the physics-based model of the process chamber (e.g., model 190A) from the trained machine learning model 190B. The calibration component 116 can then trigger a corrective action. The corrective action can include sending an alert to the client device 120. The corrective action can also include updating manufacturing parameters of the manufacturing tool 124. The corrective action can also include updating the physics-based model (e.g., model 190A) based on the calibration parameter data 162 (e.g., the calibration parameter values).

[0044] The data store 140 may be a memory (e.g., random access memory), a drive (e.g., a hard drive, a flash drive), a database system, or another type of component or device capable of storing data. The data store 140 may be local (e.g., a local computer) or remote (e.g., a cloud-based system). The data store 140 may include multiple storage components (e.g., multiple drives or multiple databases) that may span multiple computing devices (e.g., multiple server computers). The data store 140 may store sensor data 142, model data 152, and calibration parameter data 162.

[0045] The sensor data 142 can include one or more subsets of the sensor data (e.g., subset 144 of the sensor data 142). The sensor data can include trace data over the duration of a manufacturing process, associations of data with physical sensors, pre-processed data (e.g., averages and composite data), and data indicative of sensor performance over time (i.e., many manufacturing processes).

[0046] The model data 152 may include features similar to the features of the sensor data 142 .

[0047] The calibration parameter data 162 may indicate one or more calibration parameter values ​​(e.g., calibration parameters) of the physics-based model 190A of the processing chamber of the manufacturing tool 124. The calibration parameter data 162 may also include predictions of the calibration parameter values.

[0048] In some embodiments, prediction system 110 further includes server machine 170 and server machine 180. Server machine 170 includes dataset generator 172 that can generate datasets (e.g., a set of data inputs and a set of target outputs) for training, validating, and / or testing machine learning model 190B. Some operations of dataset generator 172 are described in more detail below with respect to FIGS. 2 and 4A. In some embodiments, dataset generator 172 can divide sensor data (e.g., subset 144 of sensor data 142) and model data (e.g., model input data 154 and model output data 156) into a training set (e.g., 60 percent of the data), a validation set (e.g., 20 percent of the data), and a test set (e.g., 20 percent of the data). In some embodiments, prediction system 110 (e.g., via calibration component 116) generates multiple feature sets. For example, a first feature set may correspond to a first type of sensor dataset (e.g., from the first sensor set, a first combination of values ​​from the first sensor set, a first pattern of values ​​from the first sensor set) corresponding to each dataset (e.g., training set, validation set, and test set), and a second feature set may correspond to a second type of sensor dataset (e.g., from a second sensor set different from the first sensor set, a second combination of values ​​different from the first combination, a second pattern different from the first pattern) corresponding to each dataset.

[0049] Server machine 180 includes a training engine 182, a validation engine 184, a selection engine 185, and / or a test engine 186. The engines (e.g., training engine 182, validation engine 184, selection engine 185, and test engine 186) may refer to hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, processing device, etc.), software (e.g., instructions executing on a processing device, general-purpose computer system, or dedicated machine), firmware, microcode, or a combination thereof. Training engine 182 may be capable of training machine learning model 190B using one or more feature sets associated with a training set from dataset generator 172. Training engine 182 can generate multiple trained machine learning models 190B, each corresponding to a different feature set of the training set (e.g., sensor data from a different set of sensors). For example, a first trained machine learning model may have been trained using all features (e.g., X1-X5), a second trained machine learning model may have been trained using a first subset of features (e.g., X1, X2, X4), and a third trained machine learning model may have been trained using a second subset of features (e.g., X1, X3, X4, and X5) that may partially overlap with the first subset of features. Dataset generator 172 may receive the output of the trained machine learning model (e.g., model 190B), compile the data into training, validation, and test datasets, and use the datasets to train a second machine learning model.

[0050] The validation engine 184 may be capable of validating the trained machine learning models 190B using corresponding feature sets of the validation set from the dataset generator 172. For example, a first trained machine learning model 190B trained using a first feature set of the training set may be validated using a first feature set of the validation set. The validation engine 184 may determine the accuracy of each of the trained machine learning models 190B based on the corresponding feature sets of the validation set. The validation engine 184 may discard trained machine learning models 190B having an accuracy that does not meet a threshold accuracy. In some embodiments, the selection engine 185 may be capable of selecting one or more trained machine learning models 190B having an accuracy that meets the threshold accuracy. In some embodiments, the selection engine 185 may be capable of selecting the trained machine learning model 190B with the highest accuracy among the trained machine learning models 190B.

[0051] The testing engine 186 may be capable of testing the trained machine learning model 190B using a corresponding feature set of a test set from the dataset generator 172. For example, a first trained machine learning model 190B trained using a first feature set of the training set may be tested using a first feature set of the test set. The testing engine 186 may determine, based on the test set, the trained machine learning model 190B that has the highest accuracy among all of the trained machine learning models.

[0052] The machine learning model 190B may refer to a model artifact created by the training engine 182 using a training set that includes data inputs and corresponding target outputs (correct answers for each training input). Patterns that map data inputs to target outputs (correct answers) can be found in the data set, and the machine learning model 190B is provided with a mapping that captures these patterns. In some embodiments, the machine learning model 190B reconstructs calibration parameter values ​​associated with a physics-based model of the process chamber. The machine learning model 190B can use one or more of a support vector machine (SVM), a radial basis function (RBF), clustering, supervised machine learning, semi-supervised machine learning, unsupervised machine learning, a k-nearest neighbor algorithm (k-NN), linear regression, random forests, a neural network (e.g., an artificial neural network), etc.

[0053] The calibration component 116 can provide model input data 154 to the model 190A and receive model output data 156 from the model 190A. The calibration component 116 can provide the sensor data 142 and the model data 152 to train the machine learning model 190B and adjust the calibration parameter data 162. The calibration component 116 can provide the model input data 154 and the calibration parameter data 162 to the model 190A and receive more accurate model output data 156 from the model 190A.

[0054] For purposes of illustration and not limitation, embodiments of the present disclosure describe training one or more machine learning models 190B using sensor data (e.g., sensor data 142) and model data (e.g., model data 152) to adjust calibration parameter data 162. In other implementations, heuristic or rule-based models are used to adjust calibration parameter data 162 (e.g., by specifying values ​​for parameters that control the sparsity of the fit without using a trained machine learning model). Any of the information described with respect to data input 210 in FIG. 2 may be supervised or otherwise used in a heuristic or rule-based model.

[0055] In some embodiments, the functionality of client device 120, prediction server 112, server machine 170, and server machine 180 may be provided by fewer machines. For example, in some embodiments, server machines 170 and 180 may be combined into a single machine, while in some other embodiments, server machine 170, server machine 180, and prediction server 112 may be combined into a single machine. In some embodiments, client device 120 and prediction server 112 may be combined into a single machine.

[0056] In general, functions described in one embodiment as being performed by client device 120, prediction server 112, server machine 170, and server machine 180 may also be performed on prediction server 112 in other embodiments, where appropriate. In addition, functions attributed to particular components may be performed by different components or multiple components operating together. For example, in some embodiments, prediction server 112 may determine corrective actions based on calibration parameter data 162. In another example, client device 120 may determine calibration parameter data 162 based on a trained machine learning model (e.g., model 190B).

[0057] Additionally, the functionality of a particular component may be performed by different components or multiple components working together. One or more of prediction server 112, server machine 170, or server machine 180 may be accessed as a service offered to other systems or devices via an appropriate application programming interface (API).

[0058] In embodiments, a "user" may be represented as a single individual. However, other embodiments of the present disclosure encompass a "user" that is an entity controlled by multiple users and / or automated sources. For example, a set of individual users aggregated as a group of administrators may be considered a "user."

[0059] Embodiments of the present disclosure may be applied to data quality assessment, feature enhancement, model evaluation, virtual metrology (VM), predictive maintenance (PdM), marginal optimization, and the like.

[0060] Although embodiments of the present disclosure are described with respect to generating calibration parameter data 162 for performing corrective actions at a manufacturing facility (e.g., a semiconductor manufacturing facility), the embodiments may also be applied to improved data processing in general by utilizing virtual measurement and calibration parameter data 162 based on physical information.

[0061] FIG. 1B is a block diagram illustrating a data flow 100B related to adjusting calibration parameter data 162 (e.g., to calibrate a physics-based model of a processing chamber) according to certain embodiments. In some embodiments, model 190A receives model input data 154. In some embodiments, model 190A is a physics-based model of a processing chamber. Model input data 154 may be input parameters of the physics-based model. In some embodiments, model input data 154 is for the entire operating space of the processing chamber. For example, model input data 154 may include a set of parameters covering the entire range of processing chamber operation, such as the entire range of spacing parameters, the entire range of chamber pressure parameters, the entire range of backside pressure parameters, the entire range of heater temperature parameters, and / or the entire range of gas flow parameters. Model input data 154 may include a substantial portion of the combination of input parameters associated with the processing chamber. In some embodiments, model input data 154 includes manufacturing parameters used in a substrate processing recipe.

[0062] In response to receiving the input, model 190A generates model output data 156. In some embodiments, model output data 156 is associated with the process chamber. For example, in some embodiments, model output data 156 includes one or more modeled temperature values ​​of one or more components of the process chamber based on model input data 154. In some embodiments, model output data 156 includes a set of modeled temperature values ​​for each set of model input data 154 (e.g., input parameters) input to model 190A.

[0063] Initially, the model output data 156 from the physics-based model 190A may be inaccurate. In some embodiments, the physics-based model 190A uses unadjusted (e.g., or unfurther adjusted) calibration parameter data 162 (e.g., one or more calibration parameter values) to generate the model output data 156. The accuracy of the physics-based model 190A is improved by adjusting the calibration parameter data 162 (e.g., one or more calibration parameters).

[0064] Model input data 154 and subset 144A of sensor data 142 are provided as training inputs, and model output data 156 and subset 144B of sensor data 142 are provided as target outputs to model 190B (e.g., to an untrained machine learning model, or to a trained machine learning model to be further trained). Model 190B is trained to adjust calibration parameter data 162 (e.g., to generate one or more adjusted calibration parameters). In some embodiments, model input data 154 and model output data 156 are provided to the untrained machine learning model by calibration component 116 (see FIG. 1A ). In addition, a first subset of sensor data (e.g., subset 144A of sensor data 142) and a second subset of sensor data (e.g., subset 144B of sensor data 142) are provided to the untrained machine learning model (e.g., by calibration component 116).

[0065] In some embodiments, after machine learning model 190B is trained, calibration parameter data 162 (e.g., one or more adjusted calibration parameters) are retrieved from the trained machine learning model 190B. The calibration parameter data 162 (e.g., the adjusted calibration parameters) and the model input data 154 are provided as inputs to physics-based model 190A. In some embodiments, physics-based model 190A is updated based on the adjusted calibration parameters. The accuracy of physics-based model 190A is improved based on the calibration parameter data 162. Providing the adjusted calibration parameters to the physics-based model (e.g., model 190A) enables more accurate modeling of the processing chamber. In some embodiments, providing the adjusted calibration parameters to the physics-based model generates a more accurate digital twin of the processing chamber.

[0066] 2 is a block diagram of an exemplary dataset generator 272 (e.g., dataset generator 172 of FIG. 1A ) used to create a dataset for a machine learning model (e.g., model 190B of FIG. 1A ), according to certain embodiments. Dataset generator 272 may be part of server machine 170 of FIG. 1A . In some embodiments, system 100 of FIG. 1A includes multiple machine learning models. In such cases, each model may have a separate dataset generator, or the models may share a dataset generator.

[0067] 2, a system 200 including a dataset generator 272 (e.g., dataset generator 172 of FIG. 1A) creates a dataset for a machine learning model (e.g., model 190B of FIG. 1A). The dataset generator 272 can create the dataset using sensor data and model data of a physics-based model (e.g., model 190A of FIG. 1A). In some embodiments, the dataset generator 272 creates training inputs by selecting a subset of model input data and model output data for the physics-based model (e.g., a subset of model data 152 of FIG. 1A). For example, the output of the physics-based model can be a predicted temperature value of a component within a process chamber.

[0068] The dataset generator 272 can form a training set based on the sensor data and the model input data. In some embodiments, the data input 210 includes actual data (e.g., sensor data) and simulated data (e.g., representing sensor data that may be received from a sensor). The data input can include a subset 244A-L of the sensor data 242 and a set of model input data 254A-Z. The dataset generator 272 also generates target outputs 220 for training the machine learning model. The target outputs include a subset 244M-Z of the sensor data 242 and a set of model output data 256A-Z output from a physics-based model of the process chamber. In some implementations, the target outputs 220 include a set of data outputs (e.g., model output data) from the physics-based model that indicate predictions of the physics-based model based on one or more inputs to the model (e.g., model input data). The set of data output from the physics-based model can include predicted component temperature values ​​for each component of the process chamber of interest. The data input 210 and the target output 220 are fed into a machine learning model (eg, model 190B in FIG. 1A).

[0069] 2 , in some embodiments, the dataset generator 272 generates a dataset (e.g., a training set, a validation set, a test set) that includes one or more data inputs 210 (e.g., training inputs, validation inputs, test inputs) and may include one or more target outputs 220 corresponding to the data inputs 210. The dataset may also include mapping data that maps the data inputs 210 to the target outputs 220. The data inputs 210 may also be referred to as “features,” “attributes,” or “information.” In some embodiments, the dataset generator 272 may provide a dataset to the training engine 182, the validation engine 184, or the test engine 186 of FIG. 1A , where the dataset is used to train, validate, or test the machine learning model 190B of FIG. 1A . Some embodiments of generating a training set may be further described with respect to FIG. 4A .

[0070] In some embodiments, the dataset generator 272 may generate a first data input corresponding to a first subset 244A of the sensor data 242 and a first set of model input data 254A for training, validating, or testing a first machine learning model, and the dataset generator 272 may generate a second data input corresponding to a second subset 244B of the sensor data 242 and a second set of model input data 254B for training, validating, or testing a second machine learning model.

[0071] In some embodiments, the dataset generator 272 can perform operations on one or more of the data inputs 210 and the target outputs 220. The dataset generator 272 can extract patterns from the data (slope, curvature, etc.), combine the data (average, feature generation, etc.), or group the simulated sensors to train separate models.

[0072] The data inputs 210 and target outputs 220 for training, validating, or testing a machine learning model can include information about a particular processing chamber (e.g., a particular substrate processing chamber). The data inputs 210 and target outputs 220 can include information about a particular processing chamber design (e.g., to be used for all processing chambers of that design).

[0073] In some embodiments, the information used to train the machine learning model may be from a particular type of manufacturing equipment in a manufacturing facility having particular characteristics (e.g., manufacturing equipment 124 in FIG. 1A ), allowing the trained machine learning model to determine outcomes for a particular group of manufacturing equipment 124 based on sensor data and / or model data associated with one or more components that share the characteristics of the particular group. In some embodiments, the information used to train the machine learning model may be about components from more than one manufacturing facility, allowing the trained machine learning model to determine outcomes for components based on input from one manufacturing facility.

[0074] In some embodiments, after generating a dataset and using the dataset to train, validate, or test a machine learning model, the machine learning model may be further trained, validated, or tested, or tuned.

[0075] 3 is a block diagram illustrating a system 300 for generating calibration parameter data 362 (e.g., calibration parameter data 162 of FIG. 1A), according to certain embodiments. System 300 can be used to adjust calibration parameter data 362 and implement corrective actions using input from sensors and modeling techniques (e.g., physics-based modeling and / or machine learning modeling).

[0076] 3, in block 310, system 300 (e.g., a component of forecasting system 110 of FIG. 1A) performs data splitting of historical data 360 (e.g., via dataset generator 172 of server machine 170 of FIG. 1A) to generate training set 302, validation set 304, and test set 306. For example, the training set may be 60% of the simulated data, the validation set may be 20% of the simulated data, and the test set may be 20% of the simulated data. Historical data 360 may include sensor data 342 (e.g., sensor data 142 of FIG. 1A), model input data 354 (e.g., model input data 154 of FIG. 1A), and model output data 356 (e.g., model output data 156 of FIG. 1A).

[0077] At block 312, the system 300 performs model training (e.g., via the training engine 182 of FIG. 1A ) using the training set 302. The system 300 may train multiple models using multiple feature sets of the training set 302 (e.g., a first feature set including a group of simulated sensors in the training set 302, a second feature set including a different group of simulated sensors in the training set 302, etc.). For example, the system 300 may train machine learning models to generate a first trained machine learning model using the first feature set of the training set and a second trained machine learning model using the second feature set of the training set (e.g., data different from the data used to train the first machine learning model). In some embodiments, the first trained machine learning model and the second trained machine learning model may be combined to generate a third trained machine learning model (e.g., potentially a better predictor than the first or second trained machine learning models alone). In some embodiments, the feature sets used to compare models may overlap (e.g., one model may be trained using simulated sensors 1-15, and a second model may be trained using simulated sensors 10-20). In some embodiments, hundreds of models may be generated, including models with various feature permutations and model combinations.

[0078] At block 314, the system 300 performs model validation using the validation set 304 (e.g., via the validation engine 184 of FIG. 1A ). The system 300 may validate each of the trained models using the corresponding feature set of the validation set 304. For example, the validation set 304 may use the same subset of simulated sensors used in the training set 302, but for different input conditions. In some embodiments, the system 300 may validate hundreds of models (e.g., models with various permutations of features, combinations of models, etc.) generated at block 312. At block 314, the system 300 may determine the accuracy of each of the one or more trained models (e.g., via model validation) and may determine whether one or more of the trained models have an accuracy that meets a threshold accuracy. In response to determining that none of the trained models have an accuracy that meets the threshold accuracy, flow returns to block 312, and the system 300 performs model training using a different feature set from the training set. In response to determining that one or more of the trained models have an accuracy that meets the threshold accuracy, flow proceeds to block 316. The system 300 may discard trained machine learning models that have an accuracy below the threshold accuracy (e.g., based on a validation set).

[0079] In block 316, the system 300 performs model selection (e.g., via the selection engine 185 of FIG. 1A ) to determine which of the one or more trained models that meet the threshold accuracy has the highest accuracy (e.g., the selected model 308 based on the validation of block 314). In response to determining that two or more of the trained models that meet the threshold accuracy have the same accuracy, flow can return to block 312, where the system 300 performs model training using a further refined training set corresponding to the further refined feature set to determine the trained model with the highest accuracy.

[0080] At block 318, the system 300 performs model testing (e.g., via the test engine 186 of FIG. 1A ) using the test set 306 to test the selected model 308. The system 300 may test the first trained machine learning model using a first feature set of the test set (e.g., simulated sensors 1-15) to determine (e.g., based on the first feature set of the test set 306) that the first trained machine learning model meets a threshold accuracy. In response to the accuracy of the selected model 308 not meeting the threshold accuracy (e.g., the selected model 308 is overfit to the training set 302 and / or the validation set 304 and is not applicable to other datasets, such as the test set 306), flow proceeds to block 312, where the system 30A performs model training (e.g., retraining) using a different feature set or a different training set, which may correspond to a reorganization of the boards divided into training, validation, and test sets. In response to determining that the selected model 308 has an accuracy that meets the threshold accuracy based on the test set 306, flow proceeds to block 320. At least in block 312, the model can learn patterns in the simulated sensor data to make predictions, and in block 318, the system 300 can apply the model to the remaining data (e.g., the test set 306) to test the predictions.

[0081] In block 320, the system 300 determines calibration parameter data 362 from the trained model 320 (e.g., the selected model 308). Training the model 320 adjusts the calibration parameter data 362. The adjusted calibration parameter data 362 can be used to perform an action (e.g., perform a corrective action in connection with the illustrated manufacturing equipment 124, provide and alert the client device 120 of FIG. 1A, update a physics-based model, etc.).

[0082] The calibration parameter data 362 and the model input data 354 are provided to a physics-based model (e.g., model 190A in FIG. 1A ) to generate model output data 156. The current data 346 may include sensor data 342, model input data 354, and updated model output data 356 generated based on the model input data 354 and the adjusted calibration parameter data 362.

[0083] In some embodiments, retraining of the machine learning model is performed by supplying additional data (e.g., current data 346) to further train the model. The current data 346 may be provided in block 312. The current data 346 may differ from the historical data 360 originally used to train the model by incorporating input parameters that were not part of the original training, input parameters outside the parameter space covered by the original training, or may be updated to reflect chamber-specific knowledge (e.g., variations from an ideal chamber due to manufacturing tolerance ranges, aging components, etc.). The selected model 308 may be retrained based on this data. In some embodiments, the selected model 308 is retrained or further trained until the change in the calibration parameter data 362 in response to training of the selected model 308 meets a threshold change (e.g., changes less than a threshold amount).

[0084] In some embodiments, one or more of acts 310-320 may be performed in various orders and / or with other acts not presented and described herein. In some embodiments, one or more of acts 310-320 may not be performed. For example, in some embodiments, one or more of data partitioning of block 310, model validation of block 314, model selection of block 316, or model testing of block 318 may not be performed.

[0085] 4A-4C are flowcharts of methods 400A-C associated with generating calibration parameter data for triggering corrective actions, according to certain embodiments. Methods 400A-C may be performed by processing logic, which may include hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, a processing device, etc.), software (e.g., instructions executing on a processing device, a general-purpose computer system, or a dedicated machine), firmware, microcode, or a combination thereof. In some embodiments, methods 400A-C may be performed in part by prediction system 110 of FIG. 1A. Method 400A may be performed in part by prediction system 110 (e.g., server machine 170 and dataset generator 172 of FIG. 1A, dataset generator 272 of FIG. 2). Prediction system 110 may use method 400A to generate a dataset for at least one of training, validating, or testing a machine learning model, according to embodiments of the present disclosure. Method 400B may be performed by server machine 180 (e.g., training engine 182, etc.). Method 400C may be performed by prediction server 112 (e.g., calibration component 116). In some embodiments, a non-transitory storage medium stores instructions that, when executed by a processing device (e.g., prediction system 110, server machine 180, prediction server 112, etc.), cause the processing device to perform one or more of methods 400A-C.

[0086] For ease of explanation, methods 400A-C are shown and described as a series of operations. However, operations in accordance with the present disclosure may occur in various orders and / or simultaneously, as well as with other operations not shown and described herein. Moreover, not all illustrated operations may be performed to implement methods 400A-C in accordance with the disclosed subject matter. In addition, those skilled in the art will understand and appreciate that methods 400A-C may alternatively be represented as a series of interrelated states via a state diagram or events.

[0087] FIG. 4A is a flow diagram of a method 400A for generating a dataset for a machine learning model for generating calibration parameter data (eg, calibration parameter data 162 of FIG. 1A) according to certain embodiments.

[0088] Referring to FIG. 4A, in some embodiments, at block 401, processing logic performing method 400A initializes a training set T to an empty set.

[0089] At block 402, processing logic generates a first data input (e.g., a first training input, a first validation input) that may include a first subset of sensor data (e.g., subset 144A of sensor data 142 in FIG. 1B) and model input data (e.g., set of model input data 154 in FIG. 1B). In some embodiments, the first data input may include a first set of features for a type of data, and the second data input may include a second set of features for a type of data (e.g., as described with respect to FIG. 3).

[0090] At block 403, processing logic generates a first target output for one or more of the data inputs (e.g., a first data input). In some embodiments, the first target output includes a second subset of sensor data (e.g., subset 244B of sensor data 142 of FIG. 1B) and model output data (e.g., model output data 156 of FIG. 1B). The model output data may be an output of a physics-based model (e.g., model 190A of FIG. 1B) based on inputs of model input data 154.

[0091] At block 404, processing logic optionally generates mapping data indicating an input-output mapping, which may refer to a data input (e.g., one or more of the data inputs described herein), a target output for the data input, and an association between the data input and the target output.

[0092] At block 405, processing logic, in some embodiments, adds the mapping data generated at block 404 to dataset T.

[0093] At block 406, processing logic branches based on whether dataset T is sufficient for at least one of training, validation, and / or testing of machine learning model 190 of FIG. 1A. If so, execution proceeds to block 407; if not, execution continues back to block 402. Note that while in some embodiments the sufficiency of dataset T may be determined simply based on the number of inputs in the dataset, which in some embodiments are mapped to outputs, in some other implementations the sufficiency of dataset T may be determined based on one or more other criteria in addition to or instead of the number of inputs (e.g., a measure of diversity of the data examples, accuracy, etc.).

[0094] At block 407, processing logic provides dataset T (e.g., to server machine 180 of FIG. 1A ) to train, validate, and / or test machine learning model 190. In some embodiments, dataset T is a training set and is provided to training engine 182 of server machine 180 to perform training. In some embodiments, dataset T is a validation set and is provided to validation engine 184 of server machine 180 to perform validation. In some embodiments, dataset T is a test set and is provided to test engine 186 of server machine 180 to perform testing. After block 407, the machine learning model (e.g., machine learning model 190B) can be at least one of trained using training engine 182 of server machine 180, validated using validation engine 184 of server machine 180, or tested using test engine 186 of server machine 180. The trained machine learning model can be implemented by the calibration component 116 (of the prediction server 112) to adjust the calibration parameter data 162 to perform corrective actions associated with the manufacturing equipment 124.

[0095] FIG. 4B is a method 400B for training a machine learning model (eg, model 190B of FIG. 1A) to adjust calibration parameter data and perform corrective actions, according to certain embodiments.

[0096] 4B , at block 410 of method 400B, processing logic receives sensor data from one or more sensors of a processing chamber of a manufacturing tool (e.g., sensor 126 of manufacturing tool 124). The sensor data may include sensor values ​​associated with the processing chamber during a manufacturing process for a particular configuration of manufacturing parameters (e.g., recipe parameters, process parameters, hardware settings, etc.). For example, the sensor data may be collected during an experimental run of a process operation in the processing chamber. The sensor data may include a first subset and a second subset, and the first subset is mapped to the second subset. In some embodiments, the first subset of sensor data includes one or more of interval data, chamber pressure data, heater temperature data, or chamber flow rate data. In some embodiments, the second subset includes component temperature data associated with one or more components of the processing chamber.

[0097] At block 412, processing logic identifies model input data and model output data, and the model output data is output from the physics-based model. In some embodiments, the model input data and model output data (e.g., model input data 154 and model output data 156 of FIG. 1A ) are associated with a physics-based model of the processing chamber (e.g., model 190A of FIG. 1A ). In some embodiments, the model input data is a set of inputs to the physics-based model that represents the operating space of the processing chamber. For example, the model input data can substantially represent ranges of input parameter values ​​(e.g., temperature, pressure, etc.) of the processing chamber. As another example, the model input data can represent a design of experiments (DOE) that corresponds to the physics-based model, where the variables of the DOE are one or more input parameters of the physics-based model. In some embodiments, the model output data is a set of outputs of the physics-based model, where each output corresponds to an input. For example, the model output data is a database of results from running a DOE that corresponds to the physics-based model. In some embodiments, the model output data is output from the physics-based model of the processing chamber. The model output data is mapped to the model input data. In some embodiments, the physics-based model initially utilizes calibration parameters that have not yet been adjusted. Adjusting the calibration parameters improves the accuracy of the physics-based model. In some embodiments, the calibration parameters are one or more of predicted thermal contact resistance values ​​or predicted thermal contact conductance values ​​between corresponding components of the processing chamber.

[0098] At block 414, processing logic trains the machine learning model. The machine learning model is trained using data inputs including a first subset of sensor data and model input data. The machine learning model is trained using a second subset of sensor data and target output data including model output data. Training the machine learning model involves adjusting one or more calibration parameters (e.g., calibration parameter data 162 of FIGS. 1A-1B ) associated with the physics-based model. Prior to training the machine learning model, ranges of values ​​can be assigned to the one or more calibration parameters. In some embodiments, adjusting the one or more calibration parameters includes adjusting corresponding values ​​of the one or more calibration parameters within a range of values. In some embodiments, a user (e.g., a human user) assigns a range of values ​​associated with the calibration parameters. In some embodiments, a processing device assigns a range of values ​​associated with the calibration parameters. In some embodiments, the calibration parameters are included in calibration parameter data 162 of FIG. 1A . In some embodiments, the adjusted calibration parameters are determined based on an algorithm of the machine learning model. The adjusted calibration parameters are provided to the physics-based model. In some embodiments, the physics-based model is updated based on the adjusted calibration parameters. By updating the physics-based model based on the adjusted calibration parameters, the physics-based model becomes a more accurate representation of the processing chamber and can better predict the behavior of the processing chamber.

[0099] At block 416, the machine learning model may be retrained or further trained using data inputs including the first subset of sensor data and the updated model input data, and target output data including the second subset of sensor data and the updated model output data, to further adjust the calibration parameters. In some embodiments, the updated model input data and the updated model output data are associated with an updated physics-based model (e.g., an updated physics-based model based on the adjusted calibration parameters). By retraining the machine learning model, the calibration parameters can be further adjusted. In some embodiments, the further adjusted calibration parameters are provided to the physics-based model. The physics-based model is further updated based on the further adjusted calibration parameters. By further updating the physics-based model, the physics-based model becomes a more accurate representation of the process chamber and can better predict the behavior of the process chamber.

[0100] FIG. 4C is a method 400C for using calibration parameter data adjusted via a trained machine learning model (eg, model 190 of FIG. 1A) according to certain embodiments.

[0101] 4C , at block 420 of method 400C, processing logic identifies adjusted calibration parameters by training a machine learning model (e.g., trained via block 414 of FIG. 4B). The adjusted calibration parameters and model input data are provided to a physics-based model, which outputs updated model output data (e.g., the physics-based model is updated based on the adjusted calibration parameters). In some embodiments, the machine learning model is trained according to the method described in connection with FIG. 4B.

[0102] At block 422, processing logic identifies second model input data. In some embodiments, the second model input data represents a set of input parameters (e.g., one or more of a spacing parameter, a chamber pressure parameter, a backside pressure parameter, a heater temperature parameter, a gas flow parameter, etc.) to be input to the physics-based model. In some embodiments, the second model input data is input to the updated physics-based model to calculate one or more outputs. In some embodiments, the second model input data is provided by a user (e.g., a human user). In some embodiments, the second model input data is provided to the physics-based model after the physics-based model has been updated with the adjusted calibration parameters.

[0103] At block 424, processing logic receives second model output data from the physics-based model. Receiving the second model output data is in response to providing the second model input data and the adjusted calibration parameters as inputs to the physics-based model. In some embodiments, the physics-based model is updated based on the adjusted calibration parameters in response to providing the calibration parameters. In some embodiments, the second model output data is generated by the physics-based model after the physics-based model has been updated based on the adjusted calibration parameters.

[0104] At block 426, processing logic triggers one or more corrective actions based on the second model output data. In some embodiments, the one or more corrective actions are associated with the process chamber. In some embodiments, the corrective actions include providing an alert, interrupting a manufacturing device, updating manufacturing parameters, etc.

[0105] FIG. 5 is a block diagram illustrating a computer system 500, according to certain embodiments. In some embodiments, computer system 500 may be connected to other computer systems (e.g., via a network such as a local area network (LAN), an intranet, an extranet, or the Internet). Computer system 500 can operate in the capacity of a server or a client computer in a client-server environment, or as a peer computer in a peer-to-peer or distributed network environment. Computer system 500 may be provided by a personal computer (PC), a tablet PC, a set-top box (STB), a personal digital assistant (PDA), a mobile phone, a web appliance, a server, a network router, switch, or bridge, or any device capable of executing a set of instructions (sequential or otherwise) that specify actions to be taken by that device. Furthermore, the term “computer” is intended to include any collection of computers that individually or jointly execute a set (or sets) of instructions to perform any one or more of the methodologies described herein. Computer system 500 may include one or more of client device 120, prediction server 112, server machine 170, server machine 180, etc.

[0106] In a further aspect, the computer system 500 may include a processing device 502, a volatile memory 504 (e.g., random access memory (RAM)), a non-volatile memory 506 (e.g., read-only memory (ROM) or electrically erasable programmable ROM (EEPROM)), and a data storage device 518, which may communicate with each other via a bus 508.

[0107] The processing device 502 may be provided by one or more processors, such as a general-purpose processor (e.g., a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a microprocessor implementing other types of instruction sets, or a microprocessor implementing a combination of multiple types of instruction sets) or a special-purpose processor (e.g., an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), or a network processor).

[0108] Computer system 500 may further include a network interface device 522 (e.g., coupled to a network 574). Computer system 500 may also include a video display unit 510 (e.g., an LCD), an alphanumeric input device 512 (e.g., a keyboard), a cursor control device 514 (e.g., a mouse), and a signal generating device 520.

[0109] In some implementations, the data storage device 518 may include a non-transitory computer-readable storage medium 524 (e.g., a non-transitory machine-readable storage medium) (e.g., a non-transitory machine-readable storage medium that stores instructions) that may store instructions 526 that encode any one or more of the methods or functions described herein, including instructions for encoding the components of FIG. 1A (e.g., calibration component 116, models 190A-B, etc.) and implementing the methods described herein.

[0110] The instructions 526 may also reside, completely or partially, within the volatile memory 504 and / or within the processing device 502 during execution thereof by the computer system 500, and thus the volatile memory 504 and the processing device 502 may also constitute machine-readable storage media.

[0111] Although the computer-readable storage medium 524 is shown as a single medium in the illustrative example, the term "computer-readable storage medium" is intended to include a single medium or multiple media (e.g., a centralized or distributed database and / or associated caches and servers) that store one or more sets of executable instructions. The term "computer-readable storage medium" is also intended to include any tangible medium that is capable of storing or encoding a set of instructions for execution by a computer that cause the computer to perform any one or more of the methodologies described herein. The term "computer-readable storage medium" is intended to include, but is not limited to, solid-state memory, optical media, and magnetic media.

[0112] The methods, components, and features described herein may be implemented by discrete hardware components or integrated into the functionality of other hardware components, such as an ASIC, FPGA, DSP, or similar device. Furthermore, the methods, components, and features may be implemented by firmware modules or functional circuits within a hardware device. Furthermore, the methods, components, and features may be implemented in any combination of hardware devices and computer program components, or in a computer program.

[0113] Unless otherwise specified, terms such as "receive," "execute," "provide," "obtain," "cause," "access," "determine," "add," "use," "identify," "train," "interrupt," "update," and the like refer to actions and processes performed or implemented by a computer system that manipulate and transform data represented as physical (electronic) quantities in computer system registers and memory into other data similarly represented as physical quantities in computer system memory or registers or other such information storage, transmission, or display devices. Also, as used herein, terms such as "first," "second," "third," "fourth," and the like are intended as labels to distinguish different elements and need not have any significance indicative of an ordering by their numerical designations.

[0114] The examples described herein also relate to apparatus for performing the methods described herein. This apparatus may be specially constructed to perform the methods described herein, or may comprise a general-purpose computer system selectively programmed by a computer program stored on the computer system. Such a computer program may be stored on a computer-readable tangible storage medium.

[0115] The methods and illustrative embodiments described herein are not inherently related to any particular computer or other apparatus. Various general-purpose systems may be used in accordance with the teachings described herein, or it may prove convenient to construct more specialized apparatus to perform the methods described herein and / or each of their individual functions, routines, subroutines, or operations. Examples of configurations for a variety of these systems are set forth in the description above.

[0116] The above description is intended to be illustrative, not limiting. While the present disclosure has been described with reference to particular illustrative examples and implementations, it will be recognized that the present disclosure is not limited to the described examples and implementations. The scope of the present disclosure should be determined with reference to the following claims, along with the full scope of equivalents to which such claims are entitled.

Claims

1. receiving sensor data associated with processing of a substrate through a processing chamber of a substrate processing apparatus from a plurality of sensors, the sensor data including a first subset received from one or more first sensors and a second subset received from one or more second sensors, the first subset being mapped to the second subset; identifying model input data and model output data, the model output data being output from a physics-based model based on the model input data; training a machine learning model using data inputs comprising the first subset and the model input data and target output data comprising the second subset and the model output data to adjust one or more calibration parameters of the machine learning model, wherein the one or more calibration parameters are used by the physics-based model to perform one or more corrective actions associated with the processing chamber; A method comprising:

2. the first subset of sensor data includes one or more of interval data, chamber pressure data, heater temperature data, or chamber flow data; the second subset of sensor data includes component temperature data associated with one or more components of the processing chamber; The method of claim 1.

3. 2. The method of claim 1 , wherein the first subset of sensor data and the model input data correspond to one or more first types of data, and the second subset of sensor data and the model output data correspond to one or more second types of data that are different from the one or more first types of data.

4. The method of claim 1 , wherein the calibration parameters include one or more of predicted thermal contact resistance values ​​or predicted thermal contact conductance values ​​between corresponding components of the processing chamber.

5. 10. The method of claim 1, wherein the physics-based model comprises a digital twin model used to update processing parameters of the processing chamber.

6. 10. The method of claim 1, wherein, in response to the one or more calibration parameters being adjusted, the model input data and the one or more calibration parameters are input into the physics-based model to generate updated model output data, and the updated model output data is used to perform the one or more corrective actions.

7. The one or more corrective actions: providing warnings; interrupting operation of the processing chamber; or updating manufacturing parameters of the processing chamber; The method of claim 1 , comprising one or more of:

8. 10. The method of claim 1, further comprising assigning a range of values ​​to the one or more calibration parameters prior to training the machine learning model, and wherein adjusting the one or more calibration parameters comprises adjusting corresponding values ​​of the one or more calibration parameters within the range of values.

9. identifying one or more calibration parameters tuned by training a machine learning model, the machine learning model being trained with data inputs comprising a first subset of sensor data and first model input data, and target output data comprising a second subset of the sensor data and first model output data, the sensor data being received from a plurality of sensors, the sensor data being associated with processing of substrates through a processing chamber of a substrate processing apparatus, and the first model output data being output from a physics-based model based on the first model input data; Identifying second model input data; receiving second model output data from the physics-based model in response to providing the second model input data and the calibration parameters as inputs to the physics-based model, wherein one or more corrective actions associated with the processing chamber are performed based on the second model output data; and A method comprising:

10. the first subset of sensor data includes one or more of interval data, chamber pressure data, heater temperature data, or chamber flow data; the second subset of sensor data includes component temperature data associated with one or more components of the processing chamber; 10. The method of claim 9.

11. 10. The method of claim 9, wherein the first subset of sensor data and the first model input data correspond to one or more first types of data, and the second subset of sensor data and the first model output data correspond to one or more second types of data different from the one or more first types of data.

12. 10. The method of claim 9, wherein the calibration parameters include one or more predicted thermal contact resistance or conductance values ​​between corresponding components of the processing chamber.

13. 10. The method of claim 9, wherein the physics-based model comprises a digital twin model used to update processing parameters of the processing chamber.

14. 10. The method of claim 9, wherein, in response to the one or more calibration parameters being adjusted, the first model input data and the one or more calibration parameters are input to the physics-based model to generate updated model output data, and the one or more corrective actions are performed based on the updated model output data.

15. The one or more corrective actions providing warnings; interrupting operation of the processing chamber; or updating manufacturing parameters of the processing chamber; 10. The method of claim 9, comprising one or more of:

16. A non-transitory machine-readable storage medium storing instructions that, when executed, cause a processing device to: receiving sensor data associated with processing of a substrate through a processing chamber of a substrate processing apparatus from a plurality of sensors, the sensor data including a first subset received from one or more first sensors and a second subset received from one or more second sensors, the first subset being mapped to the second subset; identifying model input data and model output data, the model output data being output from a physics-based model based on the model input data; training a machine learning model using data inputs comprising the first subset and the model input data and target output data comprising the second subset and the model output data to adjust one or more calibration parameters of the machine learning model, wherein the one or more calibration parameters are used by the physics-based model to perform one or more corrective actions associated with the processing chamber; A non-transitory machine-readable storage medium that causes operations to be performed, including:

17. 17. The non-transitory machine-readable storage medium of claim 16, wherein the first subset of sensor data and the model input data correspond to one or more first types of data, and the second subset of sensor data and the model output data correspond to one or more second types of data that are different from the one or more first types of data.

18. 20. The non-transitory machine-readable storage medium of claim 16, wherein the physics-based model comprises a digital twin model used to update processing parameters of the processing chamber.

19. 17. The non-transitory machine-readable storage medium of claim 16, wherein in response to the one or more calibration parameters being adjusted, the model input data and the one or more calibration parameters are input to the physics-based model to generate updated model output data, and the one or more corrective actions are performed based on the updated model output data.

20. 17. The non-transitory machine-readable storage medium of claim 16, wherein the processing device further assigns ranges of values ​​to the one or more calibration parameters before training the machine learning model, and wherein adjusting the one or more calibration parameters comprises adjusting corresponding values ​​of the one or more calibration parameters within the ranges of values.

Images