Method for diagnosing substrate manufacturing chambers using physics-based models - Patent Application 20070122967

Abstract

The method includes receiving first sensor data generated during the manufacturing process by a sensor associated with a substrate manufacturing chamber. The method further includes receiving simulated sensor data generated by a trained physics-based model. The method further includes determining which one or more components of the manufacturing chamber contribute to a difference between the first sensor data and the simulated sensor data. The method further includes performing corrective action in consideration of the difference.

Description

[Technical Field]

[0001] The present disclosure relates to diagnostic methods for determining defects in manufacturing equipment, and more particularly to using models based on physical phenomena to assist the diagnostic methods. [Background technology]

[0002] Manufacturing equipment may be used to produce products by performing one or more manufacturing processes. For example, semiconductor manufacturing equipment may be used to produce substrates using semiconductor manufacturing processes. Products are produced with specific characteristics suitable for a target application. Understanding and controlling the characteristics within the manufacturing chamber aids in the consistent production of products. Summary of the Invention

[0003] The following is a simplified summary of the present disclosure to provide a basic understanding of some aspects of the disclosure. This summary is not an extensive overview of the disclosure. It is not intended to identify key or critical elements of the disclosure, nor is it intended to limit the scope of particular embodiments or claims of the present disclosure. Its sole purpose is to present some concepts of the disclosure in a simplified form as a prelude to the more detailed description that is presented later.

[0004] In one aspect of the present disclosure, a method includes receiving first sensor data generated during a manufacturing process by a sensor associated with a substrate manufacturing chamber. The method further includes receiving simulated sensor data generated by a trained physics-based model. The method further includes determining which one or more components of the manufacturing chamber contribute to a difference between the first sensor data and the simulated sensor data. The method further includes performing corrective action taking the difference into account.

[0005] In another aspect of the present disclosure, a method includes receiving sensor data generated during a manufacturing process by sensors associated with substrate manufacturing equipment. The method further includes receiving simulated sensor data generated by a physical-phenomenon-based model. The method further includes determining at least one difference between the measured sensor data and the simulated sensor data. The method further includes training the physical-phenomenon-based model by adjusting parameters associated with the physical-phenomenon-based model until the at least one difference between the sensor data and the simulated sensor data is less than a threshold.

[0006] In another aspect of the present disclosure, a non-transitory computer-readable storage medium stores instructions that, when executed by a processing device, cause the processing device to perform operations. The operations include receiving first sensor data generated during a manufacturing process by a sensor associated with a substrate manufacturing chamber. The operations further include receiving simulated sensor data generated by a trained physics-based model. The operations further include determining which one or more components of the manufacturing chamber contribute to a difference between the first sensor data and the simulated sensor data. The operations further include performing corrective action in consideration of the difference.

[0007] In the figures of the accompanying drawings, the present disclosure is illustrated by way of example and not by way of limitation. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram illustrating an exemplary system architecture according to certain embodiments. [Figure 2A] FIG. 1 illustrates a dataset generator that generates a dataset for a model, according to certain embodiments. [Figure 2B]FIG. 1 illustrates a dataset generator that generates a dataset for a model, according to certain embodiments. [Figure 3] FIG. 1 is a block diagram illustrating the determination of predictive data, according to certain embodiments. [Figure 4A] 1 is a flow diagram of a method for performing corrective actions in connection with generating predictive data, according to certain embodiments. [Figure 4B] 1 is a flow diagram of a method for performing corrective actions in connection with generating predictive data, according to certain embodiments. [Figure 4C] 1 is a flow diagram of a method for performing corrective actions in connection with generating predictive data, according to certain embodiments. [Figure 4D] 1 is a flow diagram of a method for performing corrective actions in connection with generating predictive data, according to certain embodiments. [Figure 5] 1 is a cross-sectional view of a fabrication chamber in accordance with certain embodiments. [Figure 6] FIG. 1 is a block diagram illustrating a computer system according to certain embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0009] Described herein is a technology related to a digital twin-based diagnostic method for diagnosing problems with manufacturing equipment. The manufacturing equipment may be used to produce products such as substrates (e.g., wafers, semiconductors). The manufacturing equipment often includes a fabrication chamber that isolates the substrates being processed from the environment. The properties of the produced substrates are expected to meet target property values ​​to facilitate a specific function. Manufacturing parameters are selected to attempt to produce substrates that meet the target property values. There are many manufacturing parameters (e.g., hardware parameters, process parameters, etc.) that contribute to the properties of the resulting substrates. The manufacturing system controls such parameters by specifying setpoints for the property values, receiving data from sensors located within the fabrication chamber, and adjusting the manufacturing equipment until the sensor readings match the setpoints. In some cases, the sensors and controllers of several components of the manufacturing equipment form a closed control loop. That is, the manufacturing equipment is automatically adjusted when a difference between the property value measured by the sensor and the setpoint is detected.

[0010] Such closed control loops can be disrupted by, for example, sensor drift, chamber drift, component aging or failure, etc. In conventional systems, such disruption may be detected when the characteristics of produced substrates change over continued use of the manufacturing equipment. Measuring the substrate characteristics (e.g., collecting metrology data) can be time-consuming, may use costly equipment, requires energy, and introduces a delay between a control loop or chamber malfunction and the detection of that malfunction, a subsequent delay between detection and diagnosis, and another delay between diagnosis and correction. Such delays can introduce or increase unplanned manufacturing equipment downtime, or can cause one or more defective substrates to be produced while the problem is detected, resulting in wasted materials, energy, etc.

[0011] Errors in manufactured substrates may be due to one or many causes among a large list of possible causes. Diagnosing the specific cause of a substrate defect can be expensive. In conventional systems, determining the cause of a substrate defect may be performed empirically (e.g., adjusting parameters without any certainty that adjusting the parameters will improve conditions in the manufacturing chamber). Substrates are then produced, metrology is performed, quality is assessed, and parameters are readjusted until the produced substrates reach an acceptable quality level. Not only is the process of achieving acceptable substrates expensive, but the adjusted recipe may also be inefficient (e.g., if drift in performance in the chamber is due to drift in the heating device performance, but the parameter to change to achieve acceptable performance is processing time). In such cases, not only is the process of discovering the cause of problematic substrate production wasteful, but each subsequent manufacturing run may be more expensive in terms of energy, materials, time, or the like than operating under ideal manufacturing conditions would allow.

[0012] Such techniques can shorten the lifespan of some manufactured parts. They can cause components to operate in less than ideal conditions. Over time, such operation can cause premature component failure, which can result in unscheduled downtime of manufacturing equipment to replace the components.

[0013] The disclosed methods and devices address at least one or more of these deficiencies of conventional solutions. In some embodiments, a physical phenomenon-based digital twin model is used to model the manufacturing chamber. In some embodiments, a hybrid methodology is used in which the gas dynamics within the chamber are modeled using physical phenomenon-based correlations and data-driven techniques to calibrate the chamber model for chamber matching.

[0014] A corresponding model of the chamber can be constructed by solving equations for physical phenomena (e.g., heat transfer, fluid dynamics, etc.) associated with operation within the fabrication chamber. The physics-based model may include digital sensors that provide simulated readings corresponding to readings provided by physical sensors within the fabrication chamber. A physics-based model constructed in this manner is still likely to be somewhat inaccurate due to calculation errors, equipment manufacturing tolerances, equipment aging / drift, etc. Such a physics-based model may be trained by providing the model with input data (e.g., manufacturing parameters) and sensor data from the fabrication chamber that corresponds to the physics-based digital twin model. The model then uses this input data to attempt to reproduce the provided sensor data. If the simulated sensor data differs from the chamber sensor data, processing logic may adjust some parameters of the model to more closely align the model with the data provided from the fabrication chamber. This process may be repeated until the physics-based model is able to adequately reproduce the sensor data within the fabrication chamber.

[0015] After training the physics-based digital twin model, input data (e.g., manufacturing parameters) from a subsequent manufacturing run may be input into the physics-based model. The output (e.g., simulated sensor data) from the physics-based model may then be compared to sensor data collected from the manufacturing chamber during the manufacturing run. If the difference between the simulated sensor data and the measured sensor data is insignificant (e.g., within a threshold value that is zero), the manufacturing chamber components may be considered to be in service condition. If the difference between the simulated sensor data and the measured sensor data is significant, the manufacturing chamber components may be considered to be defective, damaged, or the like. Additional investigation may then be initiated to isolate the components contributing to the chamber degradation. Corrective action may be taken to return the manufacturing chamber to its previous condition.

[0016] In some embodiments, aspects of the present disclosure may be performed online, i.e., in parallel with the substrate manufacturing process. Although it may take time to update process parameters using metrology data, the methods disclosed herein may be performed during substrate manufacturing. Obtaining predictive or corrective data may take several seconds of processing time, during which time the completed substrate may be moved out of the manufacturing chamber, work may be started on a new substrate, the process recipe may be adjusted, and the next substrate may be processed under ideal conditions. In some embodiments, adjustments may be made during the manufacturing process to further reduce waste.

[0017] In some embodiments, sensor data from manufacturing runs that produced acceptable substrates may also be stored. Additional data indicative of successful runs is then available for comparison with the current sensor data. In some embodiments, data that is not adequately represented by physics-based models may be captured by "golden run" data, i.e., runs that produced acceptable products, such as radio frequency (RF) match data.

[0018] In some embodiments, the output from the physics-based model and the sensor data from the fabrication chamber may be provided as inputs to a trained machine learning model. The trained machine learning model may provide as output a list of components contributing to the difference between the simulated sensor data and the measured sensor data. The trained machine learning model may further provide data indicating the degree of contribution of each component. In some embodiments, the trained machine learning model may be provided with sensor data from a golden run. The trained machine learning model may use the golden run data and the current sensor data, in some embodiments, along with simulated sensor data, to provide an output indicating the contribution of the components to the difference between the golden run data, the measured sensor data, and the simulated sensor data.

[0019] Aspects of the present disclosure offer technical advantages over conventional solutions. The present disclosure results in a more efficient substrate manufacturing process with minimized waste. When a problem occurs in a manufacturing chamber, the method of the present disclosure may allow corrective action to be taken before processing of the next wafer begins. In this manner, wasted material, manufacturing chamber time, and energy supplied to the manufacturing process may be minimized. Product quality may be improved because root causes of wafer quality variations and chamber drift may be identified and corrected. The efficiency of producing wafers of acceptable quality may also be improved because process parameters can be more precisely tuned, improving material, energy, and time costs. Worn components may be identified and flagged for replacement or maintenance, reducing unscheduled downtime, the associated costs of expediting replacement parts, and the like.

[0020] In some embodiments, the present disclosure describes a method that includes receiving measured sensor data generated during a manufacturing process by sensors associated with a substrate manufacturing chamber. The method further includes receiving simulated sensor data generated by a trained physics-based model. The method further includes determining which one or more components of the manufacturing equipment contribute to a difference between the measured sensor data and the simulated sensor data. The method further includes performing corrective action in consideration of the difference.

[0021] In some embodiments, this disclosure describes a method for training a physical-phenomenon-based digital twin model, the method including receiving sensor data generated during a manufacturing process by sensors associated with substrate manufacturing equipment. The method further includes receiving simulated sensor data generated by the (untrained) physical-phenomenon-based model. The method further includes determining at least one difference between the sensor data and the simulated sensor data. The method further includes training the physical-phenomenon-based model by adjusting parameters of the physical-phenomenon-based model until a difference between the simulated sensor data and the measured sensor data is substantially zero (e.g., less than a threshold value from zero).

[0022] In some embodiments, the present disclosure describes a non-transitory computer-readable storage medium having stored thereon instructions that, when executed by a processing device, cause the processing device to perform operations. The operations include receiving sensor data generated during a manufacturing process by sensors associated with a substrate manufacturing chamber. The operations further include receiving simulated sensor data generated by a trained physics-based model. The operations further include determining which one or more components of the manufacturing chamber contribute to at least one difference between the simulated sensor data and the measured sensor data. The operations further include performing corrective action in consideration of the difference.

[0023] 1 is a block diagram illustrating an example system 100 (example system architecture) according to certain embodiments. System 100 includes client devices 120, manufacturing equipment 124, sensors 126, measurement equipment 128, a prediction server 112, and a data store 140. Prediction server 112 may be part of a prediction system 110. Prediction system 110 may further include server machines 170 and 180.

[0024] The sensors 126 may provide sensor data 142 related to the manufacturing equipment 124 (e.g., related to the manufacturing equipment 124 producing a corresponding product, such as a substrate). The sensor data 142 may be used for equipment health and / or product health (e.g., product quality). The manufacturing equipment 124 may produce a product according to a recipe or run over a period of time. In some embodiments, the sensor data 142 may include one or more values ​​of temperature (e.g., heater temperature), spacing (SP), pressure, high frequency radio frequency (HFRF), radio frequency (RF) match voltage, RF match current, RF match capacitor position, electrostatic chuck (ESC) voltage, actuator position, current, flow rate, power, voltage, etc. The sensor data 142 may be related to or indicative of manufacturing parameters, such as hardware parameters (e.g., settings or components, e.g., size, type, etc.) of the manufacturing equipment 124 or process parameters of the manufacturing equipment 124. Alternatively or additionally, data related to certain hardware parameters may be stored as manufacturing parameters 150, which may include historical manufacturing parameters 152 and current manufacturing parameters 154. The manufacturing parameters 150 may indicate input settings for manufacturing devices (e.g., heating power, gas flow rates, etc.). The sensor data 142 and / or manufacturing parameters 150 may be provided while the manufacturing equipment 124 is performing a manufacturing process (e.g., equipment readings while processing a product). The sensor data 142 may vary from product to product (e.g., from substrate to substrate). The substrate may have characteristics (e.g., film thickness, film strain, etc.) measured by the metrology equipment 128. Metrology data 160 may be a component of the data store 140.

[0025] In some embodiments, the sensor data 142, the metrology data 164, or the manufacturing parameters 150 may be processed (e.g., by the client device 120 and / or the prediction server 112). Processing the sensor data 142 may include generating features. In some embodiments, the features are patterns (e.g., slope, width, height, peaks, etc.) in the sensor data 142, the metrology data 160, and / or the manufacturing parameters 150, or combinations of values ​​from the sensor data 142, the metrology data, and / or the manufacturing parameters (e.g., power derived from voltage and current, etc.). The sensor data 142 may include features, which may be used by the prediction component 114 to perform signal processing and / or to obtain predicted data 168 for taking corrective action.

[0026] Each instance (e.g., set) of sensor data 142 may correspond to a product (e.g., substrate), a set of manufacturing equipment, a type of substrate produced by the manufacturing equipment, or the like. Similarly, each instance of metrology data 160 and manufacturing parameters 150 may correspond to a product, a set of manufacturing equipment, a type of substrate produced by the manufacturing equipment, or the like. The data store may further store information relating sets of different data types, e.g., information indicating that the set of sensor data, the set of metrology data, and the set of manufacturing parameters all relate to the same product, the same manufacturing equipment, the same type of substrate, etc.

[0027] In some embodiments, a physical phenomenon-based digital twin model may generate simulated sensor data 162. The simulated sensor data 162 may have been processed in any of the ways described above with respect to sensor data 142, such as to generate features, combine values, concatenate data from particular recipes, chambers, or substrates, etc.

[0028] In some embodiments, the prediction system 110 may generate the predicted data 168 using supervised machine learning (e.g., using a supervised data set, the predicted data 168 includes measured data, etc.). In some embodiments, the prediction system 110 may generate the predicted data 168 using semi-supervised learning (e.g., using a semi-supervised data set, the predicted data 168 is a predicted percentage, etc.). In some embodiments, the prediction system 110 may generate the predicted data 168 using unsupervised machine learning (e.g., using an unsupervised data set, clustering, clusters based on the measured data 160, etc.).

[0029] The client devices 120, manufacturing equipment 124, sensors 126, measurement equipment 128, prediction server 112, data store 140, server machine 170, and server machine 180 may be coupled to each other via network 130 to generate predictive data 168 for performing corrective actions.

[0030] In some embodiments, network 130 is a public network that provides client device 120 with access to prediction server 112, data store 140, and other public computing devices. In some embodiments, network 130 is a private network that provides client device 120 with access to manufacturing equipment 124, sensors 126, measurement equipment 128, data store 140, and other private computing devices. Network 130 may include one or more wide area networks (WANs), local area networks (LANs), wired networks (e.g., Ethernet networks), wireless networks (e.g., 802.11 networks or Wi-Fi networks), cellular networks (e.g., Long Term Evolution (LTE) networks), routers, hubs, switches, server computers, cloud computing networks, and / or combinations thereof.

[0031] Client device 120 may include computing devices such as personal computers (PCs), laptops, mobile phones, smartphones, tablet computers, netbook computers, network-connected televisions (“smart TVs”), network-connected media players (e.g., Blu-ray players), set-top boxes, over-the-top (OTT) streaming devices, operator boxes, etc. Client device 120 may include a corrective action component 122. Corrective action component 122 may receive user input of instructions related to manufacturing equipment 124 (e.g., via a graphical user interface (GUI) displayed via client device 120). In some embodiments, corrective action component 122 transmits the instructions to prediction system 110, receives output (e.g., prediction data 168) from prediction system 110, determines corrective action based on the output, and causes the corrective action to be implemented. In some embodiments, the corrective action component 122 obtains sensor data 142 (e.g., current sensor data 146) associated with the manufacturing equipment 124 (e.g., from the data store 140, etc.) and provides the sensor data 142 (e.g., the current sensor data 146) associated with the manufacturing equipment 124 to the prediction system 110. In some embodiments, the corrective action component 122 stores the sensor data 142 in the data store 140, and the prediction server 112 retrieves the sensor data 142 from the data store 140. In some embodiments, the prediction server 112 may store the output of the trained model 190 (e.g., predicted data 168) in the data store 140, and the client device 120 may retrieve the output from the data store 140. In some embodiments, the corrective action component 122 receives corrective action instructions from the prediction system 110 and causes the corrective action to be implemented. Each client device 120 may include an operating system that enables a user to perform one or more of creating, reviewing, or editing data (e.g., instructions related to the manufacturing equipment 124, corrective actions related to the manufacturing equipment 124, etc.).

[0032] In some embodiments, metrology data 160 corresponds to historical characteristic data of a product (e.g., a product generated using historical sensor data 144 and manufacturing parameters associated with historical manufacturing parameters 152), and forecast data 168 relates to predicted characteristic data (e.g., predicted characteristic data of a product that will be produced or that has been produced under conditions recorded by current sensor data 146 and / or current manufacturing parameters 154). In some embodiments, forecast data 168 is predicted metrology data (e.g., hypothetical metrology data) of a product that will be produced or that has been produced according to conditions recorded as current sensor data 146 and / or current manufacturing parameters 154. In some embodiments, forecast data 168 is an indication of anomalies (e.g., an abnormal product, an abnormal component, an abnormal manufacturing equipment 124, an abnormal energy usage, etc.) and one or more causes of those anomalies. In some embodiments, forecast data 168 is an indication of a change or drift over time in certain components of manufacturing equipment 124, sensors 126, measurement equipment 128, and the like. In some embodiments, the predictive data 168 is an end of life indication for a component of manufacturing equipment 124, sensor 126, measurement equipment 128, or the like.

[0033] Performing a manufacturing process that results in a defective product can be costly in terms of time, energy, product, components, manufacturing equipment 124, costs of identifying the defects and discarding the defective product, etc. By inputting sensor data 142 (e.g., manufacturing parameters that are or will be used to manufacture the product) into predictive system 110, receiving output of predictive data 168, and performing corrective action based on predictive data 168, system 100 can have the technical advantage of avoiding the costs of producing, identifying, and discarding defective product.

[0034] Running a manufacturing process that results in a failure of a component of manufacturing equipment 124 can be costly in downtime, damage to the product, damage to the equipment, rush-ordering replacement components, etc. By inputting sensor data 142 (e.g., manufacturing parameters that are being or will be used to manufacture the product), receiving output of predictive data 168, and performing corrective action (e.g., predicted operational maintenance, e.g., replacing, treating, cleaning, etc.) based on the predictive data 168, system 100 may have the technical advantage of avoiding the costs of one or more of unexpected component failures, unscheduled downtime, lost productivity, unexpected equipment failures, product waste, or the like. Component performance, e.g., of manufacturing equipment 124, sensors 126, measurement devices 128, and the like, may be monitored over time to provide an indication of deteriorating components.

[0035] The manufacturing parameters may be less than optimal for producing a product, and producing the product may have costly consequences such as increased resource (e.g., energy, coolant, gas, etc.) consumption, increased time to produce the product, increased component failures, increased amount of defective products, etc. By inputting the sensor data 142 into the trained model 190, receiving the output of the prediction data 168, and performing corrective action (e.g., based on the prediction data 168) to update the manufacturing parameters (e.g., set optimal manufacturing parameters), the system 100 may have the technical advantage of using optimal manufacturing parameters (e.g., hardware parameters, process parameters, optimal design) to avoid the costly consequences of suboptimal manufacturing parameters.

[0036] The corrective action may be related to one or more of Computational Process Control (CPC), Statistical Process Control (SPC) (e.g., SPC on electronic components to determine processes under control, SPC to predict the useful life of components, SPC for comparison to 3 sigma graphs, etc.), Advanced Process Control (APC), model-based process control, preventative operational maintenance, design optimization, manufacturing parameter updates, manufacturing recipe updates, feedback control, machine learning corrections, or the like.

[0037] In some embodiments, the corrective action includes issuing an alert (e.g., an alert to stop or not perform a manufacturing process if the predictive data 168 indicates a predicted anomaly, such as an anomaly in a product, component, or manufacturing equipment 124). In some embodiments, the corrective action includes providing feedback control (e.g., feedback control that alters a manufacturing parameter in response to the predictive data 168 indicating a predicted anomaly). In some embodiments, performing the corrective action includes causing an update to one or more manufacturing parameters to be performed.

[0038] The manufacturing parameters may include hardware parameters (e.g., replacing components, using certain components, replacing processing chips, updating firmware, etc.) and / or process parameters (e.g., temperature, pressure, flow rate, rate, current, voltage, gas flow rate, lift speed, etc.). In some embodiments, the corrective action includes performing preventive operational maintenance (e.g., replacing, treating, cleaning, etc., components of the manufacturing equipment 124). In some embodiments, the corrective action includes performing design optimization (e.g., updating manufacturing parameters, manufacturing processes, manufacturing equipment 124, etc. to optimize the product). In some embodiments, the corrective action includes updating a recipe (e.g., placing the manufacturing equipment 124 in idle mode, sleep mode, warm-up mode, etc.).

[0039] Prediction server 112, server machine 170, and server machine 180 may each include one or more computing devices such as a rack-mounted server, a router computer, a server computer, a personal computer, a mainframe computer, a laptop computer, a tablet computer, a desktop computer, a graphics processing unit (GPU), an accelerator application-specific integrated circuit (ASIC) (e.g., a tensor processing unit (TPU)), etc.

[0040] The prediction server 112 may include a prediction component 114. In some embodiments, the prediction component 114 may receive current sensor data 146 and / or current manufacturing parameters 154 (e.g., received from the client device 120 and retrieved from the data store 140) and generate outputs (e.g., predicted data 168) based on the current data for performing corrective actions related to the manufacturing equipment 124. In some embodiments, the prediction component 114 may use one or more trained models 190 to determine outputs for performing corrective actions based on the current data.

[0041] In some embodiments, model 190 may include a trained physics-based digital twin model. The physics-based model may be capable of solving simultaneous equations describing physical phenomena that may occur within a fabrication chamber, such as equations governing heat flow, energy balance, gas conductance, mass balance, fluid dynamics, or the like. In some embodiments, the physics-based model performs calculations of gas conductance within the fabrication chamber. The trained physics-based model may be provided with fabrication parameters 150. The trained physics-based model may provide as output modeled characteristic values ​​indicative of conditions within the fabrication chamber, corresponding to sensors 126 disposed within the fabrication chamber (e.g., fabrication equipment 124). The output of the physics-based model may be stored in data store 140 as simulated sensor data 162.

[0042] The prediction component 114 of the prediction server 112 may receive output from the trained physics-based digital twin model indicating simulated sensor data. The prediction component 114 may compare the output of the trained physics-based model with the current sensor data 146. Based on this comparison, the prediction component 114 may determine one or more differences between the simulated sensor readings and the actual sensor readings. The physics-based digital twin model may have been trained with data from a healthy or optimal chamber. A chamber having a healthy or optimal condition may be expected to produce measured sensor readings that closely match the simulated sensor readings of the trained digital twin model. The prediction component 114 may determine whether the determined differences are greater than one or more difference thresholds. In one embodiment, multiple difference thresholds are used, with individual difference thresholds associated with each type of sensor reading. Alternatively, a single difference threshold may be used. A difference between the simulated sensor readings from the model and the measured sensor readings from the sensor 126 that is greater than a threshold may indicate a manufacturing equipment component failure, chamber drift, sensor drift, or the like. Differences can be analyzed by other algorithms and models (which in some cases are also included in model 190). Prediction component 114 may also generate prediction data 168 indicating actual conditions in the fabrication chamber, for example, when a sensor is predicted to be providing an inaccurate reading. In one embodiment, the actual and predicted sensor readings are input to a trained machine learning model (e.g., a neural network), which outputs an indication of one or more components that are predicted to have caused or contributed to the difference between the actual and predicted sensor measurements.

[0043] The physical-phenomenon-based model may be trained using historical sensor data 144 and historical manufacturing parameters 152. In some embodiments, while the physical-phenomenon-based model may capture most of the physical phenomena that may occur in the manufacturing chamber prior to training, some aspects may not be described by the physical-phenomenon-based model. Such aspects may include aging or non-ideal components, variations induced by manufacturing tolerances or errors, incomplete descriptions of the chamber, or the like. By providing the physical-phenomenon-based model with historical data, parameters in the model may be adjusted to account for these errors.

[0044] In some embodiments, sensor data 142 may include data collected from sensors 126 (e.g., measured by metrology equipment 128) during a production run that produced an acceptable product. A production run that produced an acceptable product may be referred to as a golden run. Sensor data associated with such a production run may be stored in data store 140 as golden run sensor data 148. Prediction component 114 of prediction server 112 may compare golden run sensor data 148, current sensor data 146, and simulated sensor data 162 (e.g., output by a trained physics-based model) to determine whether a component failure, drift, or the like has occurred. In some embodiments, some or all of these operations may instead be performed by different devices, such as client device 120, server machine 170, server machine 180, etc.

[0045] In some embodiments, the prediction component 114 may determine which components contribute to differences between the simulated sensor data 162, the current sensor data 146, and / or the golden run sensor data 148. This may be done by correlating which sensor differences are likely the result of which types of defects. As an example, if all pressure sensor readings are consistent, but the current sensor data 146 provides different values ​​for the position of an actuator that controls the pressure in the chamber by adjusting gas conductance, it may be inferred that the pressure in the chamber is correct, the pressure sensor in the closed control loop that includes the actuator is operating correctly, and the actuator has drifted somewhat, perhaps due to damage, wear, deposition, corrosion, or the like.

[0046] In some embodiments, prediction component 114 may provide the various sensor data differences to a trained machine learning model (in some cases included in model 190). This trained machine learning model may be configured to receive as input data indicative of the sensor data (e.g., simulated sensor data, current sensor data, preprocessed simulated sensor data, preprocessed current sensor data version, etc.) and to generate as output components that are likely to contribute to the sensor data difference, in some embodiments, along with the degree of contribution of each component to the difference.

[0047] The machine learning model may be trained using historical sensor data 144, and simulated sensor data 162 may be generated using historical manufacturing parameters 152 corresponding to runs that measured the historical sensor data 144. One type of machine learning model that may be used to perform some or all of the above tasks is an artificial neural network, such as a deep neural network. Artificial neural networks generally include a feature representation component with a classifier or recurrent layer that maps features to a desired output space. For example, a convolutional neural network (CNN) hosts multiple layers of convolutional filters. Pooling may be performed to address nonlinearities in lower layers, and above the lower layers, multilayer perceptrons are typically added to map upper-layer features extracted by the convolutional layers to decisions (e.g., classification outputs). Deep learning is a type of machine learning algorithm that performs feature extraction and transformation using a cascade of multiple layers of nonlinear processing units. Each successive layer uses the output from the previous layer as input. Deep neural networks may learn in a supervised (e.g., classification) and / or unsupervised manner (e.g., pattern analysis). Deep neural networks contain a hierarchy of layers, with different layers learning different levels of representation corresponding to different levels of abstraction. In deep learning, each level learns to transform its input data into a slightly more abstract and complex representation. For example, in an image recognition application, the raw input may be a matrix of pixels; a first representation layer may extract the pixels and encode edges; a second layer may construct and encode the edge configuration; a third layer may encode higher-order shapes (e.g., teeth, lips, gums, etc.); and a fourth layer may recognize scanning tasks. In particular, the deep learning process can independently learn which features are optimally placed at which level. The "deep" in "deep learning" refers to the number of layers through which data is transformed. More precisely, deep learning systems have significant credit assignment path (CAP) depth. A CAP is a chain of transformations from input to output.CAP describes the potentially causal connections between inputs and outputs. For feedforward neural networks, the depth of the CAP may be the depth of the network, or the number of hidden layers + 1. For recurrent neural networks, where signals may propagate through layers more than once, the CAP depth is potentially infinite.

[0048] Using a physical phenomenon-based digital twin model with manufacturing equipment offers significant technical advantages over operating manufacturing equipment without such a model. Multiple components (sensors, hardware, processing, etc.) may be coupled together in a closed control loop. If any equipment in the closed control loop has a problem, other components may adjust setpoints, etc., causing readings to revert to target values, and the actual conditions inside the chamber may change. This can lead to inconsistent substrate production, lost productivity, energy, time, and material inefficiencies, premature component failure, increased maintenance costs, etc. In some embodiments, multiple data streams indicative of the same characteristic may be monitored to provide additional evidence of the root cause of shifts in sensor data, chamber conditions, or the like. In some embodiments, a sensor indicative of chamber pressure is monitored. These sensors may include sensors measuring chamber pressure, actuator position of actuators affecting chamber pressure, RF match voltage, RF match current, RF match capacitor position, etc. Some of these characteristics may be measured by more than one sensor. For example, a fabrication chamber may be equipped with multiple pressure sensors, a subset of which are included in a closed control loop, and another subset of which are free to provide measurements of chamber conditions.

[0049] In some embodiments, the predictive component 114 receives the current sensor data 146 and / or the current manufacturing parameters 154, performs signal processing to decompose the current data into sets of current data, provides the sets of current data as inputs to the trained model 190, and obtains output from the trained model 190 that is indicative of predicted data 168. In some embodiments, the predicted data is indicative of metrology data (e.g., predicting substrate quality). In some embodiments, the predicted data is indicative of component health.

[0050] In some embodiments, the various models discussed in connection with model 190 (e.g., physical phenomenon-based digital twin models, predictive machine learning models, etc.) may be combined into one model (e.g., an ensemble model) or may be separate models. Prediction component 114 receives current sensor data 146 and current manufacturing parameters 154, provides that data to trained model 190, and receives information indicating how much certain components in the manufacturing chamber have drifted from their previous performance. Data may be passed bidirectionally between certain separate models included in model 190 and prediction component 114. In some embodiments, some or all of these operations may instead be performed by different devices, such as client device 120, server machine 170, server machine 180, etc. One skilled in the art will understand that variations in data flow, which components perform which processes, which data is provided to which models, and the like, are within the scope of this disclosure.

[0051] Data store 140 may be memory (e.g., random access memory), a drive (e.g., a hard drive, a flash drive), a database system, or another type of component or device capable of storing data. Data store 140 may include multiple storage components (e.g., multiple drives or multiple databases) that may reside across multiple computing devices (e.g., multiple server computers). Data store 140 may store sensor data 142, manufacturing parameters 150, metrology data 160, simulated sensor data 162, and forecast data 168. Sensor data 142 may include historical sensor data 144 and current sensor data 146. Sensor data may include time tracking of sensor data throughout the duration of the manufacturing process, association of data with physical sensors, preprocessed data such as averages and composite data, and data indicative of sensor performance over time (i.e., across many manufacturing processes). Manufacturing parameters 150 and metrology data 160 may include similar features. Historical sensor data 144 and historical manufacturing parameters 152 may be historical data (e.g., at least a portion for training model 190). Current sensor data 146 may be current data (e.g., at least a portion for subsequent input to learning model 190 following historical data) for which predicted data 168 is generated (e.g., to perform corrective action). Simulated sensor data may include data indicative of predicted sensor readings for any of sensors 126 associated with manufacturing equipment 124. Simulated sensor data may include both current and historical data.

[0052] In some embodiments, prediction system 110 further includes server machine 170 and server machine 180. Server machine 170 includes a dataset generator 172 that can generate datasets (e.g., a set of data inputs and a set of target outputs) for training, validating, and / or testing model 190, which in some embodiments includes a digital twin model based on physical phenomena and one or more machine learning models. Some operations of dataset generator 172 are described in detail below with respect to FIGS. 2A-B and 4A. In some embodiments, dataset generator 172 may divide historical data (e.g., historical sensor data 144, historical manufacturing parameters 152, simulated sensor data 162 stored in data store 140) into a training set (e.g., 60 percent of the historical data), a validation set (e.g., 20 percent of the historical data), and a test set (e.g., 20 percent of the historical data). In some embodiments, prediction system 110 generates multiple sets of features (e.g., via prediction component 114). For example, the first set of features may correspond to a first set of types of sensor data (e.g., from a first set of sensors, a first combination of values ​​from the first set of sensors, a first pattern of values ​​from the first set of sensors) corresponding to each of the datasets (e.g., a training set, a validation set, and a test set), and the second set of features may correspond to a second set of types of sensor data (e.g., from a second set of sensors different from the first set of sensors, a second combination of values ​​different from the first combination, a second pattern different from the first pattern) corresponding to each of the datasets.

[0053] Server machine 180 includes a training engine 182, a verification engine 184, a selection engine 185, and / or a test engine 186. Engines (e.g., training engine 182, verification engine 184, selection engine 185, and test engine 186) may refer to hardware (e.g., circuitry, dedicated logic circuitry, programmable logic circuitry, microcode, processing device, etc.), software (e.g., instructions executing on a processing device, general-purpose computer system, or dedicated machine), firmware, microcode, or a combination thereof. Training engine 182 may be capable of training a model 190 using one or more sets of features associated with a training set from dataset generator 172. Training engine 182 may generate multiple trained models 190, where each trained model 190 corresponds to a different set of features of the training set (e.g., sensor data from a different set of sensors). For example, a first trained model may be trained using all features (e.g., X1-X5), a second trained model may be trained using a first subset of features (e.g., X1, X2, and X4), and a third trained model may be trained using a second subset of features (e.g., X1, X3, X4, and X5), where the second subset of features may overlap with the first subset of features. The dataset generator 172 may receive the output of a trained model (e.g., a digital twin model based on a physical phenomenon), assemble the data into training, validation, and testing datasets, and use those datasets to train a second model (e.g., a machine learning model configured to output predictive data, corrective actions, etc.).

[0054] The validation engine 184 may be capable of validating the trained models 190 using a corresponding set of validation set features from the dataset generator 172. For example, a first trained model 190 trained using a first set of training set features may be validated using a first set of validation set features. The validation engine 184 may determine the accuracy of each of the trained models 190 based on the corresponding set of validation set features. The validation engine 184 may discard trained models 190 with accuracies that do not meet a threshold accuracy. In some embodiments, the selection engine 185 may be capable of selecting one or more trained models 190 with accuracies that meet a threshold accuracy. In some embodiments, the selection engine 185 may be capable of selecting the trained model 190 with the highest accuracy among the trained models 190.

[0055] The testing engine 186 may be capable of testing the trained models 190 using a corresponding set of test set features from the dataset generator 172. For example, a first trained model 190 trained using a first set of training set features may be tested using a first set of test set features. The testing engine 186 may determine the trained model 190 with the highest accuracy among all of the trained models based on the test set.

[0056] In the case of a machine learning model (e.g., a model that takes as input data from sensors 126 and simulated sensor data generated by a digital twin model based on physical phenomena), model 190 may refer to a model artifact generated by training engine 182 using a training set that includes data inputs and corresponding target outputs (correct answers for each corresponding training input). Patterns in the dataset that map the data inputs to the target outputs (correct answers) can be found, and machine learning model 190 is provided with a mapping that captures these patterns. Machine learning model 190 may use one or more of support vector machines (SVMs), radial basis functions (RBFs), clustering, supervised machine learning, semi-supervised machine learning, unsupervised machine learning, k-nearest neighbor algorithms (k-NNs), linear regression, random forests, neural networks (e.g., artificial neural networks), etc.

[0057] The prediction component 114 may provide the current sensor data 146 to the model 190 and may execute the model 190 on the inputs to obtain one or more outputs. The prediction component 114 may be able to determine (e.g., extract) predicted data 168 from the output of the model 190, and from that output, determine (e.g., extract) confidence data indicating the degree of confidence that the predicted data 168 is an accurate predictor of the process associated with the current sensor data 146 and / or input data for products produced or to be produced under the current manufacturing parameters 154 using the manufacturing equipment 124. The prediction component 114 or the corrective action component 122 may use this confidence data to determine whether to cause a corrective action associated with the manufacturing equipment 124 to be performed based on the predicted data 168.

[0058] The confidence data may include or indicate a degree of confidence that the prediction data 168 is an accurate prediction for a product or component associated with at least a portion of the input data. In one example, the confidence is a real number between 0 and 1, inclusive, where 0 indicates no confidence that the prediction data 168 is an accurate prediction for a product processed according to the input data or an accurate prediction for the component health of a component of the manufacturing equipment 124, and 1 indicates absolute confidence that the prediction data 168 accurately predicts a characteristic of a product processed according to the input data or a component health of a component of the manufacturing equipment 124. In response to the confidence data indicating a confidence below a threshold level for a predetermined number of instances (e.g., a percentage of instances, a frequency of instances, a total number of instances, etc.), the prediction component 114 may retrain the trained model 190 (e.g., based on the current sensor data 146, the current manufacturing parameters 154, etc.).

[0059] For purposes of illustration and not limitation, aspects of the present disclosure describe training one or more machine learning models 190 using historical data (e.g., historical sensor data 144, historical manufacturing parameters 152) and inputting current data (e.g., current sensor data 146, current manufacturing parameters 154, and current metrology data 164) into the one or more trained machine learning models to determine predicted data 168. In other embodiments, a heuristic or rule-based model is used (e.g., without using a trained machine learning model) to determine predicted data 168. The predictive component 114 may monitor the historical sensor data 144, historical manufacturing parameters 152, simulated sensor data 162, and metrology data 160. Any of the information described with respect to data input 210 in FIGS. 2A-B may be monitored by or otherwise used in a heuristic or rule-based model.

[0060] In some embodiments, the functionality of client device 120, prediction server 112, server machine 170, and server machine 180 may be provided by fewer machines. For example, in some embodiments, server machines 170 and 180 may be combined into a single machine, and in other embodiments, server machine 170, server machine 180, and prediction server 112 may be combined into a single machine. In some embodiments, client device 120 and prediction server 112 may be combined into a single machine.

[0061] In general, functions described as being performed by client device 120, prediction server 112, server machine 170, and server machine 180 in one embodiment may, in other embodiments, be performed on prediction server 112, where appropriate. Furthermore, functions attributed to particular components may be performed by different or multiple components operating together. For example, in some embodiments, prediction server 112 may determine corrective actions based on prediction data 168. In another example, client device 120 may determine prediction data 168 based on output from a trained machine learning model.

[0062] Additionally, the functionality of a particular component may be performed by different or multiple components operating together. One or more of prediction server 112, server machine 170, or server machine 180 may be accessed as a service offered to other systems or devices through an appropriate application programming interface (API).

[0063] In embodiments, a "user" may be described as a single individual. However, other embodiments of the present disclosure encompass a "user" being an entity controlled by multiple users and / or automated sources. For example, a collection of individual users united as a group of administrators may be considered a "user."

[0064] Embodiments of the present disclosure may be applied to data quality assessment, feature enhancement, model evaluation, virtual metrology (VM), predictive maintenance (PdM), marginal optimization or the like.

[0065] 2A-B illustrate exemplary one or more dataset generators 272 (e.g., dataset generator 172 of FIG. 1) that generate datasets for training, testing, validation, etc. of a model (e.g., model 190 of FIG. 1), according to certain embodiments. Each dataset generator 272 may be part of server machine 170 of FIG. 1. In some embodiments, both dataset generators 272 of FIGS. 2A-B are the same dataset generator. In some embodiments, the dataset generators of each of FIGS. 2A-B are separate dataset generators.

[0066] Referring to FIG. 2A , a system 200A including a dataset generator 272A (e.g., dataset generator 172 in FIG. 1 ) generates a dataset for a physics-based digital twin model (e.g., model 190 in FIG. 1 ). The dataset generator 272A may generate a dataset (e.g., data input 210A) using historical manufacturing parameters 252 (e.g., historical manufacturing parameters 152 in FIG. 1 ). The system 200A may be used to generate datasets for training, testing, and validating the physics-based digital twin model. In some embodiments, the physics-based model is provided with historical sensor data 244A as a target output 220A. The physics-based model may be configured to solve a system of equations that describe physical phenomena that may occur in a manufacturing chamber. Aspects of the process, such as chamber geometry, chamber composition, and gas composition, may be represented in the model by certain parameters. The training process may adjust some or all of these parameters to produce an output from the physics-based model that is consistent with the target output 220A (e.g., to more accurately capture the physical phenomena in the chamber). The system 200A of FIG. 2A shows a data set generator 272A, a data input 210A and a target output 220A.

[0067] 2B , system 200B, including dataset generator 272B (e.g., dataset generator 172 of FIG. 1 ), generates a dataset for a machine learning model, which in some embodiments may be used in combination with a digital twin model based on physical phenomena (e.g., model 190 of FIG. 1 ). Dataset generator 272B may generate the dataset using historical sensor data 244 and simulated sensor data 262. These datasets may be provided as training inputs to a machine learning model, such as model 190 of FIG. 1 . Dataset generator 272B may also provide component contribution data as target outputs to the machine learning model being trained. The component contribution data may be obtained by running experiments introducing known offsets to certain components of manufacturing equipment. The machine learning model may be trained to reproduce the known offsets. In some embodiments, the input data may further include golden run sensor data.

[0068] 2A-B , in some embodiments, dataset generator 272 generates a dataset (e.g., a training set, a validation set, a test set), which includes one or more data inputs 210 (e.g., training inputs, validation inputs, test inputs) and may include one or more target outputs 220 corresponding to the data inputs 210. The dataset may further include mapping data that maps the data inputs 210 to the target outputs 220. The data inputs 210 may also be referred to as “features,” “attributes,” or “information.” In some embodiments, dataset generator 272 may provide the dataset to training engine 182, validation engine 184, or test engine 186, where the dataset is used to train, validate, or test model 190 (e.g., model 190, one of the models included in ensemble model 190, etc.). In some embodiments, model 190 may include a digital twin model based on a physical phenomenon and a model used to analyze the output of the physical phenomenon-based model. In some embodiments, dataset generator 272 may provide inputs to a model based on physical phenomena as training inputs (e.g., manufacturing parameters) and may provide data related to the output of a second model as target outputs (e.g., component contribution data from a machine learning model), and may use these related datasets to train model 190. Some embodiments of generating training sets are further described with respect to FIG. 4A.

[0069] In some embodiments, dataset generator 272 generates data input 210 and does not generate target output 220 to feed to an unsupervised machine learning model (e.g., dataset generator 272A generates set of historical measurement data 262A-262Z as data input 210A). In some embodiments, dataset generator 272 generates data input 210 and target output 220 (e.g., to train an unsupervised or semi-supervised model). In some embodiments, data input 210 may include one or more sets of data. As an example, system 200B may generate a set of sensor data that may include one or more of sensor data from one or more types of sensors, a combination of sensor data from one or more types of sensors, a pattern of sensor data from one or more types of sensors, a manufacturing parameter from one or more manufacturing parameters, a combination of some manufacturing parameter data and some sensor data, etc.

[0070] In some embodiments, the dataset generator 272 may generate a first data input corresponding to the first set of historical sensor data 244A and / or the first set of historical manufacturing parameters 252A for training, validating, or testing a first machine learning model, and the dataset generator 272 may generate a second data input corresponding to the second set of historical sensor data 244B and / or the second set of historical manufacturing parameters 252B for training, validating, or testing a second machine learning model.

[0071] The data inputs 210 and target outputs 220 for training, validating, or testing a machine learning model may include information for a particular manufacturing chamber (e.g., of a particular piece of substrate manufacturing equipment). For example, the historical manufacturing parameters 252, the historical sensor data 244, the simulated sensor data 262, and the component contribution data 222 may be associated with the same manufacturing chamber.

[0072] In some embodiments, the information used to train the machine learning model may be from a particular type of manufacturing equipment (e.g., manufacturing equipment 124 in FIG. 1 ) having particular characteristics of the manufacturing equipment, and this information may enable the trained model to determine an outcome for a particular group of manufacturing equipment 124 based on inputs for current sensor data (e.g., current sensor data 146) associated with one or more components that share the characteristics of that particular group. In some embodiments, the information used to train the model may be for components from more than one manufacturing equipment, and this information may enable the trained machine learning model to determine an outcome for a component based on inputs from one manufacturing equipment.

[0073] In some embodiments, following generating the dataset and using the dataset to train, validate, or test the machine learning model 190, the model 190 may be further trained, validated, or tested, or adjusted (e.g., weights or parameters, such as parameters of a physically based model or connection weights of a neural network, associated with the input data of the model 190 may be adjusted).

[0074] 3 is a block diagram illustrating a system 300 for generating output data (e.g., predicted data 168 of FIG. 1 ), according to some embodiments. In some embodiments, system 300 may be used in conjunction with a model based on physical phenomena (e.g., model 190 of FIG. 1 ) to determine simulated sensor data, which may inform the execution of corrective actions (e.g., corrective actions may be executed taking into account the simulated sensor data). In some embodiments, a system similar to system 300 may be used in conjunction with a machine learning model to determine corrective actions related to manufacturing equipment.

[0075] At block 310, system 300 (e.g., a component of forecasting system 110 of FIG. 1 ) performs data partitioning (e.g., via dataset generator 172 of server machine 170 of FIG. 1 ) of historical data 364 (e.g., historical manufacturing parameters 152 and historical sensor data 144 for model 190 of FIG. 1 ) to generate training set 302, validation set 304, and test set 306. For example, the training set may be 60% of the historical data, the validation set may be 20% of the historical data, and the test set may be 20% of the historical data.

[0076] The generation of training set 302, validation set 304, and test set 306 can be tailored to a particular application. For example, the training set can be 60% of the historical data, the validation set can be 20% of the historical data, and the test set can be 20% of the historical data. System 300 may generate multiple sets of features for each of the training, validation, and test sets. For example, if historical data 364 includes sensor data from 20 sensors (e.g., sensor 126 in FIG. 1 ) and features derived from 10 manufacturing parameters (e.g., manufacturing parameters corresponding to the sensor data from the 20 sensors), the sensor data may be divided into a first set of features including sensors 1-10 and a second set of features including sensors 11-20. Furthermore, the manufacturing parameters may be divided into multiple sets, e.g., a first set of manufacturing parameters including parameters 1-5 and a second set of manufacturing parameters including parameters 6-10. The target inputs may be divided into multiple sets, the target outputs may be divided into multiple sets, both the target inputs and the target outputs may be divided into multiple sets, or neither the target inputs nor the target outputs may be divided into multiple sets. Multiple models may be trained on different sets of data.

[0077] At block 312, the system 300 performs model training (e.g., via the training engine 182 of FIG. 1 ) using the training set 302. Training machine learning models and / or models based on physical phenomena (e.g., digital twins) may be achieved with supervised learning methods, which involve feeding a training dataset consisting of labeled inputs through the model, observing its outputs, defining an error (by measuring the difference between the output and the label values), and tuning the model's weights using techniques such as deep gradient descent and backpropagation to minimize the error. In many applications, repeating this process across many labeled inputs in the training dataset yields a model that can generate correct outputs when presented with inputs that differ from those present in the training dataset.

[0078] For each training data item in the training data set, the training data item may be input to a model (e.g., a model based on physical phenomena). The model may then process the input training data item (e.g., a process recipe from a historical golden run) to generate an output. The output may include, for example, a plurality of simulated sensor readings. The output may be compared to the label of the training data item (e.g., the actual measured sensor reading).

[0079] Processing logic may then compare the generated output (e.g., sensor reading) with the label (e.g., actual sensor reading) included in the training data item. Processing logic determines an error (i.e., classification error) based on the difference between the output and the label. Processing logic adjusts one or more weights and / or values ​​of the model based on this error.

[0080] When training a neural network, an error term or delta may be determined for each node of the artificial neural network. Based on this error, the artificial neural network adjusts one or more of its parameters (weights for one or more inputs of a node) for one or more of its nodes. The parameters may be updated in a backpropagation manner, with the nodes in the top layer updated first, followed by the nodes in the next layer, and so on. An artificial neural network includes multiple layers of "neurons," each of which receives as input values ​​from neurons in the previous layer. The parameters for each neuron include weights associated with the values ​​received from each of the neurons in the previous layer. Adjusting the parameters may therefore include adjusting the weights assigned to each of the inputs to one or more neurons in one or more layers of the artificial neural network.

[0081] System 300 may train multiple models using multiple sets of features from training set 302 (e.g., a first set of features from training set 302, a second set of features from training set 302, etc.). For example, system 300 may train a model to generate a first trained model using a first set of features in the training set (e.g., sensor data from sensors 1-10) and to generate a second trained model using a second set of features in the training set (e.g., sensor data from sensors 11-20). In some embodiments, the first trained model and the second trained model may be combined to generate a third trained model (e.g., which may, alone, be a better predictor than either the first or second trained model). In some embodiments, the sets of features used in comparing models may overlap (e.g., a first set of features may be sensor data from sensors 1-15 and a second set of features may be sensor data from sensors 5-20). In some embodiments, hundreds of models may be generated, including models with various permutations of features and combinations of models.

[0082] At block 314, the system 300 performs model validation (e.g., via validation engine 184 of FIG. 1 ) using the validation set 304. The system 300 may validate each of the trained models using a corresponding set of features in the validation set 304. For example, the system 300 may validate a first trained model using a first set of features in the validation set (e.g., sensor data from sensors 1-10) and a second trained model using a second set of features in the validation set (e.g., sensor data from sensors 11-20). In some embodiments, the system 300 may validate hundreds of models (e.g., models with various permutations of features, combinations of models, etc.) generated at block 312. At block 314, the system 300 may determine the accuracy of each of the one or more trained models (e.g., via model validation) and may determine whether one or more of the trained models have an accuracy that meets a threshold accuracy. In response to determining that none of the trained models have an accuracy that meets the threshold accuracy, flow returns to block 312, where system 300 performs model training using a different set of features from the training set. In response to determining that one or more of the trained models have an accuracy that meets the threshold accuracy, flow proceeds to block 316. System 300 may discard trained models that have an accuracy lower than the threshold accuracy (e.g., based on a validation set).

[0083] At block 316, the system 300 performs model selection (e.g., via selection engine 185 of FIG. 1 ) to determine which model of the one or more trained models that meet the threshold accuracy has the highest accuracy (e.g., selected model 308 based on the check of block 314). In response to determining that two or more models of the trained models that meet the threshold accuracy have the same accuracy, flow may return to block 312, where the system 300 performs model training to determine the trained model with the highest accuracy using a further refined training set corresponding to the further refined set of features.

[0084] At block 318, the system 300 performs model testing (e.g., via the test engine 186 of FIG. 1 ) using the test set 306 to test the selected model 308. The system 300 may test the first trained model using a first set of features in the test set (e.g., sensor data from sensors 1-10) and determine that the first trained model meets a threshold accuracy (e.g., based on the first set of features of the test set 306). In response to the accuracy of the selected model 308 not meeting the threshold accuracy (e.g., the selected model 308 is too well-fitted to the training set 302 and / or the validation set 304 and cannot be applied to other data sets, such as the test set 306), flow proceeds to block 312, where the system 300 performs model training (e.g., retraining) using a different training set (e.g., sensor data from a different sensor) corresponding to a different set of features. In response to determining that the selected model 308 has an accuracy that meets the threshold accuracy based on the test set 306, flow proceeds to block 320. At least in block 312, the model may learn patterns in the historical data to make predictions, and in block 318, the system 300 may apply the model to the remaining data (e.g., the test set 306) to test the predictions.

[0085] At block 320, the system 300 receives current production parameters 354 (e.g., current production parameters 154 of FIG. 1 ) using the trained model (e.g., selected model 308) and determines (e.g., extracts) simulated sensor data 362 (e.g., simulated sensor data 162 of FIG. 1 ) from the output of the trained model. Corrective action associated with the manufacturing equipment 124 of FIG. 1 may be performed in light of the simulated sensor data 362. In some embodiments, the current production parameters 354 may correspond to the same type of feature in the historical production parameter data. In some embodiments, the current production parameters 354 may correspond to the same type of feature as a subset of the type of feature in the historical production parameter data used to train the selected model 308.

[0086] In some embodiments, current data is received. The current data may include current sensor data 346 (e.g., current sensor data 146 of FIG. 1). The model 308 is retrained based on the current data. In some embodiments, a new model is trained based on the current sensor data 346 and current manufacturing parameters 354.

[0087] In some embodiments, one or more of operations 310-320 may be performed in various orders and / or with other operations not shown and described herein. In some embodiments, one or more of operations 310-320 may not be performed. For example, in some embodiments, one or more of data partitioning of block 310, model validation of block 314, model selection of block 316, or model testing of block 318 may not be performed.

[0088] FIG. 3 illustrates a system configured to train, validate, test, and use a physical phenomenon-based digital twin model. The physical phenomenon-based model is configured to accept manufacturing parameters (e.g., set points provided to manufacturing equipment) as inputs and provide simulated sensor data (e.g., predicted sensor data) as outputs. Other models related to the physical phenomenon-based model may follow a similar data flow during training, validation, testing, and use. In some embodiments, the model may receive simulated sensor data and measured sensor data and output a list of components predicted to contribute to the difference between the two data sets. The model may provide estimates of how much each component contributes to the variation. For example, the model may provide estimates of the contribution of pressure sensors, actuators, etc. The partition, training, validation, selection, test, and use blocks of system 300 may be similarly performed to train a second model using data of a different data type. Further, retraining may be performed using simulated current sensor data and measured current sensor data. In some embodiments, golden run data may also be provided for partitioning, training, validation, selection, testing, and use of machine learning models.

[0089] 4A-D are flow diagrams of methods 400A-D for performing corrective actions in connection with generating prediction data, according to certain embodiments. Methods 400A-D may be performed by processing logic, which may include hardware (e.g., circuitry, dedicated logic circuitry, programmable logic circuitry, microcode, a processing device, etc.), software (e.g., instructions executing on a processing device, a general-purpose computer system, or a dedicated machine), firmware, microcode, or a combination thereof. In some embodiments, methods 400A-D may be performed in part by prediction system 110. Method 400A may be performed in part by prediction system 110 (e.g., server machine 170 and dataset generator 172 in FIG. 1 , dataset generator 272 in FIGS. 2A-B ). Prediction system 110 may use method 400A to generate a dataset for at least one of training, validating, or testing a machine learning model according to embodiments of the present disclosure. Method 400C may be performed by server machine 180 (e.g., training engine 182, etc.). Methods 400B and 400D may be performed by prediction server 112 (e.g., prediction component 114). In some embodiments, a non-transitory storage medium stores instructions that, when executed by a processing device (e.g., a processing device of prediction system 110, a processing device of server machine 180, a processing device of prediction server 112, etc.), cause the processing device to perform one or more of methods 400A-D.

[0090] For ease of explanation, methods 400A-D are illustrated and described as a series of operations. However, operations in accordance with the present disclosure may be performed in various orders and / or simultaneously, and with other operations not shown and described herein. Moreover, not all illustrated operations may be performed to implement methods 400A-D in accordance with the disclosed subject matter. Furthermore, those skilled in the art will understand and appreciate that methods 400A-D may also be represented as a series of interrelated states via a state diagram or events.

[0091] FIG. 4A is a flow diagram of a method 400A for generating a dataset for a model for generating predicted data (eg, predicted data 168 of FIG. 1) according to certain embodiments.

[0092] Referring to FIG. 4A, in some embodiments, at block 401, processing logic performing method 400A initializes a training set T to be an empty set.

[0093] At block 402, processing logic generates a first data input (e.g., a first training input, a first validation input) that may include one or more of sensor data (e.g., historical sensor data 144, simulated sensor data 162 or golden run sensor data 148 in FIG. 1, historical sensor data 244 in FIG. 2B), production parameters (e.g., historical production parameters 152 in FIG. 1), etc. In some embodiments, this first data input may include a first set of features for a type of data, and the second data input may include a second set of features for the type of data (e.g., as described with respect to FIG. 3).

[0094] In some embodiments, at block 403, processing logic generates a first target output for one or more of these data inputs (e.g., a first data input). In some embodiments, this first target output is simulated sensor data (e.g., for a digital twin model based on a physical phenomenon, such as may be included in model 190). In some embodiments, the input data may be in the form of sensor data, and the target output may also be a list of components that are likely to be faulty, as in the case of a machine learning model configured to run in association with a digital twin model based on a physical phenomenon. In some embodiments, no target output is generated (e.g., an unsupervised machine learning model that can group or find correlations in input data without needing to provide a target output).

[0095] At block 404, processing logic optionally generates mapping data indicating an input / output mapping. This input / output mapping (or mapping data) may relate to a data input (e.g., one or more of the data inputs described herein), a target output for the data input, and an association between the data input and the target output. In some embodiments, such as those associated with machine learning models in which no target output is provided, block 404 may not be performed.

[0096] In some embodiments, at block 405, processing logic adds the mapping data generated at block 404 to the dataset T.

[0097] At block 406, processing logic branches based on whether dataset T is sufficient for at least one of training, validation, and / or testing of model 190. If so, execution proceeds to block 407; if not, execution returns to block 402. It should be noted that in some embodiments, whether dataset T is sufficient may be determined simply based on the number of inputs in the dataset, and in some embodiments based on the number of inputs in the dataset that are mapped to outputs, and in some other embodiments, whether dataset T is sufficient may be determined based on one or more other criteria (e.g., measures of diversity of data examples, accuracy, etc.) in addition to or instead of the number of inputs.

[0098] At block 407, processing logic provides dataset T (e.g., to server machine 180) for training, validating, and / or testing machine learning model 190. In some embodiments, dataset T is a training set, and dataset T is provided to training engine 182 of server machine 180 to perform training. In some embodiments, dataset T is a validation set, and dataset T is provided to validation engine 184 of server machine 180 to perform validation. In some embodiments, dataset T is a test set, and dataset T is provided to test engine 186 of server machine 180 to perform testing. For example, in the case of a neural network, input values ​​(e.g., numerical values ​​associated with data inputs 210) of a given input / output mapping are input to the neural network, and output values ​​(e.g., numerical values ​​associated with target outputs 220) of the input / output mapping are stored in output nodes of the neural network. The connection weights of the neural network are then adjusted according to a learning algorithm (e.g., backpropagation, etc.), and the procedure is repeated for the remaining input / output mappings of dataset T. After block 407, the model (e.g., model 190) may be at least one of trained using training engine 182 of server machine 180, validated using validation engine 184 of server machine 180, or tested using test engine 186 of server machine 180. The trained model may be implemented by prediction component 114 (of prediction server 112) to generate prediction data 168 for performing signal processing or for taking corrective actions related to manufacturing equipment 124.

[0099] FIG. 4B is a method 400B for generating data to direct corrective action using a trained physics-based model.

[0100] 4B, at block 410 of method 400B, processing logic receives first sensor data (e.g., current sensor data 146 of FIG. 1) associated with a substrate manufacturing process performed by substrate manufacturing equipment (e.g., a substrate manufacturing chamber). As described above in connection with FIGS. 2-3, the input data of method 400B may have undergone various pre-processing operations, such as grouping, pattern recognition, and combination. The sensor data may include data indicative of temperature, gas flow rate, hardware parameters (e.g., actuator or valve position), pressure, RF match voltage, RF match current, RF match capacitor position, and the like. In some embodiments, the sensor data includes data indicative of (e.g., correlated to) pressure in the chamber.

[0101] At block 412, processing logic receives simulated sensor data generated by a trained physics-based digital twin model. The physics-based model is designed to describe physical phenomena of interest within the fabrication chamber. The physics-based model may provide solutions to simultaneous equations describing conditions within the chamber, such as heat flow, mass balance, and fluid dynamics. The physics-based model may calculate gas conductance within the fabrication chamber.

[0102] At block 414, processing logic optionally receives second sensor data associated with an instance of the manufacturing process that achieved acceptable chamber conditions. Such a manufacturing run may be determined by sensors located within the chamber, by metrology of completed wafers, etc. A manufacturing run that achieved acceptable conditions is called a golden run. The golden run data may be collected from sensors associated with the same manufacturing equipment as the sensors that generated the first sensor data.

[0103] At block 416, processing logic determines which components of the fabrication chamber contribute to the difference between two or more of the first sensor data, the second sensor data, and the simulated sensor data. The processing device may compare two sets of sensor data (e.g., the simulated sensor data and the first sensor data) and determine whether there is a difference greater than a threshold. In some embodiments, sensor data associated with a steady-state condition is used. In some embodiments, processing logic may use a comparison of sensor data values ​​between several data sets to determine components that may be drifting or failing. Such a process is described in more detail with respect to FIG. 4D . In some embodiments, determining which components contribute to the difference between the sensor data includes providing data (e.g., the sensor data, the difference between the sensor data, other processing of the sensor data) to a trained machine learning model. In some embodiments, the trained machine learning model may provide a list of components along with their relative contributions to the difference between the sensor data sets.

[0104] At block 418, corrective action is performed considering the difference between the sets of sensor data. The corrective action may be associated with a component of manufacturing equipment associated with the sensors providing the first and second sensor data. The corrective action may include scheduling maintenance on the component of manufacturing equipment, such as maintenance on a part identified as contributing to the difference between the two or more sets of sensor data. The maintenance may be corrective or preventative. The corrective action may include updating a manufacturing recipe, and in some embodiments, may include updating the manufacturing recipe to account for drifting or aging components. The corrective action may include creating or updating an offset table. The offset table may contain information indicating the performance of the component of manufacturing equipment. For example, the method of the present disclosure may reveal inaccurate sensor readings. The offset table may be maintained and referenced to correlate sensor readings to true chamber conditions. The offset table may contain values ​​for sensor offsets, including pressure sensor offsets, temperature sensor offsets, flow sensor offsets, etc. The offset table may also contain values ​​for hardware sensor offsets, including position data for actuators, valves, etc. The offset table may include offset values ​​for actuators used to control pressure in the fabrication chamber. The offset values ​​in the offset table may adjust sensor readings received from sensors associated with the fabrication equipment or set points provided to components of the fabrication equipment.

[0105] FIG. 4C is a method 400C for training a digital twin model based on physical phenomena, according to certain embodiments.

[0106] 4C, at block 420 of method 400C, processing logic receives sensor data generated by sensors associated with substrate manufacturing equipment. This data is generated during the manufacturing process. In some embodiments, the sensor data associated with method 400C may have undergone pre-processing. The sensor data may take various forms and may be directed to various characteristics within the manufacturing chamber and may be within the scope of the present disclosure.

[0107] At block 422, processing logic receives simulated sensor data generated by a physics-based model. The physics-based model may be designed to describe physical phenomena within the chamber. In some embodiments, the physics-based model calculates gas conductance within the chamber and generates simulated sensor data indicative of chamber pressure. In some cases, the user-initiated physics-based model may not fully describe the fabrication chamber. There may be differences between the modeled behavior and the behavior of the physical device. The differences may be due to manufacturing tolerances, inaccurate assumptions, incomplete descriptions, aging components, etc.

[0108] At block 424, processing logic determines whether there is at least one difference between the sensor data generated by the sensors associated with the substrate manufacturing equipment and the simulated sensor data generated by the physics-based model. The difference may be greater than a threshold difference. The threshold may be related to manufacturing tolerances, measurement errors, a percentage of the sensor value, a selected level to ensure quality, etc. If no significant difference (e.g., greater than a threshold) is found, the method ends and the training of the physics-based model is successful. If at least one significant difference is found, method 400C proceeds to block 426.

[0109] At block 426, processing logic adjusts parameters associated with the physics-based model. This adjustment may include updating values ​​in a calibration table (e.g., a pressure calibration table), updating parameters of the physics-based model that correspond to physical interactions within the chamber, or the like. After the parameters are adjusted, method 400C returns to block 422 to receive new simulated sensor data from the updated physics-based model. Blocks 424, 426, and 422 are then repeated until the difference between the sensor data generated by the sensors associated with the manufacturing equipment and the simulated sensor data generated by the physics-based model is less than a threshold value. At that point, method 400C ends and the physics-based model is considered trained.

[0110] FIG. 4D is a flow diagram of a method 400D for determining which components of manufacturing equipment are contributing to unexpected sensor readings, according to some embodiments.

[0111] Method 400D is an exemplary method for determining corrective actions associated with manufacturing equipment associated with measuring and maintaining pressure in a manufacturing chamber. Performing functions in a different order or checking different data at different points in the process may return similar results, and such variations are within the scope of this disclosure. In some embodiments, method 400D may be repeated each time a wafer is processed in the manufacturing chamber. This allows for variations in the manufacturing process, corrective actions, etc., while minimizing waste. At block 430, processing logic receives sensor data. The sensor data may include current sensor data (e.g., current sensor data 146 of FIG. 1), golden run sensor data (sensor data generated during an acceptable manufacturing process, e.g., golden run sensor data 148 of FIG. 1), and simulated sensor data (sensor data generated by a model based on physical phenomena, e.g., simulated sensor data 162 of FIG. 1). In some embodiments, the manufacturing chamber may include multiple pressure sensors. In some embodiments, the sensor data indicates the position of a pressure control device, e.g., an actuator. In some embodiments, the sensor data includes sensors that measure highly correlated characteristic values, such as an RF match characteristic value. In some embodiments, data from several sensors is extracted for use in method 400D. In some embodiments, only a subset of data from selected sensors is used. For example, in some embodiments, steady-state data may be extracted from the trace data. Additionally, data related to specific operations of the recipe, such as operations long enough to reach steady state, operations in which RF plasma is used, etc., may be extracted.

[0112] At block 432, it is determined whether there are significant differences between the various sets of sensor data. In this context, a significant difference may indicate that the absolute value of the difference is greater than a threshold. If the data associated with each sensor is within an acceptable range throughout the provided data sets, the wafer run may be considered successful, and flow may return to block 430, where different sensor data associated with the next production run may be provided. If a difference is detected, the method proceeds to block 434. In some embodiments, at block 432, processing logic may compare simulated actuator position data to measured actuator position data.

[0113] At block 434, more specific sensor differences are investigated to help identify components contributing to at least one significant difference detected between the two sets of sensor data. In some embodiments, measurements from two pressure sensors (e.g., manometers) associated with the fabrication chamber are compared. In some embodiments, measurements from a sensor associated with the fabrication chamber that measures an RF match characteristic value may be compared to a value from the same sensor from the golden run data. In some embodiments, the RF match characteristic value may be voltage, current, or capacitor position. In some embodiments, combinations may be monitored and compared. In some embodiments, if both conditions are met, i.e., if the two pressure sensors do not match and the golden run RF match data does not match the current RF match sensor data, there is a high probability that the pressure sensor is defective. Method 400D proceeds to block 436. If both conditions are not met, there is a high probability that the actuator is defective. Method 400D proceeds to block 446.

[0114] Following method 400D, assuming a sensor defect is predicted, a sensor offset estimation algorithm is run for recipe operations for which sensor data is available at block 436. Possible conditions and patterns of sensor data for use in method 400D are discussed in connection with block 430 above. The algorithm may consider sensor data for various sensors from the sensor data sets, calculated previous offsets, etc., and may estimate an appropriate offset or offset function for at least one sensor. If the sensor offset is consistent throughout the recipe operations, it may be concluded that the sensor offset is the cause of the difference between the sensor data sets, and method 400D ends at block 438. Corrective action may then be taken to account for the sensor offset, including updating an offset table, adjusting recipe operations, scheduling maintenance, etc. If the sensor offset is inconsistent, method 400D proceeds to block 440.

[0115] At block 440, processing logic executes a sensor gain estimation algorithm. The sensor gain estimation algorithm may include features similar to the sensor offset estimation algorithm discussed in connection with block 436. If the sensor gain is consistent throughout the recipe runs, there is a high probability that the defect is sensor gain related. Method 400D then ends at block 442. Corrective action may be taken to account for this sensor gain defect. If the sensor gain is inconsistent throughout the recipe runs, there may be a high probability that a combination of defects exists, and method 400D ends at block 444. Additional processing may be performed to determine which combination of components is responsible for the difference between the sensor data sets, and corrective action may be taken to account for that combination of components.

[0116] Returning to block 434, in some embodiments, if no significant difference between the pressure sensor data and the RF match data is detected, method 400D proceeds to block 446. At block 436, an actuator offset algorithm is run for recipe operations for which sensor data is available. This algorithm may consider sensor data from various sensors, different sensor data sets (e.g., simulated data sets, golden run data sets, etc.), previously calculated offsets, etc., and may estimate an appropriate offset or offset function for at least one actuator. If the actuator offset is consistent throughout the recipe operations, it may be concluded that the actuator offset is the cause of the difference between the sensor data sets, and method 400D ends at block 448. Corrective action may then be taken to account for the actuator offset, including updating an offset table, adjusting recipe operations, scheduling maintenance, etc. If the actuator offset is inconsistent, method 400D proceeds to block 450.

[0117] At block 450, processing logic executes an actuator gain estimation algorithm. The actuator gain estimation algorithm may include similar features as the actuator offset estimation algorithm. If the actuator gain is consistent throughout the recipe operations, there is a high probability that the defect is actuator gain-related. Method 400D then ends at block 452. Corrective action may be taken to account for this actuator gain defect. If the actuator gain is inconsistent throughout the recipe operations, there may be a high probability that a combination of defects exists, and method 400D ends at block 444. Additional processing may be performed to determine which combination of components is responsible for the difference between the sensor data sets. In some embodiments, a trained machine learning model may be provided with data indicative of the sensor data. Data may be provided to the machine learning model at many different stages of method 400D. For example, each endpoint of method 400D may correspond to a separate trained machine learning model, and data is provided to the trained machine learning model when method 400D reaches its endpoint. In another embodiment, the sensor data may be provided to a trained machine learning model at a time corresponding to an earlier block in Figure 4D, and the trained machine learning model may perform some or all of the components of method 400D. Method 400D is an example of a data flow that can provide information regarding which components are responsible for differences in sensor data associated with a manufacturing run. Determining differences between sensors indicating other characteristic values ​​at other times in this method may allow similar information to be concluded, and such variations are within the scope of this disclosure.

[0118] FIG. 5 illustrates a cross-sectional view of a fabrication chamber 500 (e.g., a semiconductor wafer fabrication chamber) according to some embodiments of the present disclosure. The fabrication chamber 500 may be one or more of an etch chamber (e.g., a plasma etch chamber), a deposition chamber (including an atomic layer deposition chamber, a chemical vapor deposition chamber, a physical vapor deposition chamber, or plasma-enhanced versions thereof), an anneal chamber, or the like. For example, the fabrication chamber 500 may be a chamber for a plasma etcher, a chamber for a plasma cleaner, etc. Example chamber components may include a substrate support assembly 504, an electrostatic chuck, a ring (e.g., a process kit ring), chamber walls, a base, a showerhead 506, a gas distribution plate, a liner, a liner kit, a shield, a plasma screen, a flow balancer, a cooling base, a chamber viewport, a chamber lid, a nozzle, etc.

[0119] In one embodiment, the fabrication chamber 500 includes a chamber body 508 and a showerhead 506 that enclose an interior volume 510. In some chambers, a lid and nozzle may be used instead of the showerhead 506. The chamber body 508 may be constructed from aluminum, stainless steel, or other suitable material. The chamber body 508 generally includes sidewalls 512 and a bottom 514.

[0120] An exhaust port 516 may be defined in the chamber body 508 and may couple the interior volume 510 to a pumping system 518. The pumping system 518 may include one or more pumps and valves utilized to evacuate the interior volume 510 of the fabrication chamber 500 and to regulate the pressure of the interior volume 510 of the fabrication chamber 500. An actuator may be located at or near the exhaust port 516 to control the gas flow rate from the chamber and / or the pressure within the chamber.

[0121] The showerhead 506 may be supported on a sidewall 512 of the chamber body 508 or on the top of the chamber body. The showerhead 506 (or a lid in some embodiments) may be open to allow access to the interior volume 510 of the fabrication chamber 500, and when closed, the showerhead 506 (or the lid in some embodiments) may provide a seal for the fabrication chamber 500. A gas panel 520 may be coupled to the fabrication chamber 500 to supply process or cleaning gases to the interior volume 510 through the showerhead 506 (or the lid and nozzle). The showerhead 506 may include multiple gas supply holes on its entire surface. Examples of process gases that may be used to process substrates in the fabrication chamber 500 include halogen-containing gases, such as C2F6, SF6, SiCl4, HBr, NF3, CF4, CHF3, F2, Cl2, CCl4, BCl3, and SiF4, among others, as well as other gases such as O2 or N2O. Examples of carrier gases include N2, He, Ar, and other gases that are inert (eg, non-reactive) to the process gas.

[0122] The substrate support assembly 504 is disposed below the showerhead 506 within the interior volume 510 of the fabrication chamber 500. In some embodiments, the substrate support assembly 504 includes a susceptor 522 and a shaft 524. The substrate support assembly 504 supports a substrate during processing. In some embodiments, the fabrication chamber 500 further includes one or more heating devices 526 and a reflector 528.

[0123] In some embodiments, the showerhead 506 is configured to generate a plasma via RF discharge. Maximum power delivery relies on matching impedance between the RF source and the plasma. Impedance matching may be performed by a closed-loop control system. Sensors measuring characteristics related to RF impedance match (RF match) may be monitored. The impedance within the fabrication chamber 500 is highly correlated with the chamber pressure. Monitoring characteristics related to RF impedance match (e.g., RF match voltage, RF match current, RF match capacitor position) may provide insight into the pressure inside the fabrication chamber 500.

[0124] Additional sensors are used to measure additional characteristics within fabrication chamber 500. Sensors associated with fabrication chamber 500 may include temperature sensors, pressure sensors (in some embodiments, there may be at least two pressure sensors), flow sensors, etc. By utilizing a physics-based digital twin model of fabrication chamber 500, failed components can be identified and corrective action can be taken.

[0125] In one embodiment of the present disclosure, the chamber includes two pressure sensors (one sensor is included in a closed pressure control loop and the other sensor measures conditions within the chamber), a sensor measuring the position of the actuator that regulates the chamber pressure, and a sensor measuring the voltage used to achieve RF impedance matching. This arrangement of sensors allows patterns in sensor measurements to indicate defects in fabrication chamber components. For example, an unexpected value for the actuator position (e.g., different from simulated or golden run data) may be caused by several components. Trying to empirically discover the cause can be expensive. However, if the readings of the free pressure sensor and the RF match voltage sensor are consistent with the simulated and / or golden run data, the actuator is behaving unexpectedly, but the chamber pressure is likely set correctly. The actuator may be behaving in an unexpected way due to deposition, wear, damage, or the like, and it may be prudent to schedule actuator maintenance. This and other situations may be understood using simulated sensor data generated by trained, physics-based models.

[0126] 6 is a block diagram illustrating a computer system 600 according to certain embodiments. In some embodiments, computer system 600 may be connected to other computer systems (e.g., via a network such as a local area network (LAN), an intranet, an extranet, or the Internet). Computer system 600 may operate in the capacity of a server or a client computer in a client-server environment, or as a peer computer in a peer-to-peer or distributed network environment. Computer system 600 may be provided by a personal computer (PC), a tablet PC, a set-top box (STB), a personal digital assistant (PDA), a mobile phone, a web appliance, a server, a network router, a switch, or a bridge, or any other device capable of executing a set of instructions (sequential or otherwise) that specify actions to be taken by the device. Furthermore, the term "computer" is intended to include a collection of computers that individually or jointly execute a set (or sets) of instructions to perform any one or more of the methods described herein.

[0127] In additional aspects, computer system 600 may include a processing device 602, a volatile memory 604 (e.g., random access memory (RAM)), a non-volatile memory 606 (e.g., read-only memory (ROM) or electrically erasable programmable ROM (EEPROM)), and a data storage device 618, which may communicate with each other via a bus 608.

[0128] The processing device 602 may be provided by one or more processors, such as a general-purpose processor (e.g., a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a microprocessor implementing another type of instruction set, or a microprocessor implementing a combination of instruction set types), or a specialized processor (e.g., an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), or a network processor).

[0129] Computer system 600 may further include a network interface device 622 (e.g., coupled to a network 674). Computer system 600 may further include a video display unit 610 (e.g., an LCD), an alphanumeric input device 612 (e.g., a keyboard), a cursor control device 614 (e.g., a mouse), and a signal generating device 620.

[0130] In some embodiments, data storage device 618 may include a non-transitory computer-readable storage medium 624 (e.g., a non-transitory machine-readable medium) having stored thereon instructions 626 encoding one or more of the methods or functions described herein, including instructions encoding the components of FIG. 1 (e.g., prediction component 114, corrective action component 122, model 190, etc.) and instructions for implementing the methods described herein.

[0131] The instructions 626 may further reside, completely or partially, within the volatile memory 604 and / or within the processing device 602 during execution of the instructions 626 by the computer system 600; thus, the volatile memory 604 and the processing device 602 may also constitute machine-readable storage media.

[0132] While the computer-readable storage medium 624 is shown as a single medium in the illustrative example, the term "computer-readable storage medium" is intended to include a single medium or multiple media (e.g., centralized or distributed databases and / or associated caches and servers) that store one or more sets of executable instructions. The term "computer-readable storage medium" is also intended to include any tangible medium that can store or encode a set of instructions for execution by a computer, causing the computer to perform one or more of the methods described herein. The term "computer-readable storage medium" is intended to include, but is not limited to, solid-state memory, optical media, and magnetic media.

[0133] The methods, components, and features described herein may be implemented by discrete hardware components or may be integrated into the functionality of other hardware components, such as an ASIC, FPGA, DSP, or similar device. Further, the methods, components, and features described herein may be implemented by firmware modules or by functional circuitry within a hardware device. Furthermore, the methods, components, and features described herein may be implemented in any combination of hardware devices and computer program components, or in a computer program.

[0134] Unless otherwise specified, terms such as "receive," "perform," "provide," "acquire," "perform," "access," "determine," "add," "use," "train," "reduce," "generate," "correct," or other similar terms refer to operations and processes performed or implemented by a computer system that manipulate and convert data in computer system registers and memory, represented as physical (electronic) quantities, into other data in the computer system memory or registers, or other such information storage, transmission, or display device, also represented as physical quantities. Furthermore, as used herein, terms such as "first," "second," "third," "fourth," etc., are meant as labels to distinguish between different elements and may not have an ordinal meaning based on the numerical designation of those terms.

[0135] The examples described herein further relate to apparatus for performing the methods described herein. The apparatus may be specially constructed to perform the methods described herein, or may include a general-purpose computer system that is selectively programmed by a computer program stored on the computer system. Such a computer program may be stored on a computer-readable tangible storage medium.

[0136] The methods and illustrative examples described herein are not inherently related to any particular computer or other apparatus. Various general-purpose systems may be used in accordance with the teachings described herein, or it may prove convenient to construct more specialized apparatus to perform the methods described herein and / or each of the individual functions, routines, subroutines, or operations of those methods. Example structures for these various systems are provided in the description above.

[0137] The above description is intended to be illustrative, and not limiting. While the present disclosure has been described with reference to particular illustrative examples and embodiments, it will be recognized that the present disclosure is not limited to the described examples and embodiments. The scope of the present disclosure should be determined with reference to the appended claims, along with the full scope of equivalents to which such claims are entitled.

Claims

1. receiving first sensor data generated during the manufacturing process by a sensor associated with the substrate manufacturing chamber; receiving simulated sensor data generated by the trained physics-based model; determining that an actuator of the substrate fabrication chamber contributes to a difference between the first sensor data and the simulated sensor data by determining that a difference between first data indicative of a chamber pressure and second data indicative of a chamber pressure is less than a first threshold; and performing corrective action in consideration of the difference between the first sensor data and the simulated sensor data. A method comprising:

2. The corrective action is: Updating the offset table; Schedule maintenance, or Update crafting recipes The method of claim 1 , comprising at least one of:

3. The method of claim 2 , wherein the offset table includes one or more of pressure sensor offset values ​​or offset values ​​of actuators used to control pressure in the substrate manufacturing chamber.

4. receiving second sensor data associated with instances of the manufacturing process in which acceptable chamber conditions were achieved; and Determining which components of the substrate manufacturing chamber contribute to a difference between the first sensor data and the second sensor data. The method of claim 1 further comprising:

5. 10. The method of claim 1, wherein the substrate manufacturing chamber includes two pressure sensors, a first pressure sensor and a second pressure sensor, the first pressure sensor being part of a pressure control loop and the second pressure sensor not being part of the pressure control loop.

6. The first sensor data is pressure sensor data, data indicating the position of the actuator; Radio frequency (RF) match voltage data, RF match current data, or RF Match Capacitor Position Data The method of claim 1 , comprising one or more of:

7. 10. The method of claim 1, wherein the physics-based model comprises a physics-based digital twin model, the physics-based digital twin model calculating gas conductance within the substrate fabrication chamber.

8. 10. The method of claim 1, further comprising determining a pressure sensor contribution to a difference between the first sensor data and the simulated sensor data by comparing third data indicative of a chamber pressure to fourth data indicative of a chamber pressure and finding a difference between the third data and the fourth data to be greater than a threshold value from zero.

9. 10. The method of claim 1, wherein determining which one or more components of the substrate manufacturing chamber contribute to the difference between the first sensor data and the simulated sensor data comprises providing the first sensor data and the simulated sensor data to a trained machine learning model, wherein the trained machine learning model outputs, for each component of the one or more components, an estimated contribution of that component to the difference between the first sensor data and the simulated sensor data.

10. receiving first sensor data generated during the manufacturing process by a sensor associated with the substrate manufacturing equipment; receiving first simulated sensor data generated by a model based on physics; determining at least one difference between the first sensor data and the simulated sensor data; and training the physical-phenomenon-based model by adjusting parameters associated with the physical-phenomenon-based model until the at least one difference between the first sensor data and the simulated sensor data is less than a threshold value; the physics-based model providing the second simulated sensor data to an analysis module configured to determine an actuator's contribution to one or more differences between the second simulated sensor data and the first sensor data. method.

11. receiving third simulated sensor data from the trained physics-based model; receiving third sensor data from a sensor associated with the substrate manufacturing equipment; determining which one or more components of a substrate manufacturing chamber contribute to a difference between the third sensor data and the third simulated data; and Taking corrective action to take account of said differences The method of claim 10 further comprising:

12. 11. The method of claim 10, wherein the first sensor data generated by the physics-based model includes pressure sensor data and position data of an actuator used to control pressure in a substrate manufacturing chamber.

13. 11. The method of claim 10, wherein the physics-based model comprises a physics-based digital twin model, the physics-based digital twin model calculating gas conductance within a substrate fabrication chamber.

14. the one or more components of the substrate manufacturing chamber contributing to a difference between the second sensor data and the second simulated data include one or more pressure sensors and one or more actuators; determining the actuator contribution includes comparing first data indicative of the chamber pressure to second data indicative of the chamber pressure and finding a difference therebetween that is within a threshold value that is zero; The method of claim 11.

15. the first and second data indicative of chamber pressure; Sensor data, or Simulated Sensor Data wherein the sensor data or simulated sensor data includes one or more of: pressure sensor data, data indicating the position of the actuator; Radio frequency (RF) match voltage data, RF match current data, or RF Match Capacitor Position Data 15. The method of claim 14, comprising one or more of:

16. A non-transitory computer-readable storage medium having stored thereon instructions that, when executed by a processing device, receiving first sensor data generated during the manufacturing process by a sensor associated with the substrate manufacturing chamber; receiving simulated sensor data generated by the trained physics-based model; determining that an actuator of the substrate fabrication chamber contributes to a difference between the first sensor data and the simulated sensor data by determining that a difference between first data indicative of a chamber pressure and second data indicative of a chamber pressure is less than zero to a first threshold; and performing corrective action in consideration of the difference between the first sensor data and the simulated sensor data. a non-transitory computer-readable storage medium that causes the processing device to perform operations including:

17. The operation further comprises: storing second sensor data associated with instances of the manufacturing process in which acceptable chamber conditions were achieved; and Determining which components of the fabrication chamber contribute to a difference between the first sensor data and the second sensor data.

20. The non-transitory computer-readable storage medium of claim 16, comprising:

18. 17. The non-transitory computer-readable storage medium of claim 16, wherein determining which one or more components of the fabrication chamber contribute to the difference between the first sensor data and the simulated sensor data comprises providing the first sensor data and the simulated sensor data to a trained machine learning model, the trained machine learning model configured to predict the contribution of each component of a plurality of components of the fabrication chamber to the difference between the first sensor data and the simulated sensor data.

Images