Monitoring of plasma generation quality using multi-channel sensor data

Abstract

The method includes the step of obtaining a measurement value of a calibrated feedback control device of a processing chamber using a processing device. Furthermore, the method includes the step of determining a first performance index of the plasma generator of the processing chamber based on the measurement value of the calibrated feedback control device using the processing device. Furthermore, the method includes the step of obtaining a second performance index of the plasma generator from a first sensor of the processing chamber. Furthermore, the method includes the step of providing the first and second performance indexes of the plasma generator to a plasma monitoring module. Furthermore, the method includes obtaining a combined performance index of the plasma generator from the plasma monitoring module. Furthermore, the method includes performing corrective actions considering the combined performance index of the plasma generator.

Description

Technical Field

[0001] The present disclosure relates to a method for monitoring the quality of plasma generation in a processing chamber. More specifically, the present disclosure relates to a method for monitoring the quality of plasma generation in a processing chamber using multi-channel sensor data. Background

[0002] Products are produced by performing one or more manufacturing processes using manufacturing equipment. For example, a substrate can be manufactured by a semiconductor manufacturing process using semiconductor manufacturing equipment. Products are manufactured to have specific characteristics suitable for specific applications. Various models, such as machine learning models, digital twin models, statistical models, and empirical models, are utilized to assist the manufacturing process. Using the models, it is possible to perform monitoring of processing conditions, monitoring of product quality, adjustment of processing conditions, and the like. Summary

[0003] The following is a simplified summary for deepening a basic understanding of some aspects of the present disclosure. This summary is not an extensive summary of the present disclosure. It is not intended to identify key or important elements of the present disclosure, nor to delineate the scope of particular embodiments or the scope of the claims of the present disclosure. The sole purpose of this summary is to present some concepts of the present disclosure in a simplified form as a prelude to the more detailed description that follows.

[0004] In one aspect of the present disclosure, the method includes obtaining measurements of a calibrated feedback control device of a processing chamber using a processing device. The method further includes determining a first performance index of the plasma generator of the processing chamber based on the measurements of the calibrated feedback control device using the processing device. The method further includes obtaining a second performance index of the plasma generator from a first sensor of the processing chamber. The method further includes providing the first and second performance indexes of the plasma generator to a plasma monitoring module. The method further includes obtaining a combined performance index of the plasma generator from the plasma monitoring module. The method further includes taking corrective action in consideration of the combined performance index of the plasma generator.

[0005] In other aspects of the present disclosure, the method includes providing a trained machine learning model with a first performance metric of the plasma generator of a processing chamber. The first performance metric of the plasma generator indicates the power supplied to the plasma of the plasma generator. Furthermore, the method includes providing a second performance metric of the plasma generator to the trained machine learning model. Furthermore, the method includes receiving a composite performance metric of the plasma generator from the trained machine learning model based on the first and second performance metrics. Furthermore, the method includes taking corrective action in consideration of the composite performance metric.

[0006] In other aspects of this disclosure, a non-temporary machine-readable storage medium stores instructions that, when executed, cause a processing device to perform an operation. The operation includes the processing device obtaining measurements of a calibrated feedback control device of a processing chamber. Further, the operation includes the processing device determining a first performance indicator of the plasma generator of the processing chamber based on the measurements of the calibrated feedback control device. Further, the operation includes obtaining a second performance indicator of the plasma generator from a first sensor of the processing chamber. Further, the operation includes providing the first and second performance indicators of the plasma generator to a plasma monitoring module. Further, the operation includes obtaining a combined performance indicator of the plasma generator from the plasma monitoring module. Further, the operation includes taking corrective action in consideration of the combined performance indicator of the plasma generator. [Brief explanation of the drawing]

[0007] This disclosure is illustrated by the figures in the attached drawings and is not limited thereto. [Figure 1] This block diagram shows exemplary system architectures in several embodiments. [Figure 2] A block diagram of a system including an exemplary dataset generator for creating datasets for one or more supervised models is shown, according to several embodiments. [Figure 3] This is a block diagram showing a system for generating output data according to several embodiments. [Figure 4A] This is a flowchart illustrating a method for generating a dataset for a machine learning model, using several embodiments. [Figure 4B] This is a flowchart illustrating a method for utilizing a trained machine learning model to determine plasma generation quality, according to several embodiments. [Figure 4C] This is a flowchart illustrating a method for performing corrective measures related to plasma generation in a processing chamber, according to several embodiments. [Figure 5A] This block diagram shows exemplary schematics for constructing a digital twin model of a plasma processing system according to several embodiments. [Figure 5B] Several embodiments of the logical decision boundary are shown. [Figure 5C] The dependent hard logic boundaries are shown in several embodiments. [Figure 5D] The logistic decision boundary is shown in several embodiments. [Figure 5E] The decision boundaries determined by the trained machine learning model are shown in several embodiments. [Figure 6] This is a block diagram showing computer systems in several embodiments. Detailed explanation

[0008] This specification describes a technique for monitoring the performance of a plasma generation system in a processing chamber using a digital twin model. Manufacturing equipment is used to manufacture products such as substrates (e.g., wafers, semiconductors). Manufacturing equipment may include a manufacturing chamber or processing chamber for isolating the substrate from the environment. The characteristics of the manufactured substrate must meet target values ​​to promote specific functions. Manufacturing parameters are selected to manufacture a substrate that meets the target characteristic values. Many manufacturing parameters (e.g., hardware parameters, process parameters, etc.) contribute to the characteristics of the processed substrate. The manufacturing system can control parameters by specifying set values ​​for the characteristic values, receiving data from sensors placed within the manufacturing chamber, and adjusting the manufacturing equipment until the sensor readings match the set values. In some embodiments, a trained machine learning model is used to improve the performance of the manufacturing equipment.

[0009] Several processing procedures involve plasma processing operations. Plasma generation involves supplying radio frequency (RF) power to one or more electrodes. By generating a strong electric field that changes over time, electrons are detached from atoms in the process gas, generating plasma for the plasma processing operation.

[0010] Once generated, the plasma in the processing chamber is maintained stably by continuously supplying RF power to the plasma within the chamber. However, plasma generation can sometimes become unstable. For example, once the plasma generation process is started, the plasma generation may be unstable until it reaches equilibrium. Furthermore, process conditions such as unstable supply of processing gas may interrupt or destabilize the existing plasma in the processing chamber.

[0011] It may be desirable to monitor the state and quality of the plasma within the processing chamber. It may also be desirable to adjust process operations based on the observed plasma quality within the processing chamber.

[0012] In some systems, plasma quality can be monitored by a spectrometer tuned to measure the wavelength of radiation expected to be emitted from the active plasma. In some systems, plasma quality can be monitored by monitoring the high-frequency power reflected by the plasma generator. Each of these plasma monitoring techniques has its drawbacks. A constant value of plasma emission measured by a spectrometer does not clearly indicate whether plasma generation is stable. Similarly, a constant value of reflected power does not clearly indicate whether plasma generation is stable.

[0013] The methods and systems of this disclosure can address one or more drawbacks of conventional methods. A digital twin model of a plasma generation system can be generated. The digital twin may include a simplified circuit diagram of the plasma generation system. The digital twin can abstract the electrical effects of numerous components associated with the plasma generation apparatus.

[0014] A plasma generation system may include one or more adjustable components. A plasma generation system may include one or more static components (e.g., components with fixed electrical properties). A plasma generation system may include one or more capacitors having adjustable capacitance. In some embodiments, the adjustable components may be included in a feedback system. The adjustable components may be automatically adjusted by the feedback system. The feedback system may be configured to adjust one or more adjustable components of the plasma generation system to adjust the power supplied to the plasma in the processing chamber. The feedback system may be configured to adjust one or more adjustable components of the plasma generation system to maximize the power supply to the plasma.

[0015] The characteristics of the components of a plasma generation system can be calibrated using a digital twin model. Parameters of adjustable components (e.g., the capacitance of an adjustable capacitor) can be monitored. Parameters of the adjustable components of the plasma generation system can be monitored as the system operates under various operating conditions. Gas pressure, gas mixing ratio, RF power, temperature, and other variables affecting plasma generation can be modified during the calibration operation. A calibrated digital twin of the plasma generation system can be generated by calibrating the actual responses of the plasma generation system components under various plasma generation conditions. In some embodiments, details of an abstracted plasma generator or plasma generation component can be determined by additional calibration work, such as supplying RF power without generating plasma.

[0016] The output of the digital twin model can be used to monitor plasma generation. Furthermore, the output of the digital twin model can be combined with other data to determine the quality of the plasma generated in the processing chamber. For example, the output of digital twin plasma generation can be combined with spectrometer data and power reflection data to determine whether the plasma generation system is producing acceptable plasma. Digital twin plasma generation can also be used in combination with one or more other data to determine the possibility of plasma faults (for example, for fault detection).

[0017] The methods and systems of this disclosure offer technical advantages over conventional solutions. A calibrated digital twin model enables the determination of the power supplied to the plasma in the processing chamber. The power supply determination can be made by reading the output of an existing monitoring tool. The power supply determination can be made by reading the output of a calibrated electrical component. The power supply determination can be made without performing voltage and / or current measurements, which may be difficult to perform accurately at the relevant power and frequency. Measurements indicating the power distributed in the plasma enable the determination of plasma quality. Improved accuracy and / or reliability can be achieved by including one or more additional metrics of plasma generation. Improved accuracy in plasma quality determination can be achieved by further including spectroscopic and / or power reflectance measurements. Improved accuracy in determining the likelihood of plasma hazards can be achieved by combining the output of the digital twin with spectroscopic and / or power reflectance measurements. Improved accuracy in determining the likelihood of plasma hazards can be achieved by providing multiple plasma performance metrics to the regression model. Improved accuracy in plasma hazard determination can be achieved by providing multiple plasma performance metrics to the logistic model. Furthermore, by providing a trained machine learning model with multiple plasma performance metrics, the accuracy of plasma fault detection can be improved.

[0018] Improving the accuracy of plasma quality assessment improves process system operation. Accurate plasma quality assessment allows for improved processing recipes. Improved recipes result in reduced processing time, increased processing throughput, reduced energy consumption, reduced material consumption, and reduced impact on the processing environment. Accurate plasma quality assessment allows for adjustments to ongoing processing operations. These adjustments improve process operation results. They increase the success rate of process procedures, reduce the cost of discarding defective products, and reduce the cost of diagnosing defects and their root causes. Accurate plasma quality assessment allows for monitoring the health of the process system. Plasma generation quality can indicate the health of one or more components of the plasma generation system. Evaluating the health of the plasma generation system allows for more appropriate maintenance scheduling, component replacement planning, and reduction of costs associated with unplanned downtime (reduced productivity, emergency repairs, emergency delivery of replacement components, etc.). Accurate measurement of plasma quality over time allows for monitoring of aging and malfunctioning components in the plasma generation system. By accurately measuring plasma quality, unproductive processing steps can be stopped, reducing the time, energy, material, and environmental impact of continuing to execute process recipes.

[0019] Improving the accuracy of plasma quality assessment reduces the risk of damage to processing system components due to plasma system malfunctions or failures. For example, abnormal plasma behavior may manifest as arc discharge, overload of one or more components, etc. By evaluating the quality of plasma generation and taking corrective actions based on this evaluation, damage to one or more components of the substrate processing system can be prevented or mitigated, maintenance and component replacement costs can be reduced, unplanned downtime can be minimized, the percentage of substrates of acceptable quality can be increased, and unwanted arc discharges in the processing chamber can be reduced.

[0020] In one aspect of the present disclosure, the method includes obtaining, by a processing device, a measurement of a calibrated feedback control device of a processing chamber. Further, the method includes determining, by the processing device, a first performance metric of a plasma generation device of the processing chamber based on the measurement of the calibrated feedback control device. Further, the method includes obtaining, from a first sensor of the processing chamber, a second performance metric of the plasma generation device. Further, the method includes providing the first performance metric of the plasma generation device and the second performance metric of the plasma generation device to a plasma monitoring module. Further, the method includes obtaining, from the plasma monitoring module, a composite performance metric of the plasma generation device. Further, the method includes performing a corrective action in consideration of the composite performance metric of the plasma generation device.

[0021] In another aspect of the present disclosure, the method includes providing, to a trained machine learning model, a first performance metric of a plasma generation device of a processing chamber. The first performance metric of the plasma generation device indicates the power supplied to the plasma of the plasma generation device. Further, the method includes providing, to the trained machine learning model, a second performance metric of the plasma generation device. Further, the method includes receiving, from the trained machine learning model, a composite performance metric of the plasma generation device based on the first performance metric and the second performance metric. Further, the method includes performing a corrective action in consideration of the composite performance metric.

[0022] In other aspects of the present disclosure, a non-transitory machine-readable storage medium stores instructions that, when executed, cause a processing device to perform operations. The operations include a processing device obtaining measurements of a calibrated feedback control device of a processing chamber. Further, the operations include a processing device determining a first performance metric of a plasma generation device of the processing chamber based on the measurements of the calibrated feedback control device. Further, the operations include obtaining a second performance metric of the plasma generation device from a first sensor of the processing chamber. Further, the operations include providing the first performance metric of the plasma generation device and the second performance metric of the plasma generation device to a plasma monitoring module. Further, the operations include obtaining a composite performance metric of the plasma generation device from the plasma monitoring module. Further, the operations include performing corrective actions in consideration of the composite performance metric of the plasma generation device.

[0023] FIG. 1 is a block diagram illustrating an exemplary system 100 (exemplary system architecture) according to some embodiments. System 100 includes a client device 120, a manufacturing device 124, sensors 126, a measurement device 128, a prediction server 112, and a data store 140. Prediction server 112 may be part of a prediction system 110.Further, prediction system 110 may include server machines 170 and 180.

[0024] Sensor 126 can provide sensor data 142 related to the manufacturing apparatus 124 (for example, sensor data 142 related to the manufacturing of corresponding products such as substrates by the manufacturing apparatus 124). Sensor data 142 can be used to verify the health of the apparatus and / or the health of the products (e.g., product quality). The manufacturing apparatus 124 can continuously manufacture products according to a recipe or over a period of time. In some embodiments, sensor data 142 may include one or more values ​​such as optical sensor data, spectral data, temperature (e.g., heater temperature), spacing (SP), pressure, high frequency (HFRF), high frequency (RF) matching voltage, RF matching current, RF matching capacitor position, electrostatic chuck (ESC) voltage, actuator position, current, flow rate, power, voltage, etc. Sensor data 142 may include historical sensor data 144 and current sensor data 146. Current sensor data 146 may relate to the product currently being processed, the most recently processed product, the number of most recently processed products, etc. Current sensor data 146 can be used as input to a trained machine learning model to generate, for example, predictive data 168. Historical sensor data 144 may include data stored in relation to previously manufactured products. Historical sensor data 144 may include data related to previous recipes performed by the manufacturing equipment 124. Historical sensor data 144 may include data related to calibration operations. Historical sensor data 144 may include sensor data related to operations performed for the calibration of a digital twin model. Historical sensor data 144 can be used to train a digital twin model (e.g., model 190). Historical sensor data 144 can be used to train a machine learning model (e.g., model 190). Historical sensor data 144 and / or current sensor data 146 may include attribute data (e.g., labels for the manufacturing equipment's ID or design, sensor ID, type and / or location, current fault, service life, etc.).

[0025] Sensor data 142 can be associated with or indicate manufacturing parameters such as hardware parameters and process parameters of the manufacturing apparatus 124. Hardware parameters may include hardware settings, identification information, size, type, etc. Process parameters may include heater settings, gas flow rate settings, RF settings, etc. Data associated with several hardware parameters and / or process parameters can be stored as manufacturing parameters 150, either alternatively or additionally. Manufacturing parameters 150 may include historical manufacturing parameters (e.g., those related to past processing runs) and current manufacturing parameters. Manufacturing parameters 150 may indicate input settings to the manufacturing apparatus (e.g., heater power, gas flow rate, etc.). Sensor data 142 and / or manufacturing parameters 150 can be provided while the manufacturing apparatus 124 is performing the manufacturing process (e.g., measurements of the apparatus while processing a product). Sensor data 142 may differ for each product (e.g., for each substrate). The substrate may have characteristic values ​​(film thickness, film strain, etc.) measured by measuring instrument 128 in a standalone measuring facility. The measurement data 160 may also be a component of the data store 140. The measurement data 160 may include historical measurement data (for example, measurement data related to previously processed products).

[0026] In some embodiments, measurement data 160 can be provided without using standalone measurement equipment. For example, measurement data 160 may include in-situ measurement data, measurement data, or proxy measurement data collected during processing. Measurement data 160 may include integrated measurement data, measurement data, or proxy measurement data collected while the product is in the chamber or under vacuum and not during processing operations. Measurement data 160 may include in-line measurement data collected after the substrate is removed from the vacuum. Measurement data 160 may include further types of measurement data obtained from further types of measurement data generation techniques. Measurement data 160 may include current measurement data (e.g., measurement data related to currently or most recently processed products).

[0027] In some embodiments, sensor data 142, measurement data 160, or manufacturing parameters 150 can be processed (e.g., by a client device 120 and / or a prediction server 112). Processing of sensor data 142 may include the generation of features. In some embodiments, features are patterns (e.g., slope, width, height, peak, etc.) within the sensor data 142, measurement data 160, and / or manufacturing parameters 150. In some embodiments, features may include combinations of numerical values ​​from sensor data 142, measurement data, and / or manufacturing parameters (e.g., power derived from voltage and current, etc.). Sensor data 142 may include features, which can be used to obtain prediction data 168 for signal processing and / or corrective action by the prediction component 114.

[0028] Each instance (e.g., set) of sensor data 142 can correspond to a product (e.g., substrate), a set of manufacturing equipment, a calibration procedure, the type of substrate manufactured by the manufacturing equipment, etc. Similarly, each instance of measurement data 160 and manufacturing parameters 150 can correspond to a product, a set of manufacturing equipment, the type of substrate manufactured by the manufacturing equipment, etc. Furthermore, the data store can store information that associates sets of different data types (e.g., information indicating that the sensor dataset, measurement dataset, and manufacturing parameter set are all associated with the same product, manufacturing equipment, calibration procedure, substrate type, etc.).

[0029] In some embodiments, the prediction system 110 can generate prediction data 168 using supervised machine learning. The prediction data 168 may include the output from a machine learning model trained on labeled data (e.g., sensor data labeled with plasma quality data). In some embodiments, the prediction system 110 can generate prediction data 168 using unsupervised machine learning. The prediction data 168 may include the output from a machine learning model trained on unlabeled data. The output from the unsupervised machine learning model may include clustering results, principal component analysis, anomaly detection, etc. In some embodiments, the prediction system 110 can generate prediction data 168 using semi-supervised learning (e.g., the training data may contain a mixture of labeled and unlabeled data).

[0030] Furthermore, the data store 140 includes calibration data 162. The calibration data may also include data related to the calibration of the digital twin model used to generate the prediction data 168. The calibration data 162 may also include data related to the calibration and / or operation of the physical-based model used to generate the prediction data 168. The calibration data 162 may include calculated values ​​of component characteristics. The calibration data 162 may include correction values ​​of the nominal characteristics of the components of the plasma generation system. The calibration data 162 may include tables, curves, surfaces, etc., of component parameters that depend on one or more variables such as temperature, power supply, and voltage division. The calibration data 162 may include data related to the operation of the digital twin model for determining the prediction data 168.

[0031] The client device 120, manufacturing equipment 124, sensor 126, measuring device 128, prediction server 112, data store 140, server machines 170 and 180 are interconnected via network 130 and can generate prediction data 168 and execute corrective actions. In some embodiments, network 130 can provide access to cloud-based services. Operations performed by the client device 120, prediction system 110, data store 140, etc., can be performed by virtual cloud-based devices.

[0032] In some embodiments, network 130 is a public network providing client devices 120 with access to the prediction server 112, the data store 140, and other public computing devices. In some embodiments, network 130 is a private network providing client devices 120 with access to manufacturing equipment 124, sensors 126, measuring equipment 128, the data store 140, and other private computing devices. Network 130 may include one or more wide area networks (WANs), local area networks (LANs), wired networks (e.g., Ethernet networks), wireless networks (e.g., 802.11 networks or Wi-Fi networks), cellular networks (e.g., Long Term Evolution (LTE) networks), routers, hubs, switches, server computers, cloud computing networks, and / or combinations thereof.

[0033] The client device 120 may include computing devices such as personal computers (PCs), laptops, mobile phones, smartphones, tablet computers, netbooks, network-connected televisions ("smart TVs"), network-connected media players (e.g., Blu-ray players), set-top boxes, over-the-top (OTT) streaming devices, and operator boxes. The client device 120 may include a corrective action component 122. The corrective action component 122 may receive user input of instructions related to the manufacturing apparatus 124 (e.g., via a graphical user interface (GUI) displayed through the client device 120). In some embodiments, the corrective action component 122 sends instructions to a prediction system 110, receives output from the prediction system 110 (e.g., prediction data 168), determines corrective action based on the output, and has the corrective action implemented. In some embodiments, the corrective action component 122 acquires sensor data 142 associated with the manufacturing apparatus 124 (e.g., current sensor data 146) (e.g., from a data store 140, etc.) and provides the sensor data 142 associated with the manufacturing apparatus 124 (e.g., current sensor data 146) to the prediction system 110.

[0034] In some embodiments, the corrective action component 122 can acquire sensor data related to the feedback component of the plasma generation system. The corrective action component 122 can provide the sensor data to the prediction system 110. The corrective action component 122 can receive an output from the prediction system 110 indicating the quality of the plasma generated by the manufacturing apparatus 124. The corrective action component 122 can perform corrective actions based on the output from the prediction system 110.

[0035] In some embodiments, the corrective action component 122 receives instructions for corrective action from the prediction system 110 and implements the corrective action. Each client device 120 may include an operating system that enables the user to perform one or more actions such as generating, displaying, or editing data. The client device 120 can provide instructions related to the manufacturing equipment 124, corrective actions related to the manufacturing equipment 124, etc. The client device 120 can provide instructions for prediction data 168 to the user and provide a graphical user interface for receiving instructions from the user.

[0036] In some embodiments, measurement data 160 corresponds to historical characteristic data of the product, and prediction data 168 relates to characteristic data predicted based on plasma generation quality. In some embodiments, prediction data 168 is or includes prediction measurement data of a product or a manufactured product that is manufactured according to conditions recorded as current sensor data 146, current measurement data, current measurement data and / or current manufacturing parameters. In some embodiments, prediction data 168 is or includes an indicator of any anomaly (e.g., an abnormal product, an abnormal component, an abnormal manufacturing equipment 124, abnormal energy usage, etc.), and additionally includes one or more causes of those anomalies. In some embodiments, prediction data 168 is an indicator of the change or drift over time of components such as manufacturing equipment 124, sensor 126, and measurement device 128. In some embodiments, prediction data 168 is an indicator of the lifespan of components such as manufacturing equipment 124, sensor 126, and measurement device 128. Also, in some embodiments, prediction data 168 is an indicator of the progress of a processing operation in progress (e.g., used for process control).

[0037] Implementing a manufacturing process that produces defective products can incur costs such as time, energy, products, components, manufacturing equipment 124, and costs associated with identifying defects and disposing of defective products. By inputting sensor data 142 (such as sensor data indicating plasma quality) into the prediction system 110, receiving the output of prediction data 168, and performing corrective actions based on the prediction data 168, the system 100 can gain the technical advantage of avoiding the costs associated with manufacturing, identifying, and disposing of defective products.

[0038] Executing a manufacturing process that causes component failure in the manufacturing equipment 124 can result in significant costs, including downtime, product damage, equipment damage, and emergency ordering of replacement parts. By inputting sensor data 142 (e.g., manufacturing parameters used or planned to be used in product manufacturing), measurement data, etc., receiving output of predictive data 168, and performing corrective actions based on the predictive data 168 (e.g., predictive operational maintenance such as component replacement, processing, or cleaning), system 100 can gain the technical advantage of avoiding one or more costs such as unexpected component failure, unplanned downtime, decreased productivity, unexpected equipment failure, and product disposal. Monitoring the performance of components such as the manufacturing equipment 124, sensor 126, and measuring device 128 over time can provide indications of component degradation.

[0039] Manufacturing parameters may not be optimal for product manufacturing, potentially leading to increased costs due to increased resource consumption (energy, coolant, gas, etc.), longer product manufacturing times, increased component failures, and higher defect rates. By inputting sensor data into the prediction system 110, receiving plasma quality indicators, and adjusting process operations based on these indicators, the manufacturing system has the advantage of being able to improve the optimization of manufacturing parameters for processing the target product.

[0040] Executing a manufacturing process with suboptimal plasma generation can increase the environmental impact of the processing procedure. This environmental impact can be exacerbated by increased energy and material consumption due to longer processing times. Furthermore, the environmental impact can be affected by an increased probability of defective products, increased waste, and increased resources used to dispose of defective products. By providing sensor data to the prediction system 110, receiving plasma quality indicators, and implementing corrective actions based on plasma quality, the manufacturing system 100 can enjoy the technical advantage of reducing the environmental impact of the substrate processing process.

[0041] Corrective actions can be associated with one or more of the following: computational process control (CPC), statistical process control (SPC, e.g., SPC for electronic components to determine the process under control, SPC for predicting the lifespan of components, SPC for comparison with a 3-sigma graph, etc.), advanced process control (APC), model-based process control, predictive maintenance, design optimization, updating manufacturing parameters, updating manufacturing recipes, feedback control, and machine learning modifications.

[0042] In some embodiments, corrective action includes providing a warning to the user. The warning may be an alarm that prevents the manufacturing process from being stopped or run if the predictive data 168 indicates a predicted anomaly, such as a product, component, plasma generation system, or manufacturing equipment 124. In some embodiments, a machine learning model is trained to monitor the progress of the process execution (e.g., monitor in-situ sensor data to predict whether the manufacturing process is progressing as expected). In some embodiments, the machine learning model may send an instruction to terminate the process execution when the model determines that the process is complete. In some embodiments, corrective action includes providing feedback control (e.g., modifying manufacturing parameters in response to predictive data 168 indicating a predicted anomaly). In some embodiments, the execution of corrective action includes updating one or more manufacturing parameters. In some embodiments, the execution of corrective action may include retraining the machine learning model associated with the manufacturing equipment 124. In some embodiments, the execution of corrective action may include training a new machine learning model associated with the manufacturing equipment 124.

[0043] The manufacturing parameters 150 may include hardware parameters (e.g., information indicating which components are installed in the manufacturing apparatus 124, information indicating component replacement, information indicating component age, information indicating software version or update, etc.) and / or process parameters (e.g., temperature, pressure, flow rate, speed, current, voltage, gas flow rate, lift speed, etc.). In some embodiments, corrective actions include performing preventive operational maintenance (e.g., replacing, processing, cleaning, etc., components of the manufacturing apparatus 124). In some embodiments, corrective actions include design optimization (e.g., updating manufacturing parameters, manufacturing processes, manufacturing apparatus 124, etc., for product optimization). In some embodiments, corrective actions include updating recipes (e.g., changing the timing of when the manufacturing subsystem enters idle mode or active mode, changing the settings of various property values, etc.).

[0044] Each of the prediction server 112, server machine 170, and server machine 180 may include one or more computing devices such as rack-mount servers, router computers, server computers, personal computers, mainframe computers, laptop computers, tablet computers, desktop computers, graphics processing units (GPUs), and application-specific integrated circuits (ASICs) (e.g., tensor processing units (TPUs)). The operations of the prediction server 112, server machine 170, server machine 180, data store 140, etc., can be powered by cloud computing services, cloud data storage services, etc.

[0045] The prediction server 112 may include a prediction component 114. In some embodiments, the prediction component 114 may receive current sensor data 146 and / or current manufacturing parameters (e.g., received from a client device 120 and retrieved from a data store 140) and generate an output (e.g., prediction data 168) for performing corrective actions related to the manufacturing equipment 124 based on the current data. In some embodiments, the prediction data 168 may include one or more predicted dimensional measurements of a processed product. In some embodiments, the prediction component 114 may use one or more trained machine learning models and / or digital twin models 190 to determine an output for performing corrective actions based on the current data.

[0046] The manufacturing apparatus 124 can be associated with one or more machine learning models (e.g., model 190). The machine learning models associated with the manufacturing apparatus 124 can perform many tasks, such as process control, classification, and performance prediction. The training of model 190 can be performed using data associated with the manufacturing apparatus 124 or data of products processed by the manufacturing apparatus 124, such as sensor data 142 (e.g., collected by sensor 126), manufacturing parameters 150 (e.g., associated with process control of the manufacturing apparatus 124), measurement data 160 (e.g., generated by measurement equipment 128), etc.

[0047] One type of machine learning model that can be used to perform some or all of the tasks described above is an artificial neural network, such as a deep neural network. Generally, an artificial neural network includes a feature representation component with classifier or regression layers that map features to a desired output space. For example, a convolutional neural network (CNN) has multiple convolutional filter layers. Pooling is performed in the lower layers to also handle nonlinearity. Above that, a multilayer perceptron is usually added to map the top layer features extracted by the convolutional layers to a decision (e.g., a classification output).

[0048] A recurrent neural network (RNN) is another type of machine learning model. Recurrent neural network models are designed to interpret a set of inputs that are inherently related to each other, such as time-trace data or sequential data. The output of an RNN perceptron is fed back into the perceptron as input to generate the next output.

[0049] Deep learning is a type of machine learning algorithm that uses a cascading array of nonlinear processing units for feature extraction and transformation. Each layer uses the output of the previous layer as input. Deep neural networks can perform supervised learning (e.g., classification) and / or unsupervised learning (e.g., pattern analysis). Deep neural networks have a hierarchical structure, with each layer learning different levels of representation corresponding to different levels of abstraction. In deep learning, each layer learns to transform the input data into a more abstract and complex representation. For example, in an image recognition application, the raw input is a matrix of pixels, the first representation layer abstracts the pixels and encodes the edges, the second layer synthesizes and encodes the arrangement of the edges, the third layer encodes higher-order shapes (e.g., teeth, lips, gums, etc.), and the fourth layer can perform the role of scanning. Notably, the deep learning process can independently learn which features are best placed at which level. The "deep" in "deep learning" refers to the number of layers to which the data is transformed. More precisely, deep learning systems have a considerable credit allocation path (CAP) depth. A CAP is a chain of transformations from input to output. A CAP describes potentially causal connections between inputs and outputs. In the case of a feedforward neural network, the CAP depth may be the network depth plus the number of hidden layers plus one. In the case of a recurrent neural network, where a signal can propagate through a single layer multiple times, the CAP depth is potentially unlimited.

[0050] Another type of machine learning model that can be used in relation to the manufacturing system 100 is a support vector machine (SVM). The SVM classifies the input into categories of interest. The SVM can determine complex decision boundaries between the classified categories. The SVM can provide sensor data related to the manufacturing apparatus 124 and, based on labeled training data, determine whether the plasma generation is normal or abnormal.

[0051] In some embodiments, the prediction component 114 receives current sensor data 146, performs signal processing to decompose the current data into a current dataset, provides the current dataset as input to a trained model 190, and obtains an output from the trained model 190 showing prediction data 168.

[0052] In some embodiments, the various models described in relation to Model 190 (e.g., supervised machine learning models, unsupervised machine learning models, physically based models, digital twin models, etc.) can be combined into a single model (e.g., an ensemble model) or they can be separate models.

[0053] Data may be exchanged between Model 190 and multiple different models included in the prediction component 114. In some embodiments, some or all of these operations may be performed by different devices such as client device 120, server machine 170, and server machine 180. It will be understood by those skilled in the art that variations in data flow, which components perform which processes, and which data is provided to which models are within the scope of this disclosure.

[0054] The data store 140 may be memory (e.g., random access memory), drives (e.g., hard drives, flash drives), a database system, a cloud-accessible memory system, or other types of components or devices capable of storing data. The data store 140 may include multiple storage components (e.g., multiple drives or multiple databases) spanning multiple computing devices (e.g., multiple server computers). The data store 140 can store sensor data 142, manufacturing parameters 150, measurement data 160, calibration data 162, and prediction data 168.

[0055] Sensor data 142 may include historical sensor data 144 and current sensor data 146. Sensor data may include a time trace of sensor data over the duration of the manufacturing process, correlation with data from physical sensors, pre-processed data such as averages and composite data, and data showing sensor performance over time (e.g., many manufacturing processes). Manufacturing parameters 150 and measurement data 160 may include similar features such as historical measurement data and current measurement data. Historical sensor data 144, historical measurement data, and historical manufacturing parameters may be historical data (e.g., at least a portion of this data can be used to train model 190). Current sensor data 146, current measurement data, and current manufacturing parameters may be current data for generating predictive data 168 (e.g., for taking corrective actions) (e.g., at least a portion of which is input to the learning model 190 following historical data).

[0056] Furthermore, in some embodiments, the prediction system 110 includes server machines 170 and 180. Server machine 170 includes a dataset generator 172 capable of generating datasets (e.g., a set of data inputs and a set of target outputs) for training, validating, and / or testing a model 190 that includes one or more machine learning models. Some operations of the dataset generator 172 are described in detail below with reference to Figures 2 and 4A. In some embodiments, the dataset generator 172 can divide historical data (e.g., historical sensor data 144, historical manufacturing parameters, historical measurement data) into a training set (e.g., 60% of the historical data), a validation set (e.g., 20% of the historical data), and a test set (e.g., 20% of the historical data).

[0057] In some embodiments, the prediction system 110 generates multiple feature sets (for example, via a prediction component 114). For example, a first feature set may correspond to a first type of sensor dataset corresponding to each dataset (e.g., a training set, a validation set, and a test set) (e.g., data from the first sensor set, a first combination of values ​​from the first sensor set, and a first pattern of values ​​from the first sensor set). A second feature set may correspond to a second type of sensor dataset corresponding to each dataset (e.g., data from a second sensor set different from the first sensor set, a second combination of values ​​different from the first combination, and a second pattern different from the first pattern).

[0058] In some embodiments, the machine learning model 190 is provided with historical data as training data. In some embodiments, the digital twin model 190 may be provided with calibration data 162 or sensor data related to the calibration procedure as training / calibration data. The historical data and / or calibration data may be data indicating the performance of the plasma generation system, or may include such data. The type of data provided will vary depending on the application of the machine learning model. For example, the machine learning model can be trained by providing the model with historical sensor data 144 as training input and corresponding measurement data 160 as target output. In some embodiments, a large amount of data may be used to train the model 190. For example, sensor and measurement data from hundreds of substrates may be used. In some embodiments, the amount of data available to train the model 190 may be quite small. For example, the model 190 may be trained to recognize rare events such as equipment failure, or to generate predictions for newly seasoned or maintained chambers.

[0059] The server machine 180 includes a training engine 182, a validation engine 184, a selection engine 185, and / or a test engine 186. An engine (e.g., training engine 182, validation engine 184, selection engine 185, and test engine 186) can mean hardware, software, firmware, microcode, or a combination thereof. Hardware solutions can include circuits, dedicated logic, programmable logic, microcode, processing devices, etc. Software solutions can include processing devices, general-purpose computer systems, or instructions executed on dedicated machines. The training engine 182 can train a model 190 using one or more feature sets associated with a training set from the dataset generator 172. The training engine 182 can generate multiple trained models 190, each corresponding to a different feature set of the training set (e.g., sensor data from different sensor sets). For example, the first trained model may be trained using all features (e.g., X1-X5), the second trained model may be trained using a subset of all features (e.g., X1, X2, X4), and the third trained model may be trained using a subset of features that may partially overlap with the first feature subset (e.g., X1, X3, X4, X5). The dataset generator 172 receives the output of the trained model (e.g., synthetic data 162 from the synthetic data generator 174), collects this data into a training dataset, a validation dataset, and a test dataset, and can use these datasets to train a second model (e.g., a machine learning model configured to output predictive data, corrective actions, etc.).

[0060] The validation engine 184 can validate the trained models 190 using the corresponding feature sets of the validation set from the dataset generator 172. For example, a first trained machine learning model 190 trained using the first feature set of the training set can be validated using the first feature set of the validation set. The validation engine 184 can determine the accuracy of each trained model 190 based on the corresponding feature sets of the validation set. The validation engine 184 can discard trained models 190 that do not meet the threshold accuracy. In some embodiments, the selection engine 185 can select one or more trained models 190 that meet the threshold accuracy. In some embodiments, the selection engine 185 can select the trained model 190 with the highest accuracy among the trained models 190.

[0061] The test engine 186 can test the trained model 190 using the corresponding feature set of the test set provided by the dataset generator 172. For example, a first trained machine learning model 190 trained using the first feature set of the training set can be tested using the first feature set of the test set. Based on the test set, the test engine 186 can determine which trained model 190 has the highest accuracy among all the trained models.

[0062] In the case of a machine learning model, model 190 can refer to the model artifact created by the training engine 182 using a training set that includes data inputs and corresponding target outputs (ground truth for each training input). The dataset contains patterns that map data inputs to target outputs (ground truth), and the machine learning model 190 provides mappings that capture these patterns. One or more machine learning models 190 can be, such as support vector machines (SVMs), radial basis functions (RBFs), clustering, supervised machine learning, semi-supervised machine learning, unsupervised machine learning, k-nearest neighbors (k-NNs), linear regression, random forests, and neural networks (e.g., artificial neural networks, recurrent neural networks).

[0063] In some embodiments, one or more machine learning models 190 can be trained using historical data (e.g., historical sensor data 144). In some embodiments, the models 190 can be trained using calibration data 162, sensor data associated with a calibration procedure, or a combination of historical data and calibration data.

[0064] The prediction component 114 provides current data to the model 190, and the model 190 can be run on that input to obtain one or more outputs. For example, the prediction component 114 provides current sensor data 146 to the model 190, and the model 190 can be run on that input to obtain one or more outputs. The prediction component 114 can determine (e.g., extract) prediction data 168 from the output of the model 190. The prediction component 114 can determine (e.g., extract) confidence data from the output. The confidence data indicates the confidence level that the prediction data 168 is an accurate prediction of the process associated with the input data regarding the product manufactured or to be manufactured using the manufacturing equipment 124 with the current sensor data 146 and / or current manufacturing parameters. The prediction component 114 or the corrective action component 122 can use the confidence data to determine whether to perform a corrective action associated with the manufacturing equipment 124 based on the prediction data 168.

[0065] Confidence data may include or indicate the confidence that the prediction data 168 is an accurate prediction about a product or component related to at least a portion of the input data. In one example, the confidence is a real number between 0 and 1. 0 indicates no confidence that the prediction data 168 is an accurate prediction about the health of a product or component of the manufacturing equipment 124 processed according to the input data. 1 indicates absolute confidence that the prediction data 168 accurately predicts the characteristics of a product or component of the manufacturing equipment 124 processed according to the input data. If the confidence data indicates a confidence below a threshold for a given number of instances (e.g., percentage of instances, frequency of instances, total number of instances, etc.), the prediction component 114 can retrain the trained model 190 (e.g., based on current sensor data 146, current manufacturing parameters, etc.). In some embodiments, retraining may include generating one or more datasets using historical and / or synthetic data (e.g., via a dataset generator 172).

[0066] For illustrative purposes only, rather than limiting them, aspects of this disclosure describe training one or more machine learning models 190 using historical data (e.g., historical sensor data 144, historical manufacturing parameters), inputting current data (e.g., current sensor data 146, current manufacturing parameters, and current measurement data) into one or more trained machine learning models to determine predictive data 168. In other embodiments, heuristic models, physical-based models, or rule-based models are used to determine predictive data 168 (e.g., without using trained machine learning models). In some embodiments, such models can be trained using historical data. In some embodiments, these models can be retrained using historical data. The predictive component 114 can monitor historical sensor data 144, historical manufacturing parameters, and measurement data 160. Any information described with respect to the data input 210 in Figure 2 can be monitored or used in heuristic models, physical-based models, or rule-based models.

[0067] In some embodiments, the functions of the client device 120, prediction server 112, server machine 170, and server machine 180 can be provided by fewer machines. For example, in some embodiments, server machines 170 and 180 can be integrated into a single machine. In some other embodiments, server machine 170, server machine 180, and prediction server 112 can be integrated into a single machine. In some embodiments, the functions of the client device 120 and prediction server 112 can be integrated into a single machine. In some embodiments, the functions of the client device 120, prediction server 112, server machine 170, server machine 180, and data store 140 can be performed by a cloud-based service.

[0068] In general, functions described as being performed by the client device 120, prediction server 112, server machine 170, and server machine 180 in one embodiment can also be performed on the prediction server 112 in other embodiments as needed. Furthermore, functions belonging to a particular component can also be performed by different components or multiple components working together. For example, in some embodiments, the prediction server 112 can determine corrective actions based on prediction data 168. In other examples, the client device 120 can determine the prediction data 168 based on the output from a trained machine learning model.

[0069] Furthermore, the functionality of a specific component can be performed by different components or multiple components working together. One or more of the prediction server 112, server machine 170, or server machine 180 can be accessed as a service provided to other systems or devices via an appropriate application programming interface (API).

[0070] In some embodiments, “User” may be represented as a single individual. However, in other embodiments of this disclosure, “User” may be an entity controlled by multiple users and / or automated sources. For example, a collection of individual users integrated as a group of administrators may be considered “User”.

[0071] Embodiments of this disclosure can be applied to data quality assessment, feature enhancement, model evaluation, virtual measurement (VM), predictive maintenance (PdM), marginal optimization, process control, and the like.

[0072] Figure 2 shows a block diagram of a system 200 including a dataset generator 272 (e.g., the dataset generator 172 in Figure 1) for creating datasets for training, testing, and validation of a model (e.g., model 190 in Figure 1) according to several embodiments. The dataset generator can be used to create datasets for one model or multiple models. Each model used may have its own dedicated dataset generator, or multiple models may share one dataset generator. Each dataset generator 272 may be part of the server machine 170 in Figure 1. In some embodiments, multiple machine learning models associated with a manufacturing apparatus 124 may be trained, used, and maintained (e.g., within a manufacturing facility). Each machine learning model may be associated with one dataset generator 272, or multiple machine learning models may share one dataset generator 272.

[0073] Figure 2 shows a system 200 including a dataset generator 272 for creating datasets for one or more supervised models (e.g., model 190 in Figure 1). The dataset generator 272 can create datasets (e.g., data input 210, target output 220) using historical data and / or calibration data. In some embodiments, an unsupervised machine learning model can be trained using a dataset generator similar to the dataset generator 272. For example, the target output 220 may not be generated by the dataset generator 272. Unsupervised models can be used for anomaly detection, outlier detection, clustering, etc. Unsupervised models can be used for data related to plasma generation systems, for example, to determine outliers that require further investigation.

[0074] The dataset generator 272 can generate datasets for training, testing, and validating a model. In some embodiments, the dataset generator 272 can generate datasets for machine learning models. In some embodiments, the dataset generator 272 can generate datasets for training, testing, and / or validating a model configured to classify plasma generation operations as normal or abnormal. The machine learning model is provided with a historical sensor dataset 244A as a data input 210A. The machine learning model can be configured to receive sensor data as input data and generate plasma quality classification data as output.

[0075] The input sensor data set may include various data types indicating plasma quality. The input sensor data may include the output of sensors related to the operation of the plasma feedback system, such as the adjustable capacitor position of the plasma generation system. The input sensor data may include the output of a digital twin model of the plasma feedback system (e.g., showing the power supplied to the plasma generated by the plasma generation system). The input sensor data may include electromagnetic (e.g., light) emission data collected from the plasma generation region of the processing chamber. The input sensor data may include data from an emission spectrometer. The input sensor data may include plasma emission measured by an optical sensor. The input sensor data may include an indicator of reflected power from the plasma generation system. Reflected power may be caused by impedance mismatch between the plasma generation device (e.g., plasma coil) and one or more other components of the plasma generation system.

[0076] The dataset generator 272 can be used to generate data for any type of machine learning model that accepts sensor data as input. The dataset generator 272 can be used to generate data for a machine learning model that generates substrate anomaly predictions based on plasma generation quality. The dataset generator 272 can be used to generate data for a machine learning model that recommends corrective actions based on plasma generation quality. The dataset generator 272 can be used with a model that generates predictive measurement data for a substrate. The dataset generator 272 can be used to generate data for a machine learning model configured to provide process control instructions, such as changing the plasma generation procedure or halting the plasma processing operation if the plasma generation process meets certain threshold failure criteria. The dataset generator 272 can be used to generate data for a machine learning model configured to identify product anomalies and / or processing equipment failures.

[0077] In some embodiments, the dataset generator 272 generates a dataset (e.g., a training set, a validation set, a test set) containing one or more data inputs 210 (e.g., a training input, a validation input, a test input). The data inputs 210 can be provided to the training engine 182, the validation engine 184, or the test engine 186. The dataset is used to train, validate, or test a model (e.g., model 190 in Figure 1).

[0078] In some embodiments, the data input 210 may include one or more datasets. For example, the system 200 may generate a sensor dataset that includes one or more sensor data from one or more types of sensors, a combination of sensor data from one or more types of sensors, a pattern from sensor data from one or more types of sensors, and / or a composite version thereof.

[0079] In some embodiments, the dataset generator 272 can generate a first data input corresponding to a first historical sensor dataset 244A for training, validation, or testing a first machine learning model. The dataset generator 272 can generate a second data input corresponding to a second historical sensor dataset 244B for training, validation, or testing a second machine learning model. The dataset generator 272 can use further historical sensor datasets 244C, ... 244Z for further training, validation, and / or testing of machine learning models.

[0080] In some embodiments, the dataset generator 272 generates a dataset (e.g., training set, validation set, test set) containing one or more data inputs 210 (e.g., training input, validation input, test input). The dataset may also contain one or more target outputs 220 corresponding to the data inputs 210. The dataset may also contain mapping data that maps the data inputs 210 to the target outputs 220. In some embodiments, the dataset generator 272 can generate data for training a machine learning model configured to output classifications of plasma generation quality by generating a dataset containing output plasma quality data 268. The dataset generator 272 can generate a dataset containing performance classification data of a plasma generator as a target output. The data inputs 210 may also be referred to as “features,” “attributes,” “feature vectors,” or “information.” In some embodiments, the dataset generator 272 can provide the dataset to the training engine 182, validation engine 184, or test engine 186. The dataset is used to train, validate, or test machine learning models (for example, one of the machine learning models included in Model 190, Ensemble Model 190, etc.).

[0081] The data input 210 for training, validating, or testing a machine learning model may include information about a specific manufacturing chamber (e.g., a specific substrate manufacturing device). In some embodiments, the data input 210 may include information about a specific type of manufacturing device (e.g., a manufacturing device with specific characteristics). The data input 210 may include data related to a specific type of device (e.g., intended function, design, manufactured according to a specific recipe, etc.). By training a machine learning model based on types of equipment, devices, recipes, etc., the trained model can generate predictive data in various settings (e.g., various equipment, products, etc.).

[0082] In some embodiments, a dataset is generated, and after training, validating, or testing a machine learning model using that dataset, the model can be further trained, validated, or tested, or tuned (for example, by adjusting weights or parameters associated with the model's input data, such as connection weights in a neural network).

[0083] In some embodiments, a similar approach using a dataset generator can be used for training / calibrating digital twin models, physics-based models, etc. In some embodiments, calibration data for a digital twin is obtained by performing one or more experimental procedures. For example, plasma is generated under various combinations of process conditions. The model can be calibrated using data obtained from experimental / calibration operations. The digital twin model includes a schematic, and the attributes of the schematic can be calibrated based on the calibration procedure.

[0084] Figure 3 is a block diagram showing a system 300 that generates output prediction data (e.g., prediction data 168 in Figure 1) according to several embodiments. The system 300 can be used in combination with a machine learning model (e.g., via model 190 in Figure 1) configured to generate data indicating the quality of plasma generation. In some embodiments, the system 300 can be used in combination with a machine learning model to determine corrective actions related to manufacturing equipment. In some embodiments, the system 300 can be used in combination with a machine learning model to determine failures in manufacturing equipment. In some embodiments, the system 300 can be used in combination with a machine learning model to cluster or classify substrates. The system 300 can be used in combination with a machine learning model having different functions related to a manufacturing system.

[0085] In block 310, system 300 performs data partitioning of data used for training, validation, and / or testing of machine learning models. System 300 may include components of the prediction system 110 in Figure 1. Data partitioning can be performed via the data set generator 172 in Figure 1. In some embodiments, plasma quality data 364 includes historical data such as historical sensor data indicating plasma generation, historical plasma quality data, and historical classification data (e.g., classification of whether the plasma meets performance thresholds). Plasma quality data 364 can be partitioned in block 310 to generate a training set 302, a validation set 304, and a test set 306. For example, the training set may be 60% of the training data, the validation set 20% of the training data, and the test set 20% of the training data.

[0086] The generation of training set 302, validation set 304, and test set 306 can be adjusted to suit a specific application. For example, the training set may consist of 60% of the training data, the validation set 20% of the training data, and the test set 20% of the training data. System 300 can generate multiple feature sets for each of the training set, validation set, and test set. For example, if the plasma quality data 364 includes sensor data containing features obtained from sensor data from 20 sensors (e.g., sensor 126 in Figure 1), the sensor data can be split into a first feature set containing sensors 1-10 and a second feature set containing sensors 11-20. Target inputs, target outputs, or both can be split into sets, or neither can be split. Multiple models can be trained on different datasets.

[0087] In block 312, system 300 performs model training using training set 302 (for example, via training engine 182 in Figure 1). Training of machine learning models and / or physically based models (e.g., digital twins) can be implemented using supervised training methods. In supervised training, a training dataset containing labeled inputs is provided to the model, its output is observed, the error is defined (by measuring the difference between the output and the label value), and the model's weights are adjusted to minimize the error using techniques such as deep gradient descent or backpropagation. In many applications, by repeating this process for a large number of labeled inputs in the training dataset, a model can be obtained that can produce the correct output even when inputs different from those present in the training dataset are presented. In some embodiments, training of machine learning models can be implemented using unsupervised methods. For example, labels or classifications may not be provided during training. Unsupervised models can be configured to perform anomaly detection, clustering of results, etc.

[0088] Each training data item in the training dataset is input to a model (e.g., a machine learning model). The model can then process the input training data items (e.g., the number of measured dimensions of a manufactured device, a cartoon image of a manufactured device, etc.) to generate an output. For example, the output may include predicted plasma generation quality data. The output can be compared to the labels of the training data items (e.g., labels assigned to the data by experts).

[0089] The processing logic then compares the generated output with the labels included in the training data items. Based on the difference between the output and the labels, the processing logic determines the error (classification error). Based on the error, the processing logic adjusts one or more weights or values ​​of the model.

[0090] When training a neural network, an error term or delta can be determined for each node in the artificial neural network. Based on this error, the artificial neural network adjusts one or more parameters (weights of one or more inputs to a node) for one or more nodes. The parameters are updated in a backpropagation manner, with the nodes of the top layer being updated first, followed by the nodes of the next layer, and so on. An artificial neural network contains multiple layers of "neurons," each layer receiving input values ​​from the neurons of the previous layer. The parameters of each neuron include weights associated with the values ​​received from each neuron of the previous layer. Therefore, parameter adjustment can include adjusting the weights assigned to each input of one or more neurons in one or more layers of the artificial neural network.

[0091] System 300 can train multiple models using multiple feature sets of the training set 302 (e.g., a first feature set of training set 302, a second feature set of training set 302, etc.). For example, System 300 can train a model to generate a first trained model using a first feature set in the training set (e.g., sensor data from sensors 1-10). Furthermore, System 300 can train a model to generate a second trained model using a second feature set in the training set (e.g., sensor data from sensors 11-20). In some embodiments, a third trained model can be generated by combining the first and second trained models (e.g., this may result in a better predictor or synthetic data generator than the first or second trained model alone). In some embodiments, the feature sets used for comparing models may overlap (e.g., the first feature set is sensor data from sensors 1-15, and the second feature set is sensors 5-20). In some embodiments, hundreds of models may be generated, including various permutations of features and combinations of models.

[0092] In block 314, system 300 performs model validation using validation set 304 (e.g., via validation engine 184 in Figure 1). System 300 can validate each of the trained models using the corresponding feature sets of validation set 304. For example, system 300 can validate a first trained model using a first feature set in the validation set (e.g., sensor data from sensors 1-10) and validate a second trained model using a second feature set in the validation set (e.g., sensor data from sensors 11-20). In some embodiments, system 300 can validate hundreds of models generated in block 312 (e.g., models having various feature permutations, model combinations, etc.). In block 314, system 300 can determine the accuracy of one or more trained models (e.g., via model validation) and determine whether one or more trained models have an accuracy that meets a threshold accuracy. If it is determined that none of the trained models meet the threshold precision, the flow returns to block 312, and system 300 performs model training using a different feature set of the training set. If one or more of the trained models meet the threshold precision, the flow proceeds to block 316. System 300 can discard trained models with precision below the threshold precision (for example, based on the validation set).

[0093] In block 316, system 300 performs model selection (e.g., via selection engine 185 in Figure 1) to determine which of the one or more trained models that meet the threshold accuracy has the highest accuracy (e.g., model 308 selected based on the validation in block 314). If two or more trained models that meet the threshold accuracy are determined to have the same accuracy, the flow returns to block 312, where system 300 performs model training using a more refined training set corresponding to a more refined feature set to determine the trained model with the highest accuracy.

[0094] In block 318, system 300 performs model testing (e.g., via test engine 186 in Figure 1) using test set 306 to test the selected model 308. System 300 tests the first trained model using a first feature set in the test set (e.g., sensor data from sensors 1-10) and determines whether the first trained model meets the threshold precision (e.g., based on the first feature set in test set 306). In response that the precision of the selected model 308 does not meet the threshold precision, the flow proceeds to block 312, where system 300 performs model training (e.g., retraining) using a different training set corresponding to a different feature set (e.g., sensor data from different sensors). If the precision of the selected model 308 does not meet the threshold precision, it may indicate that the selected model 308 is overfitted to training set 302 and / or validation set 304. If the precision of the selected model 308 does not meet the threshold precision, it may indicate that the selected model 308 is not applicable to other datasets. If, based on the test set 306, it is determined that the accuracy of the selected model 308 meets the threshold accuracy, the flow proceeds to block 320. At least in block 312, the model can learn patterns in the training data to make predictions or generate synthetic data, and in block 318, the system 300 can apply the model to the remaining data (e.g., test set 306) to test the predictions.

[0095] In block 320, system 300 receives current sensor data 322 using a trained model (e.g., selected model 308) and determines (e.g., extracts) output prediction data 324 from the output of the trained model. Current sensor data 322 may include data from one or more tunable components of the plasma generation feedback system. Current sensor data 322 may include the output of a digital twin model. The output of the digital twin model is provided as input to the tunable component data. Current sensor data 322 may include emission data, reflected power data, etc. Corrective actions related to the manufacturing apparatus 124 in Figure 1 can be performed taking the output prediction data 324 into consideration. In some embodiments, current sensor data 322 may correspond to the same type of features in the historical data used to train the machine learning model. In some embodiments, current sensor data 322 may correspond to a subset of the types of features in the historical data used to train the selected model 308 (e.g., the machine learning model can be trained using a large number of sensor measurements and can be configured to generate an output based on a subset of the sensor measurements).

[0096] In some embodiments, the performance of machine learning models trained, validated, and tested by system 300 may degrade. For example, the manufacturing system associated with a trained machine learning model may change gradually or suddenly. Changes in the manufacturing system may lead to a degradation in the performance of the trained machine learning model. A new model can be generated to replace a degraded machine learning model. A new model can be generated by modifying the old model, such as by retraining or generating a new model.

[0097] Modifying a trained machine learning model may include providing the current sensor data 322 for training / retraining the model in block 312. Modifying a trained machine learning model may also include providing additional training data 346 for training / retraining in block 312. The additional training data may include classification labels associated with the current sensor data 322, additional sensor data and additional corresponding classification labels, etc.

[0098] In some embodiments, one or more of operations 310-320 can be performed in various orders and / or in conjunction with other operations not shown or described herein. In some embodiments, one or more of operations 310-320 may not be performed. For example, in some embodiments, one or more of the following may not be performed: data partitioning of block 310, model verification of block 314, model selection of block 316, or model testing of block 318.

[0099] Figure 3 shows a system configured for training, validating, testing, and using one or more machine learning models. The machine learning models are configured to receive input data (e.g., setpoints provided to manufacturing equipment, sensor data, measurement data, etc.) and provide output data (e.g., prediction data, corrective action data, classification data, etc.). The division, training, validation, selection, testing, and use of each block of the system 300 can be performed in the same way as training a second model with different types of data. Retraining can also be performed using the current sensor data 322 and / or additional training data 346.

[0100] Figures 4A-C are flowcharts of methods 400A-C related to determining and / or monitoring plasma generation quality according to several embodiments. In some embodiments, methods 400A-C may include training and utilizing machine learning models. Methods 400A-C may be executed by processing logic. Processing logic may include hardware (e.g., circuits, dedicated logic, programmable logic, microcode, processing devices, etc.), software (e.g., instructions executed on processing devices, general-purpose computer systems or dedicated machines), firmware, microcode, or a combination thereof. In some embodiments, methods 400A-C may be partially executed by a prediction system 110. Method 400A may be partially executed by a prediction system 110 (e.g., server machine 170 and dataset generator 172 in Figure 1, dataset generator 272 in Figure 2). In embodiments of this disclosure, the prediction system 110 may use method 400A to generate a dataset for training, validating, or testing at least one of machine learning models. Methods 400B to C can be executed by the prediction server 112 (e.g., prediction component 114) and / or the server machine 180 (e.g., training, verification, and test operations are performed by the server machine 180). Methods 400B to C can also be executed by the client device 120, the corrective action component 122, etc. In some embodiments, a non-temporary machine-readable storage medium, when executed by a processing device (e.g., prediction system 110, server machine 180, prediction server 112, etc.), stores instructions that cause the processing device to execute one or more of Methods 400A to C.

[0101] For the sake of simplicity, methods 400A–C are illustrated and described as a series of operations. However, the operations provided herein can be performed in various orders and / or simultaneously, and in parallel with other operations not shown or described herein. Furthermore, not all illustrated operations are performed in order to carry out methods 400A–C by the disclosed subject matter. Moreover, those skilled in the art will understand and recognize that methods 400A–C can also be represented as a series of interrelated states via a state diagram or events.

[0102] Figure 4A is a flowchart of method 400A for generating a dataset for a machine learning model, according to several embodiments. Referring to Figure 4A, in some embodiments, in block 401, the processing logic that executes method 400A initializes the training set T to an empty set.

[0103] In block 402, the processing logic generates a first data input (e.g., a first training input, a first validation input) which may include one or more of the following: sensors, manufacturing parameters, measurement data, etc. In some embodiments, the first data input may include a first feature set relating to the type of data, and the second data input may include a second feature set relating to the type of data (as described with reference to Figure 3, for example). The input data may include historical data. The input data may be associated with plasma generation quality. The input data may include the output of a digital twin model of the plasma generation system. The input data may include sensor data related to plasma generation, such as reflected power values ​​and plasma spectral data.

[0104] In some embodiments, in block 403, the processing logic optionally generates a first target output for one or more data inputs (e.g., a first data input). In some embodiments, the input includes one or more sensor measurements, and the target output is an indicator of plasma quality. In some embodiments, the input includes one or more sensor channels, and the target output includes recommended corrective actions. In some embodiments, the first target output is predictive data. In some embodiments, the input data may be in the form of sensor data, and the target output may be a list of potentially failing components, as in the case of a machine learning model configured to identify a failing manufacturing system. In some embodiments, the input may be sensor data indicating plasma quality, and the output may be a predicted likelihood that plasma generation meets a target threshold condition. In some embodiments, the output may include the likelihood that plasma generation meets one or more threshold performance criteria. In some embodiments, no target output is generated (e.g., an unsupervised machine learning model that does not require the provision of a target output and allows for the discovery of grouping or correlations of input data).

[0105] In block 404, the processing logic optionally generates mapping data that indicates input / output mappings. The input / output mapping (or mapping data) can represent data inputs (e.g., one or more data inputs as described herein), target outputs for the data inputs, and associations between the data inputs and target outputs. In some embodiments, such as machine learning models where no target outputs are provided, block 404 may not be executed.

[0106] In block 405, in some embodiments, the processing logic adds the mapping data generated in block 404 to the dataset T.

[0107] In block 406, the processing logic branches based on whether the dataset T is sufficient for at least one of the training, validation, and / or testing of a machine learning model, such as model 190 in Figure 1. If it is sufficient, execution proceeds to block 407; otherwise, execution returns to block 402. In some embodiments, the sufficiency of dataset T can be determined solely on the number of inputs in the dataset (which in some embodiments are mapped to outputs). In other embodiments, the sufficiency of dataset T can be determined in addition to, or instead of, the number of inputs, on one or more other criteria (e.g., a measure of the diversity of data examples, precision, etc.).

[0108] In block 407, the processing logic provides a dataset T (for example, to a server machine 180) to train, validate, and / or test a machine learning model 190. In some embodiments, dataset T is a training set and is provided to the training engine 182 of the server machine 180 to perform training. In some embodiments, dataset T is a validation set and is provided to the validation engine 184 of the server machine 180 to perform validation. In some embodiments, dataset T is a test set and is provided to the test engine 186 of the server machine 180 to perform testing. For example, in the case of a neural network, input values ​​for a given input / output mapping (e.g., numerical values ​​associated with data input 210A) are input to the neural network, and output values ​​for the input / output mapping (e.g., numerical values ​​associated with target output 220A) are stored in the output nodes of the neural network. Next, the connection weights of the neural network are adjusted according to a learning algorithm (e.g., backpropagation, etc.), and this procedure is repeated for other input / output mappings in dataset T. After block 407, the model (e.g., model 190) can be trained using the training engine 182 of server machine 180, validated using the validation engine 184 of server machine 180, or tested using the test engine 186 of server machine 180. The trained model can be implemented by the prediction component 114 of prediction server 112 and generate prediction data 168 for performing signal processing or corrective actions related to manufacturing equipment 124.

[0109] Figure 4B is a flowchart of Method 400B, which utilizes a trained machine learning model to determine plasma generation quality in several embodiments. In block 410 of Method 400B, the processing logic provides a first performance metric of the plasma generator in the processing chamber to the trained machine learning model. The trained machine learning model may be any type or architecture suitable for classification. The trained machine learning model may be a support vector machine. The first performance metric indicates the power supplied to the plasma generated by the plasma generator. The first performance metric can be generated by providing one or more inputs to a digital twin model of the plasma generation system associated with the plasma generator.

[0110] The first performance indicator may relate to the power supplied to the plasma. The first performance indicator may relate to the current resistance associated with the plasma. The generation of the first performance indicator may include providing one or more performance indicators of the plasma generation system to a digital twin model. The generation of the first performance indicator may include providing the characteristics of one or more tunable components of the plasma generation system to a digital twin model. The generation of the first performance indicator may include providing data on the characteristics of the plasma feedback system to a digital twin model. The generation of the first performance indicator may include providing measured plasma generation feedback data (e.g., data measured from the plasma generation feedback system) to a digital twin model. The generation of the first performance indicator may include providing the capacitance of one or more tunable capacitors to a digital twin model. The generation of the first performance indicator may include comparing the characteristics determined using the circuit model with calibration characteristics (e.g., characteristics of the plasma generator under conditions where no plasma is generated).

[0111] In block 412, the processing logic provides a second performance metric of the plasma generator to a trained machine learning model. The second performance metric may include a different input channel from the first performance metric. The second performance metric may be selected to provide information that complements the first performance metric. The second performance metric may be spectral data related to plasma emission. The second performance metric may include plasma emission data collected by an optical sensor. The second performance metric may be the reflected power of the plasma generation system. In some embodiments, three or more performance metrics of the plasma generator may be provided to the trained machine learning model.

[0112] In block 414, the processing logic receives a composite performance index for the plasma generator from a trained machine learning model. The composite performance index is based on a first performance index and a second performance index. The composite performance index may also be the output of the trained machine learning model. The composite performance index may be a classification such as normal plasma, abnormal plasma, or extremely abnormal plasma. The composite performance index may also be a score indicating the quality of the plasma. The composite performance index may also be the probability that plasma generation meets one or more threshold conditions, such as the probability that plasma generation is normal, the probability that plasma generation is abnormal, or the probability that the plasma meets one or more threshold performance criteria. The composite performance index may include recommended corrective actions.

[0113] In block 416, the processing logic takes corrective actions considering a composite performance index. Corrective actions may include providing warnings to the user. Corrective actions may include updating the process recipe. Corrective actions may include adjusting operations during the process considering plasma quality. Corrective actions may include suspending operational procedures. Corrective actions may include scheduling maintenance. Corrective actions may include scheduling measurements or other additional investigations.

[0114] Figure 4C is a flowchart of method 400C for performing corrective measures related to plasma generation in a processing chamber, according to several embodiments. In block 420, the processing logic obtains measurements from a calibrated feedback control device of the processing chamber. The feedback control device may be part of the plasma generation system. The feedback control device may be a tunable capacitor.

[0115] A feedback control device can be calibrated to determine the correspondence between the measurements of the feedback control device and the characteristics of the plasma generation system. Calibration of a feedback control device may include generating a schematic diagram representing the plasma generation system. Calibration of a feedback control device may include generating a digital twin model based on the plasma generation system. Calibration of a feedback control device may include generating plasma under various processing conditions and determining the characteristic values ​​of the components of the plasma generation system. Calibration of a feedback control device may include fitting the characteristic values ​​of the components of the plasma generation system. Calibration of a feedback control device may include fitting the characteristic values ​​of the components of the plasma generation system so that variations in characteristics known to be static / invariant with respect to process conditions are minimized. Calibration of a feedback control device may utilize known constants of components such as model circuits, digital twins, and plasma generation systems.

[0116] In block 422, the processing logic determines a first performance indicator of the plasma generator in the processing chamber based on measurements of a calibrated feedback control device. Determining the first performance indicator may include providing measurements of the calibrated feedback control device to a digital twin model. The first performance indicator may relate to the power supplied to the plasma. The first performance indicator may relate to the resistance to the current supplied by the plasma. Furthermore, the first performance indicator may be based on a calibration value (e.g., the resistance value of the bare plasma device when no plasma is being generated).

[0117] Determining the first performance indicator involves determining the characteristics of a plasma generator under electrical load when no plasma is being generated. For example, the plasma generator may be powered, but the chamber may be evacuated to avoid plasma generation. Determining the first performance indicator involves comparing the characteristics of the plasma generator when no plasma is being generated with the characteristics of the plasma generator during plasma processing operation.

[0118] In block 424, the processing logic obtains a second performance indicator of the plasma generator from a first sensor in the processing chamber. The second performance indicator may be, or may include, data related to the emission of electromagnetic radiation from the plasma. The second performance indicator can be collected by an electromagnetic sensor, an optical sensor, etc. The second performance indicator may be based on measured emission intensity. The second performance indicator may be based on measured emission intensity in a target wavelength band, a target wavelength set, etc. The second performance indicator may include measured emission intensity at wavelengths that indicate plasma generation.

[0119] In some embodiments, the second performance indicator includes a measurement of reflected power. The second performance indicator may include a measurement of reflected RF power supplied to the plasma generator. Reflected RF power may be due to impedance mismatch between components of the plasma generation system. High reflected RF power may indicate a high probability of plasma generation failure.

[0120] In block 426, the processing logic provides the plasma monitoring module with a first performance indicator and a second performance indicator of the plasma generator. The plasma monitoring module may be a heuristic model or a rule-based model, or may include both. The plasma monitoring module may be a statistical model, or may include both. The plasma monitoring module may be a logistic model, or may include both. The plasma monitoring module may be a coupled stochastic model, or may include both. The plasma monitoring module may be a trained machine learning model, or may include both. In some embodiments, a combination of models can be used. For example, the output of a logistic model can be used to train a machine learning model. In some embodiments, the output of a logistic model can serve as a starting point for using a machine learning model to determine one or more decision boundaries for recommending corrective actions related to plasma generation.

[0121] In block 428, the processing logic obtains a composite performance index for the plasma generator from the plasma monitoring module. The composite performance index may be based on first and second performance indicators. Furthermore, the composite performance index may be based on a third performance indicator of the plasma generator. The composite performance index may be based on a target number of performance indicators for the plasma generator. One or more performance indicators for the plasma generator can be provided by various sensors in the processing chamber. The plasma monitoring module can receive inputs indicating plasma quality and output predictions of plasma quality. The plasma monitoring module can be configured to output the probability of plasma generation failures. The plasma monitoring module can be configured to output recommendations for corrective actions related to plasma generation. The composite performance index may be a combined probability of plasma generation failures considering the various indicators provided to the plasma monitoring module. The composite performance index may be a probability of plasma failures based on the output of a trained machine learning model.

[0122] In block 430, the processing logic executes corrective actions based on the combined performance indicators of the plasma generator. These corrective actions may include, or may include, scheduling maintenance, updating process recipes, modifying ongoing operations, suspending or stopping ongoing operations, initiating plasma strike operations, providing warnings to users, and scheduling measurements or other investigations.

[0123] Figure 5A shows an exemplary schematic 500A for constructing a digital twin model of a plasma processing system according to several embodiments. Schematic 500A approximates the arrangement of components in a plasma generation system. Schematic 500A may also be a simplified representation of the components in a plasma generation system. Schematic 500A represents a specific plasma generation system. Schematic 500A represents a model circuit representing a plasma generation system. The arrangement of components in the figure, the overall shape of the circuit, the flow of electricity, etc., may differ from the example shown in Figure 5A.

[0124] Circuit diagram 500A includes static components (e.g., non-adjustable components). These static components include resistor 502, first inductor 504, and second conductor 506. Components shown in the circuit diagram may include the characteristics of multiple physical components. For example, multiple resistors may be collectively represented as resistor 502, and inductance, capacitance, and / or inductance of additional components or wiring may be included in the components of circuit diagram 500A.

[0125] Circuit diagram 500A includes dynamic components (e.g., components with adjustable characteristics). Dynamic components can be adjusted mechanically (e.g., by adjusting the capacitance by adjusting the physical spacing of conductive components). The electrical characteristics of dynamic components can be adjusted by other means. Dynamic components can provide information indicating their characteristics, for example, through a feedback system. The dynamic components of circuit example 500A include a first capacitor 508 and a second capacitor 510. One or more dynamic components of the plasma generation system may be included in a feedback system for plasma generation. The first capacitor 508 and / or the second capacitor 508 can be automatically adjusted. The capacitance of the capacitors can be adjusted. The capacitors can be adjusted to maximize the power supplied to the plasma.

[0126] Furthermore, circuit diagram 500A includes a plasma generator 512 and an A / C (alternating current) power supply 514. The plasma generator 512 may include multiple types of components, and multiple circuit parameters (e.g., resistance, inductance, etc.) may have multiple hidden variables. Any effects related to the plasma generator may be abstracted in some digital twin models. In some embodiments, as knowledge of the plasma generation system increases, multiple models of increasing complexity can be constructed. The digital twin is generated stepwise, and different levels of abstraction can be utilized depending on the generation stage of the digital twin model. The AC power supply 514 can supply high-frequency power to the plasma generator 512.

[0127] The exact characteristic values ​​of components included in a plasma generation system may vary. Characteristic values ​​may differ between systems. Characteristic values ​​may differ due to the use of different components. Characteristic values ​​may differ even between nominally identical components (e.g., components with the same manufacturer and part number). Characteristic values ​​may differ due to component aging, differences in manufacturing processes, differences in manufacturing tolerances, etc.

[0128] Directly measuring the characteristics of the components included in circuit diagram 500A can be difficult. Characteristics may have power dependence, RF frequency dependence, or other difficult-to-capture dependencies. Measuring component characteristics at RF frequency can be difficult. In particular, the characteristics of the plasma generator 512 can change significantly depending on electrical conditions, plasma conditions, etc.

[0129] Circuit diagram 500A can be used to determine the characteristics of one or more components of a plasma generation system. The plasma generation system can be used to generate plasma under various process conditions. For example, various process gases, gas mixtures, gas pressures, gas flow rates, etc., can be used. Nominal characteristic values ​​(e.g., resistance, capacitance, inductance, etc.) can be used as a starting point for generating a digital twin of the plasma generation system. The knowledge that some parameters are static can be used when determining the actual characteristic values ​​of the components of the plasma generation system. Determining the actual values ​​of the components in the plasma generation system can be considered as calibration of the model circuit diagram.

[0130] The plasma generation system can automatically adjust one or more components. The plasma generation system can automatically adjust one or more components via a plasma generation feedback loop. The plasma generation system can automatically adjust one or more components to increase the amount of power supplied to the plasma. The plasma generation system can adjust the capacitance of one or more capacitors. The plasma generation system can adjust the capacitance of the capacitor represented by capacitor 510. The plasma generation system can adjust the capacitance of the capacitor represented by capacitor 508. A digital twin model can accept the nominal setpoints of capacitors 510 and 508 as input. By monitoring several adjustable parameters of the plasma generation system, the characteristics of the static components and the characteristic curves of the dynamic components can be determined. In some embodiments, the characteristics of various components are selected to minimize the deviation of the characteristic values ​​of the static components.

[0131] In some embodiments, the plasma generation system may involve complexity beyond what is shown in the circuit diagrams, such as circuit diagram 500A. For example, the plasma generator 512 may include multiple components, including plasma effects that are only observed during plasma generation. Additional calibration operations can be performed to determine these "hidden" parameters. For example, the plasma generator 512 can be powered under conditions where no plasma is being generated. The plasma generator 512 can be powered while the processing chamber is under vacuum. The plasma generator can be powered when the pressure in the chamber is below a threshold. These procedures can characterize the "bare" plasma generator 512 without the effect of supplying energy to generate and / or maintain the plasma. In some embodiments, additional parameters can be systematically tested. The plasma generator 512 can abstract one or more tunable components. The tuning effects of these components can be understood by taking measurements (e.g., capacitance of calibrated capacitors 508 and 510) while intentionally tuning these components. In some embodiments, the plasma generator 512 may include one or more adjustable capacitors for, for example, dividing current between multiple plasma generation coils. The division parameter can be changed to determine the characteristics at various values ​​of the division parameter. The characteristics can be determined at various current ratios. A physical or empirical model can be generated that describes the behavior of the plasma generator 512 with respect to one or more hidden variables, such as the current division ratio. In some embodiments, such operations can be used to obtain characteristic values ​​of the bare plasma generator at various input conditions of the hidden variables. In some embodiments, the difference between the expected operation and the measured operation, including plasma generation, for a combination of hidden variables of the bare plasma generator can be determined. The effect of the plasma load on the system can be calculated by comparing the measured characteristics (e.g., resistance, power consumption, etc.) with the expected characteristics of the system.The effect of plasma loading on a system can be calculated by comparing the measured characteristics with those of a system without plasma loading (e.g., a bare system). The effect of plasma loading on a system can also be determined based on a comparison of the measured characteristics of the system in use with those of the system when no plasma is generated.

[0132] In some embodiments, the power supplied to the plasma can be a useful indicator of plasma generation quality. By comparing the characteristics of the plasma generation system in the absence of plasma (e.g., the plasma generator alone) with the characteristics of the plasma generation system during operation, information regarding the power supply to the plasma can be obtained. The difference between the resistance exhibited by the plasma generator 512 during operation and the resistance exhibited by the plasma generator 512 in the absence of plasma (under similar or extrapolated other process conditions, hidden variables, etc.) can serve as an indicator of plasma quality.

[0133] The operation of the digital twin model may include providing inputs to the digital twin model and receiving outputs from the digital twin model indicating plasma quality. In some embodiments, measurements related to the plasma generation feedback system can be provided as inputs to the digital twin model. Measurements of the position of one or more adjustable components can be provided as inputs to the digital twin model. Measurements of the capacitance of one or more adjustable capacitors can be provided as inputs to the digital twin model. Measurements of capacitors 508 and 510 can be provided to the digital twin model corresponding to schematic 500A.

[0134] Plasma generation indicators can be provided as outputs of the digital twin model. These indicators may include surrogate indicators of plasma generation, such as resistance attributable to the plasma generation device, resistance attributable to the plasma itself (e.g., resistance derived from the resistance of the device alone), and power supplied to the plasma. From the various values ​​provided by the digital twin model, the output that most appropriately classifies the plasma generation quality can be provided as the output of the digital twin model.

[0135] Plasma generation metrics related to measurements of a plasma generation system can be enhanced by further metrics of plasma generation quality. Additional channels can improve the ability to generate accurate predictions of plasma generation quality. One or more additional channels can be probed along with a digital twin model of the plasma generation system. The decision-making model can integrate data from multiple input channels to determine plasma generation quality.

[0136] Additional input channels can include any data indicating plasma generation quality. Input channels can be selected to complement the digital twin model (e.g., channels based on different assumptions than the digital twin model, channels that may succeed in situations where the digital twin model might fail, etc.). For example, using a digital twin model of a plasma generation system including a feedback system might include the assumption that the feedback system nearly enables maximum power delivery to the plasma. Power reflected by the plasma generator may suggest that this assumption is not valid, potentially complementing the digital twin model in plasma quality prediction.

[0137] An additional input channel may include spectral data from the plasma. Plasma can emit radiation. Plasma may emit radiation in one or more wavelength bands. The wavelength bands may depend on the process gas, pressure, etc. Sensors (such as optical sensors) can receive radiation from the plasma. The radiation received from the plasma can be integrated over the relevant wavelength bands to provide information indicating plasma generation.

[0138] Figures 5B–E show various decision boundaries 500B–E corresponding to two input channels related to plasma generation in several embodiments. Decision boundaries 500B–E visually illustrate the prediction region based on two input channels. However, in practice, more channels can be utilized using corresponding multidimensional decision boundaries.

[0139] Figure 5B shows logical decision boundaries according to several embodiments. In some cases, a decision boundary is set for each input. For example, an "optimal" decision boundary is set for each input channel based on training data. The decision boundary for an input channel may be a decision boundary that provides optimal decision accuracy (e.g., normal plasma generation boundary and abnormal plasma generation boundary), a decision boundary that provides a target false negative rate, a decision boundary that provides a target false positive rate, etc.

[0140] A logical decision boundary can provide multiple regions within the decision space. The operation of the prediction system is determined by which region a measurement falls into. Figure 5B includes four regions, 520 to 526. Depending on the objective, measurement values ​​in one or more of the four regions can trigger corrective actions. In some embodiments, multiple regions can correspond to corrective actions. In some embodiments, one region can trigger a different corrective action than others. The separation between regions can be determined by the decision boundary associated with the decision boundary selected for the relevant input channels. For example, region 520 may include measurement values ​​that do not show plasma faults, and region 526 may include measurement values ​​where both input channels show plasma faults. Measurement values ​​in region 526 can trigger a first corrective action, such as stopping the manufacturing procedure. Measurement values ​​in regions 522 and / or 524 can trigger a second corrective action, such as scheduling measurements or maintenance. Measurement values ​​in region 520 may not trigger a corrective action.

[0141] Figure 5C shows dependent hard logic boundaries according to several embodiments. The decision boundary value of the first input channel is considered to depend on the value provided by the second input channel. The decision boundary isolation regions 530 and 532 may not have a single value in either input channel. The decision boundary isolation regions 530 and 532 may have a single value in the first input channel at a predetermined value in the second input channel. Furthermore, the regions can be defined so that different combinations of input channel values ​​can correspond to different corrective actions.

[0142] Figure 5D shows logistic decision boundaries according to several embodiments. A logistic function can be used to determine the decision boundary. The logistic function can be associated with the coupled probability of plasma generation failure given a first input value and a second input value. Region 544 can indicate that the probability of plasma generation failure is close to zero. Region 540 can indicate that the probability of plasma generation failure is high. Decision boundary 500D includes an intermediate region 542. In the intermediate region 542, the probability of failure can smoothly transition from a low probability near the boundary with region 544 to a high probability near the boundary with region 540. A target probability of failure can be used to determine further actions at the logistic decision boundary. Multiple decision boundaries can be incorporated. For example, no corrective action can be taken if the failure probability is below the target probability, a small-impact corrective action can be taken between the second and third target probabilities, and a large-impact corrective action can be taken if it is above the fourth target probability. Corrective actions may include scheduling maintenance, updating process recipes, updating equipment constants, adjusting ongoing processes, updating digital twin models or machine learning models, updating one or more decision boundaries, or halting ongoing processes.

[0143] Figure 5E shows decision boundaries 500E determined by a trained machine learning model in several embodiments. By providing input channel measurements as training inputs and classification data as training outputs, the machine learning model can generate complex decision boundaries for performing corrective actions. The trained machine learning model may be a support vector machine, a neural network, or other type of machine learning model. In some embodiments, multiple decision boundaries can demarcate multiple regions. Each region can be associated with a different corrective action. Each decision boundary can be associated with a different probability of plasma generation failure.

[0144] Figure 6 is a block diagram showing computer systems 600 according to several embodiments. In some embodiments, computer systems 600 can be connected to other computer systems (for example, via a network such as a local area network (LAN), intranet, extranet, or internet). Computer systems 600 can operate as a server or client computer in a client-server environment, or as a peer computer in a peer-to-peer or distributed network environment. Computer systems 600 can be provided by personal computers (PCs), tablet PCs, set-top boxes (STBs), personal digital assistants (PDAs), mobile phones, web appliances, servers, network routers, switches, bridges, or any device capable of executing a set of instructions (sequential instructions or other instructions) that specify the actions that a device should perform. Furthermore, the term “computer” includes any set of computers that individually or collectively execute a set of instructions (or instructions) in order to perform one or more of the methods described herein.

[0145] In a further embodiment, the computer system 600 includes a processing device 602, a volatile memory 604 (e.g., random access memory (RAM)), a non-volatile memory 606 (e.g., read-only memory (ROM) or electrically erasable programmable ROM (EEPROM)), and a data storage device 618, which can communicate with each other via a bus 608.

[0146] The processing device 602 can be provided by one or more processors, such as a general-purpose processor (e.g., a composite instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a microprocessor implementing other types of instruction sets, or a microprocessor implementing a combination of multiple types of instruction sets) or a dedicated processor (e.g., an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), or a network processor).

[0147] Furthermore, the computer system 600 may include a network interface device 622 (for example, connected to network 674). The computer system 600 may also include a video display unit 610 (for example, an LCD), an alphanumeric input device 612 (for example, a keyboard), a cursor control device 614 (for example, a mouse), and a signal generation device 620.

[0148] In some embodiments, the data storage device 618 may include a non-temporary computer-readable storage device 624 (e.g., a non-temporary machine-readable medium). This medium may store instructions 626 that encode one or more of the methods or functions described herein, such as instructions for encoding the components of Figure 1 (e.g., the prediction component 114, the corrective action component 122, the model 190, etc.) or instructions for carrying out the methods described herein.

[0149] Furthermore, instruction 626 can reside entirely or partially in the volatile memory 604 and / or processing device 602 while being executed by the computer system 600. Thus, the volatile memory 604 and processing device 602 can also constitute a machine-readable storage medium.

[0150] The computer-readable storage device 624 is shown as a single medium in the illustrated example. However, the term “computer-readable storage medium” includes a single medium or multiple mediums that store one or more executable instruction sets (e.g., a centralized or distributed database, and / or associated caches and servers). The term “computer-readable storage medium” also includes any tangible medium that stores or encodes instruction sets executed by a computer, and allows the computer to execute one or more of the methods described herein. The term “computer-readable storage medium” includes, but is not limited to, solid memory, optical media, magnetic media, etc.

[0151] The methods, components, and features described herein may be implemented by individual hardware components or integrated into the functionality of other hardware components such as ASICs, FPGAs, DSPs, or similar devices. Furthermore, the methods, components, and features may be implemented by firmware modules or functional circuits within hardware devices. Moreover, the methods, components, and features may be implemented by any combination of hardware devices and computer program components or by computer programs.

[0152] Unless otherwise specified, terms such as “receive,” “execute,” “provide,” “acquire,” “cause,” “access,” “determine,” “add,” “use,” “train,” “reduce,” “generate,” and “correct” refer to actions and processes performed or implemented by a computer system. These actions and processes manipulate and convert data represented as physical quantities (electronic quantities) in the computer system’s registers and memory into other data similarly represented as physical quantities in the computer system’s memory or registers or other information storage, transmission, or display devices. Furthermore, terms such as “first,” “second,” “third,” and “fourth” as used herein are intended as labels to distinguish different elements and may not have an ordinal meaning corresponding to their numerical designation.

[0153] Furthermore, the examples described herein also relate to apparatus for carrying out the methods described herein. This apparatus may be specifically configured for carrying out the methods described herein and may include a general-purpose computer system selectively programmed by a computer program stored in the computer system. Such a computer program may be stored in a computer-readable tangible storage medium.

[0154] The methods and illustrative examples described herein are not inherently related to any particular computer or other device. Various general-purpose systems can be used in accordance with the teachings provided herein. Furthermore, it may be convenient to construct more specialized devices to perform the methods and / or their individual functions, routines, subroutines, or calculations described herein. Examples of the construction of these various systems are given above.

[0155] The above description is illustrative and not limiting. While this disclosure is described with reference to specific examples and embodiments, it is understood that this disclosure is not limited to those described. The scope of this disclosure should be determined by reference to the entire scope of the claims and equivalents for which rights are granted.

Claims

1. The process involves obtaining measurements of the calibrated feedback control device of the processing chamber using a processing device, A process of determining a first performance indicator of the plasma generation device of the processing chamber based on measurements of a calibrated feedback control device using a processing device, A step of obtaining a second performance indicator of the plasma generator from a first sensor in the processing chamber, A process of providing a plasma monitoring module with a first performance indicator and a second performance indicator of the plasma generation device, A process to obtain a combined performance index of the plasma generator from the plasma monitoring module, A method comprising the step of taking corrective action considering a composite performance index of a plasma generator.

2. The method according to claim 1, comprising the step of obtaining a third performance index of the plasma generator from a second sensor of the processing chamber, wherein the composite performance index of the plasma generator is based on the first, second, and third performance indexes of the plasma generator.

3. The method according to claim 1, wherein the first sensor includes an electromagnetic radiation sensor, and the second performance indicator is based on the measured radiation intensity in a target wavelength set indicating plasma generation.

4. The method according to claim 1, wherein the first sensor comprises a reflected power sensor, and the second performance indicator is based on power reflected from a plasma generator.

5. The method according to claim 1, wherein the plasma monitoring module comprises a logistic model, and the composite performance index of the plasma generator includes a coupled probability of plasma generation failure considering a first performance index of the plasma generator and a second performance index of the plasma generator.

6. The method according to claim 1, wherein the plasma monitoring module comprises a trained machine learning model, the trained machine learning model is configured to determine the possibility of a plasma generation failure by taking into account a first performance indicator and a second performance indicator of the plasma generation device.

7. Calibrating a calibrated feedback control device involves The process involves generating a model circuit representing the plasma generation system of the processing chamber, wherein the plasma generation system includes a feedback control device. A process of generating plasma under multiple plasma conditions using a plasma generation device of a plasma generation system, The method according to claim 1, further comprising the step of calibrating a feedback control device using known constants of a model circuit.

8. The step of determining the first performance indicator of the plasma generation device of the processing chamber is: A process for determining the characteristics of a plasma generator when an electrical load is applied but no plasma is being generated, The method according to claim 1, further comprising the step of comparing the characteristics of the plasma generator when it is not generating plasma with the characteristics of the plasma generator during a plasma processing operation.

9. The method according to claim 1, wherein the calibrated feedback control device comprises an adjustable capacitor.

10. The method according to claim 1, wherein the corrective action includes one or more of updating the process recipe, stopping an ongoing process operation, initiating a plasma strike operation, setting a maintenance schedule, and providing a warning to the user.

11. A step of providing a trained machine learning model with a first performance indicator of the plasma generator of a processing chamber, wherein the first performance indicator of the plasma generator indicates the power supplied to the plasma of the plasma generator, A process of providing a second performance metric of the plasma generation device to a trained machine learning model, A process of receiving a composite performance index for the plasma generator based on a first performance index and a second performance index from a trained machine learning model, A method that includes a step of taking corrective action considering a composite performance index.

12. The method according to claim 11, wherein a second performance indicator of the plasma generator includes an indicator of plasma emission measured by an optical sensor.

13. A process of providing a machine learning model with multiple first performance indicators of a plasma generator as training inputs, A process of providing a machine learning model with multiple second performance indicators of a plasma generator as training inputs, The process involves providing performance classification data from multiple plasma generators to a machine learning model as the target output, The method according to claim 11, further comprising the step of training a machine learning model based on training inputs and target outputs to generate a trained machine learning model.

14. The method according to claim 11, wherein the composite performance index of the plasma generator includes the likelihood that the plasma generator satisfies a threshold performance criterion.

15. The process for generating the first performance indicator of the plasma generation device is: The process of generating a digital twin of the plasma generation device, The process of providing the measured plasma generation feedback data to a digital twin, The process involves receiving output from a digital twin based on measured plasma generation feedback data, The method according to claim 11, comprising the step of comparing the output from a digital twin with calibration data, wherein the first performance indicator of the plasma generator is based on the comparison of the output from the digital twin with the calibration data.

16. A non-temporary, machine-readable storage medium that stores instructions that cause a processing device to perform an operation when executed, wherein the operation is: A step of obtaining measurements of the calibrated feedback control device of the processing chamber, A step of determining a first performance indicator of the plasma generation device of the processing chamber based on measurements of a calibrated feedback control device, A step of obtaining a second performance indicator of the plasma generator from a first sensor in the processing chamber, A process of providing a plasma monitoring module with a first performance indicator and a second performance indicator of the plasma generation device, A process to obtain a combined performance index of the plasma generator from the plasma monitoring module, A machine-readable storage medium, including a process for performing corrective actions considering the combined performance indicators of a plasma generator.

17. The operation includes the step of obtaining a third performance index of the plasma generator from a second sensor of the processing chamber, wherein the composite performance index of the plasma generator is based on the first, second, and third performance indexes of the plasma generator, according to claim 16, for a non-temporary machine-readable storage medium.

18. A non-temporary machine-readable storage medium according to claim 16, wherein the first sensor comprises an electromagnetic radiation sensor, and the second performance indicator is based on the measured radiation intensity in a target wavelength set indicating plasma generation.

19. The process of calibrating a calibrated feedback control device is: A step of generating a model circuit representing the plasma generation system of a processing chamber, wherein the plasma generation system includes a feedback control device, A process of generating plasma under multiple plasma conditions using a plasma generation device of a plasma generation system, A non-temporary machine-readable storage medium according to claim 16, comprising the step of calibrating a feedback control device using known constants of a model circuit.

20. The step of determining the first performance indicator of the plasma generation device of the processing chamber is: A process for determining the characteristics of a plasma generator when an electrical load is applied but no plasma is being generated, A non-temporary machine-readable storage medium according to claim 16, comprising the step of comparing the characteristics of the plasma generator when it is not generating plasma with the characteristics of the plasma generator during a plasma processing operation.

Images