Diagnostic tool-to-tool matching and full-trace drill down analysis method for manufacturing equipment

Abstract

A method includes receiving trace sensor data associated with a first manufacturing process of a processing chamber. The method also includes processing the trace sensor data using one or more trained machine learning models that produce a representation of the trace sensor data, and then producing reconstructed sensor data based on the representation of the trace sensor data. The method also includes comparing the trace sensor data to the reconstructed sensor data. The method also includes determining one or more differences between the reconstructed sensor data and the trace sensor data. The method also includes determining whether to recommend a corrective action associated with the processing chamber based on the one or more differences between the trace sensor data and the reconstructed sensor data.

Description

Technical Field

[0001] The present disclosure relates to a diagnostic method for determining a manufacturing equipment failure, and in some embodiments, to a tool-to-tool matching method for determining a manufacturing equipment failure and full-trace analysis for drill-down analysis. Background Art

[0002] Products can be produced by performing one or more manufacturing processes using manufacturing equipment. For example, semiconductor manufacturing equipment can be used to produce substrates via semiconductor manufacturing processes. The products produced should have specific properties suitable for the target application. Understanding and controlling the properties within the manufacturing chamber helps in the consistent production of products. Summary of the invention

[0003] The following is a simplified summary of the disclosure to provide a basic understanding of certain aspects of the disclosure. This summary is not an extensive overview of the disclosure. This summary is neither intended to identify key or important elements of the disclosure nor to describe any scope of specific embodiments of the disclosure or any scope of the scope of the invention claims. The sole purpose of this summary is to present some concepts of the disclosure in a simplified form as a prelude to a more detailed description presented later.

[0004] In one aspect of the present disclosure, a method includes receiving trace sensor data associated with a first manufacturing process of a processing chamber. The method also includes processing the trace sensor data using one or more trained machine learning models, the one or more trained machine learning models generating a representation of the trace sensor data, and then generating reconstructed sensor data based on the representation of the trace sensor data. The one or more trained machine learning models output the reconstructed sensor data. The method also includes comparing the trace sensor data with the reconstructed sensor data. The method also includes determining one or more differences between the reconstructed sensor data and the trace sensor data based on the comparison. The method also includes determining whether to recommend a corrective action associated with the processing chamber based on the one or more differences between the trace sensor data and the reconstructed sensor data.

[0005] In another aspect, a system comprising a memory and a processing device coupled to the memory is enabled. The processing device is configured to receive trace sensor data associated with a first manufacturing process of a processing chamber. The processing device is also configured to process the trace sensor data using one or more trained machine learning models, the one or more trained machine learning models generating a representation of the trace sensor data, and then generating reconstructed sensor data based on the representation of the trace sensor data. The one or more trained machine learning models output the reconstructed sensor data. The processing device is also configured to compare the trace sensor data with the reconstructed sensor data. The processing device is also configured to determine one or more differences between the reconstructed sensor data and the trace sensor data based on the comparison. The processing device is also configured to determine whether to recommend a corrective action associated with the processing chamber based on the one or more differences between the trace sensor data and the reconstructed sensor data.

[0006] In another aspect of the present disclosure, a non-transitory machine-readable storage medium is disclosed. The medium stores instructions that, when executed, cause a processing device to perform operations. The operations include receiving trace sensor data associated with a first manufacturing process of a processing chamber. The operations include processing the trace sensor data using one or more trained machine learning models, the one or more trained machine learning models generating a representation of the trace sensor data, and then generating reconstructed sensor data based on the representation of the trace sensor data. The one or more trained machine learning models output the reconstructed sensor data. The operations also include comparing the trace sensor data with the reconstructed sensor data. The operations also include determining one or more differences between the reconstructed sensor data and the trace sensor data based on the comparison. The operations also include determining whether to recommend a corrective action associated with the processing chamber based on the one or more differences between the trace sensor data and the reconstructed sensor data. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] The present disclosure is illustrated by way of example and not by way of limitation in the figures of the accompanying drawings.

[0008] Figure 1 is a block diagram illustrating an exemplary system architecture according to some embodiments.

[0009] Figure 2 Depicted is a cross-sectional view of a fabrication chamber according to some embodiments.

[0010] Figure 3 A description of a dataset generator for creating datasets for training, testing, and validating models according to some embodiments.

[0011] Figure 4Ais a block diagram illustrating a system for generating output data according to some embodiments.

[0012] Figure 4B Flowchart of a method for performing tool-to-tool matching analysis using trace sensor data according to some embodiments.

[0013] Figure 4C is a flow chart of a method for generating summary data according to some embodiments.

[0014] Figure 5A Flowchart of a method for generating a data set for a model for generating predictive data according to some embodiments.

[0015] Figure 5B is a flow chart of a method for utilizing one or more models to perform corrective actions according to some embodiments.

[0016] Figure 5C A flowchart of a method for performing a training operation for tool-to-tool matching analysis is described according to some embodiments.

[0017] Figure 5D A flow chart illustrating a method of performing inference operations for tool-to-tool matching analysis according to some embodiments.

[0018] Figure 5E 5 is a flow chart of a method 500E for causing performance of corrective action based on a metric indicative of processing chamber performance, according to some embodiments.

[0019] Fig. 5F is a flow chart of a method for performing inlier analysis according to some embodiments.

[0020] Figure 5G is a flow chart of a method for performing trace sensor data analysis according to some embodiments.

[0021] Figure 5H is a flow chart of a method for performing outlier cluster analysis according to some embodiments.

[0022] Fig.5I is a flow chart of a method for performing normal value drift detection according to some embodiments.

[0023] Fig. 6A A visualization depicting an operation for segmenting a trace into transient and steady-state portions according to some embodiments.

[0024] Figure 6B Depicting a visualization of a digital twin output for a summary data generation operation according to some embodiments.

[0025] Figure 6C Depicting a visualization of the results of a clustering operation according to some embodiments.

[0026] Fig.6D Describe the results of the elastic dissimilarity model applied to a reference time trace and similar target time traces.

[0027] Fig. 6E Describe the results of the elastic dissimilarity model applied to a reference time trace and dissimilar target time traces.

[0028] FIG. 6F to FIG. 6G Depicting an example visualization of trace data analysis results according to some embodiments.

[0029] Figure 7 A diagram of the operation of a model capable of reducing the dimensionality of input data according to some embodiments.

[0030] Figure 8 is a block diagram illustrating a computer system according to some embodiments. DETAILED DESCRIPTION

[0031] This article describes techniques related to tool to tool matching (TTTM) diagnostic methods, which can be used to diagnose problems in manufacturing equipment and / or perform corrective actions. Manufacturing equipment can be used to produce products, such as substrates (e.g., wafers, semiconductors, displays, photovoltaics, etc.). Manufacturing equipment (e.g., manufacturing tools) typically include a manufacturing chamber that separates the processed substrate from the environment. The properties of the produced substrate should meet the target property values ​​to promote performance, function, etc. Manufacturing parameters are selected to produce substrates that meet the target property values. Manufacturing parameters may include process parameters, such as temperature, gas flow, plasma conditions, pressure, etc., and hardware parameters, such as component use, component settings, component positions, etc. The manufacturing system controls these parameters by specifying the set point of the property value and receiving data from one or more sensors disposed in the manufacturing chamber. The manufacturing system adjusts the set points of various components until the sensor readings match the set points (within the tolerance threshold value).

[0032] Manufacturing systems may exhibit or develop properties that result in non-optimal performance, for example, due to component drift, aging or failure, improper maintenance, changes in target process parameters, etc. Sensor data collected during a manufacturing process may contain information indicating subsystems and / or components that are drifting, aging, malfunctioning, or failing. Manufacturing systems may include a large number of sensors, including dozens or even hundreds to thousands of sensors in some systems. In some systems, sensors may collect data (e.g., trace data) over time during a manufacturing procedure. A single time trace (e.g., associated with one sensor and one process run or one product manufactured) may include hundreds of data points, thousands of data points, or more.

[0033] Ensuring consistent conditions, e.g., from run to run in a single chamber, between chambers, etc., can improve the consistency, performance, quality, etc. of the manufactured product. Tool-to-tool matching (TTTM) is a process designed to ensure consistency. In some cases, sensor data is used as a measure of process conditions. In conventional systems, it may be inconvenient to use sensor data for diagnostics and corrective actions. In some systems, sensor data can be used to identify the root cause of non-optimal tool performance. Non-optimal performance can be detected by measuring the finished product, e.g., via random quality control, sparse sampling, etc. In response to a measurement of non-optimal performance (e.g., a product that exceeds manufacturing tolerances), the sensor data of the tool can be investigated. The sensor data can be used to identify tool subsystems, components, etc. that are aging, drifting, faults, malfunctions, etc. that cause non-optimal tool performance. Identifying sensor data of interest may include comparing trace data from many different process operations and many different sensors, and may include comparisons with historical data as well as current data. When such a large amount of sensor data is available, isolating sensor data that indicates corrective action may be cumbersome. This analysis system is wasteful in terms of time spent, computer processing time and associated energy costs, and runs the risk of discovering some information but missing other indicative sensor data due to the tedious analysis being stopped after multiple components are found for which maintenance, replacement, etc. are to be performed.

[0034] In some systems, sensor data investigation may be triggered by non-optimal performance, for example, by the properties of one or more products produced exceeding manufacturing specifications. In some cases, only a subset of products undergo performance measurements, such as metrology measurements, to determine production quality. Performance measurements (e.g., metrology) may be expensive, for example, may require a large amount of time to generate. Manufacturing equipment can continue to be used to produce products while metrology measurements are being taken on previous products and / or while previous products are queued for metrology measurements. In the event of degradation of manufacturing equipment (e.g., equipment performance becomes non-optimal, such as due to component aging, drift, failure, etc.), metrology measurements of products can only be taken after many products have been processed using non-optimal equipment. As a result, non-optimal products may be produced. Such systems are wasteful in terms of time spent processing non-optimal products, energy consumed, materials consumed, etc.

[0035] In conventional systems, the difficulty of isolating informative sensor data (e.g., identifying data indicative of pending corrective actions associated with manufacturing equipment from a large volume of sensor data) results in metrology-based fault detection. In some cases, sensor data may be significantly affected by drift, aging, or a faulty component before metrology is significantly affected. In such cases, the sensor data may be used to schedule corrective actions consistent with planned downtime (e.g., preventive maintenance operations) before metrology is affected, thereby reducing expensive unscheduled downtime of the manufacturing system. If the sensor data cannot be used reliably, corrective actions may be taken in response to non-optimal metrology data and may result in unscheduled downtime. In some embodiments, unscheduled downtime may result in additional costs, such as express delivery of replacement parts.

[0036] The method and apparatus of the present disclosure solve at least one or more of the above-mentioned deficiencies in conventional solutions. In some embodiments, summary data is enabled and generated, which allows fast processing time and reduces communication bandwidth, complexity of analyzing complete trace data, etc. Generating summary data may include applying statistical methods, machine learning methods, digital twin methods, etc. to trace data (e.g., trace sensor data).

[0037] One or more metrics (e.g., measurement standards) can be used to indicate the system health of the manufacturing equipment. In some embodiments, summary data can be used to generate one or more metric values. The metric value can be used as an indication of further investigation associated with the chamber, process, product, component, etc. to be implemented. In some embodiments, a metric value (e.g., an index, a quality score, a quality index score, etc.) indicating the absence of observable problems can be calculated. In some embodiments, a tool-to-tool match (TTTM) index can be generated to achieve effective isolation of problem systems, tools, products, components, etc. Metric values ​​and indices can be aggregated in a variety of ways to help drill-down analysis, for example, to increase the process of understanding the tool, from determining that there is a problem to determining the corrective action to be taken. For example, a metric related to a large number of outliers (e.g., more than the historical amount) in the sensor data can be used (e.g., as reflected in the summary data). In some embodiments, a metric value that meets a threshold value can trigger further analysis, including generating another metric value associated with a different metric. In some embodiments, a series of metrics can be used as part of the drill-down analysis. For example, a first metric value (eg, index) may indicate that a fabrication chamber is operating suboptimally, another index may be aggregated indicating performance attributable to one subsystem, another index may be generated indicating a process operation in which performance degradation occurred, and so on.

[0038] In some embodiments, data (e.g., summary data) may be provided to one or more models (e.g., machine learning models) to identify root causes, subsystems, corrective actions to take, etc. In some embodiments, data at any stage of the drill-down analysis may be visualized and displayed via a graphical interface.

[0039] In some embodiments (e.g., in response to one or more metrics or indices indicating non-optimal performance), trace sensor data may be utilized for analysis. The trace data may be provided to a machine learning model. The machine learning model may be trained to identify root causes, identify problematic subsystems, identify corrective actions to be performed, etc. The trace data analysis may be displayed via a graphical interface.

[0040] Compared with traditional solutions, various aspects of the present disclosure have technical advantages. The present disclosure brings a more efficient substrate manufacturing process with minimized waste. If a problem occurs in the manufacturing chamber, the method of the present disclosure may be able to perform corrective actions before the processing of the next product begins, before a non-optimal product is submitted for quality measurement, before the quality measurement is completed, etc. In this way, the wasted materials, manufacturing chamber time, and energy provided to the manufacturing process can be minimized. Product quality may be improved because the root causes of product quality and chamber drift can be identified and corrected. The efficiency of producing products of acceptable quality can also be improved because process parameters can be tuned more accurately, thereby improving the cost of materials, energy and time. Aging parts can be identified and marked for replacement or maintenance, thereby reducing unscheduled downtime, the costs associated with express transportation of replacement parts, etc.

[0041] In some embodiments, the present disclosure describes a method that includes receiving trace sensor data associated with a first manufacturing process of a manufacturing chamber. The method also includes processing, by a processing device, the trace sensor data to generate summary data associated with the trace sensor data. The method also includes generating a quality index score based on the summary data. The method also includes providing an alert to a user based on the quality index score. The alert includes an indication that a performance of the manufacturing chamber does not meet a first threshold value.

[0042] In some embodiments, the present disclosure describes a method that includes receiving first data associated with measurements taken by a sensor during a first manufacturing process of a manufacturing chamber. The method also includes receiving second data, wherein the second data includes reference data associated with the first data. The method also includes providing the first data and the second data to a comparison model. The method also includes receiving a similarity score associated with the first data and the second data from the comparison model. The method also includes causing execution of a corrective action in view of the similarity score.

[0043] In some embodiments, the present disclosure describes a method that includes receiving trace sensor data associated with a first manufacturing process of a processing chamber. The method also includes processing the trace sensor data using one or more trained machine learning models, the one or more trained machine learning models generating representations of the trace sensor data, and then generating reconstructed sensor data based on the representations of the trace sensor data. The one or more trained machine learning models output the reconstructed sensor data. The method also includes comparing the trace sensor data with the reconstructed sensor data. The method also includes determining one or more differences between the reconstructed sensor data and the trace sensor data based on the comparison. The method also includes determining whether to recommend a corrective action associated with the processing chamber based on the one or more differences between the trace sensor data and the reconstructed sensor data.

[0044] Figure 1 1 is a block diagram illustrating an exemplary system 100 (exemplary system architecture) according to some embodiments. System 100 includes client device 120, manufacturing equipment 124, sensor 126, metrology equipment 128, prediction server 112, and data storage 140. Prediction server 112 may be part of prediction system 110. Prediction system 110 may also include server machines 170 and 180.

[0045] The sensor 126 may provide sensor data 142 associated with the manufacturing equipment 124 (e.g., associated with the manufacturing equipment 124 producing a corresponding product (e.g., a substrate)). The sensor data 142 may be used for equipment health and / or product health (e.g., product quality). The manufacturing equipment 124 may produce a product according to a recipe or run over a period of time. In some embodiments, the sensor data 142 may include values ​​of one or more of temperature (e.g., heater temperature), spacing (SP), pressure, high frequency radio frequency (HFRF), radio frequency (RF) matching voltage, RF matching current, RF matching capacitor position, voltage of an electrostatic chuck (ESC), actuator position, current, flow rate, power, voltage, etc. The sensor data 142 may be associated with or indicate manufacturing parameters such as hardware parameters (e.g., settings or components, such as size, type, etc.) of the manufacturing equipment 124 or process parameters of the manufacturing equipment 124. Data associated with some hardware parameters may alternatively or additionally be stored as manufacturing parameters 150, which may include historical manufacturing parameters 152 and current manufacturing parameters 154. Manufacturing parameters 150 may indicate input settings of manufacturing equipment (e.g., heater power, air flow, etc.). Sensor data 142 and / or manufacturing parameters 150 may be provided when manufacturing equipment 124 performs a manufacturing process (e.g., may be equipment readings generated during product processing). Sensor data 142 may be different for each product (e.g., each substrate). The substrate may have a property value (e.g., film thickness, film strain, etc.) measured by metrology equipment 128. Metrology data 160 may be a type of data stored in data storage 140.

[0046] In some embodiments, the sensor data 142, the metrology data 160, and / or the manufacturing parameters 150 may be processed (e.g., by the client device 120 and / or the prediction server 112). The processing of the sensor data 142, the metrology data 160, and / or the manufacturing parameters 150 may include generating features. In some embodiments, features are patterns in the sensor data 142, the metrology data 160, and / or the manufacturing parameters 150 (e.g., slope, width, height, peak, etc.), or combinations of values ​​from the sensor data 142, the metrology data 160, and / or the manufacturing parameters 150 (e.g., power generated from voltage and current, etc.). The sensor data 142 may include features, and the prediction component 114 may use these features to perform signal processing and / or obtain the prediction data 168 for corrective action.

[0047] Each instance (e.g., a set) of sensor data 142 may correspond to a product (e.g., a substrate), a group of manufacturing equipment, a type of substrate produced by a manufacturing equipment, etc. Each instance of metrology data 160 and manufacturing parameters 150 may likewise correspond to a product, a group of manufacturing equipment, a type of substrate produced by a manufacturing equipment, etc. The data storage may also store information associating sets of different data types, for example, information indicating that a set of sensor data, a set of metrology data, and a set of manufacturing parameters are all associated with the same product, manufacturing equipment, type of substrate, etc.

[0048] In some embodiments, data associated with the processing of one or more products may be used to generate summary data 162. Summary data 162 may include data that represents characteristics of other data. For example, summary data 162 may be generated from trace sensor data 142. Trace sensor data 142 may include a large amount of data (e.g., data from hundreds of sensors in a tool, each sensor making hundreds or thousands of measurements for each product, etc.). The operation of summary data 162 may be less complex than trace sensor data 142, and may be designed in a manner that information (e.g., information indicating a chamber failure) may be preserved. In some embodiments, data (e.g., trace sensor data 142) may be provided to a processing device (e.g., prediction server 112, client device 120, etc.) to generate summary data 162. Summary data 162 may include, for example, metadata (e.g., tool ID, recipe name, product ID, product information, etc.), contextual data (e.g., sensor ID, step number, timestamp, subsystem, etc.), and / or basic statistics (e.g., mean, maximum, minimum, quartiles, kurtosis, control limits, etc.). In some embodiments, generation of summary data 162 may include using a digital twin of one or more components of manufacturing equipment 124. As used herein, a digital twin is a digital replica of a physical asset, such as a manufactured part, chamber component, or processing chamber. The digital twin includes properties of the physical asset, including (but not limited to) coordinate axis dimensions, weight properties, material properties (e.g., density, surface roughness), electrical properties (e.g., conductivity), optical properties (e.g., reflectivity), and the like. In some embodiments, the digital twin model can be used to generate predictions of the temporal behavior of data (e.g., sensor data 142). Summary data 162 may include indications of how characteristics of the measured data differ from predictions of the digital twin model (e.g., overshoot, rise time, settling time, steady-state value error, etc.). In some embodiments, summary data 162 may be generated based on portions of the input data, such as only a steady-state portion of the summary data 162, only a transient portion of the summary data 162, and the like. In combination Figure 5B Summary data 162 is described in more detail.

[0049] In some embodiments, the prediction system 110 may use machine learning to generate the prediction data 168, such as supervised machine learning (e.g., a machine learning model may be configured to generate labels associated with input data, such as metrology predictions, performance predictions, etc.). In some embodiments, the prediction system 110 may use unsupervised machine learning to generate the prediction data 168 (e.g., a machine learning model may be trained with unlabeled data, such as a model configured to perform clustering, dimensionality reduction, etc.). In some embodiments, the prediction system 110 may use semi-supervised learning to generate the prediction data 168 (e.g., a machine learning model may be trained with labeled and unlabeled input data sets).

[0050] Client devices 120 , manufacturing equipment 124 , sensors 126 , metrology equipment 128 , prediction server 112 , data storage 140 , server machine 170 , and / or server machine 180 may be coupled to one another via network 130 to generate prediction data 168 for corrective action.

[0051] In some embodiments, network 130 is a public network that provides client devices 120 with access to prediction server 112, data storage 140, and other publicly available computing devices. In some embodiments, network 130 is a private network that provides client devices 120 with access to manufacturing equipment 124, sensors 126, metrology equipment 128, data storage 140, and other special computing devices. In some embodiments, the functions of one or more of server machine 170, server machine 180, and / or prediction server 112 may be performed by virtual machines, for example, using cloud-based services. Network 130 may provide access to these virtual machines. Network 130 may include one or more wide area networks (WANs), local area networks (LANs), wired networks (e.g., Ethernet networks), wireless networks (e.g., 802.11 networks or Wi-Fi networks), cellular networks (e.g., Long Term Evolution (LTE) networks), routers, hubs, switches, server computers, cloud computing networks, and / or combinations thereof.

[0052] The client device 120 may include a computing device, such as a personal computer (PC), a laptop, a mobile phone, a smartphone, a tablet, a laptop, a network-connected television ("smart TV"), a network-connected media player (e.g., a Blu-ray player), a set-top box, an over-the-top (OTT) streaming media device, an operator box, etc. The client device 120 may include a corrective action component 122. The corrective action component 122 may receive user input of an indication associated with the manufacturing equipment 124 (e.g., via a graphical user interface (GUI) displayed by the client device 120). In some embodiments, the corrective action component 122 sends an indication to the prediction system 110, receives output (e.g., prediction data 168) from the prediction system 110, determines a corrective action based on the output, and causes the corrective action to be implemented. In some embodiments, the corrective action component 122 obtains sensor data 142 (e.g., current sensor data 146) associated with the manufacturing equipment 124 (e.g., from the data storage 140, etc.), and provides the sensor data 142 (e.g., current sensor data 146) associated with the manufacturing equipment 124 to the prediction system 110. In some embodiments, the corrective action component 122 stores the sensor data 142 in the data storage 140, and the prediction server 112 retrieves the sensor data 142 from the data storage 140. In some embodiments, the prediction server 112 may store the output of the training model 190 (e.g., prediction data 168) in the data storage 140, and the client device 120 may retrieve the output from the data storage 140. In some embodiments, the corrective action component 122 receives an indication of a corrective action from the prediction system 110, and causes the corrective action to be implemented. Each client device 120 may include an operating system that allows a user to perform one or more of generating, viewing, or editing data (eg, instructions associated with manufacturing equipment 124, corrective actions associated with manufacturing equipment 124, etc.).

[0053] In some embodiments, the metrology data 160 corresponds to historical property data of a product (e.g., produced using manufacturing parameters associated with the historical sensor data 144 and the historical manufacturing parameters 152), and the prediction data 168 is associated with the prediction property data (e.g., prediction data of a product that will be produced or has been produced under the conditions recorded by the current sensor data 146 and / or the current manufacturing parameters 154). In some embodiments, the prediction data 168 is predicted metrology data (e.g., virtual metrology data) of a product that will be produced or has been produced according to the conditions recorded as the current sensor data 146 and / or the current manufacturing parameters 154. In some embodiments, the prediction data 168 is an indication of anomalies (e.g., abnormal products, abnormal components, abnormal manufacturing equipment 124, abnormal energy usage, etc.) and, as the case may be, one or more causes of the anomalies. In some embodiments, the prediction data 168 is an indication of changes or drifts over time in certain components of the manufacturing equipment 124, sensors 126, metrology equipment 128, etc. In some embodiments, the prediction data 168 is an indication of the end of life of components of the manufacturing equipment 124, sensors 126, metrology equipment 128, etc.

[0054] Performing a manufacturing process that results in a defective product can be costly in terms of time, energy, products, parts, manufacturing equipment 124, costs of identifying defects and discarding defective products, etc. By inputting sensor data 142 (e.g., manufacturing parameters that are being or will be used to manufacture a product) into the predictive system 110, receiving output of predictive data 168, and taking corrective action based on the predictive data 168, the system 100 may have a technical advantage of avoiding the costs of producing, identifying, and discarding defective products.

[0055] Conducting a manufacturing process that results in a component failure of the manufacturing equipment 124 may be costly in terms of downtime, damaged product, damaged equipment, quickly ordering replacement parts, etc. By inputting sensor data 142 (e.g., indicative of manufacturing parameters that are being used or will be used to manufacture a product) into the predictive system 110, receiving output of predictive data 168, and taking corrective actions (e.g., predicted operational maintenance, such as replacement, handling, cleaning, etc. of a component) based on the predictive data 168, the system 100 may have a technical advantage of avoiding the costs of one or more of unexpected component failures, unscheduled downtime, lost productivity, unexpected equipment failures, product scrapping, etc. Monitoring the performance of components (e.g., manufacturing equipment 124, sensors 126, metrology equipment 128, etc.) over time may provide an indication of components that are degrading.

[0056] Manufacturing parameters may be non-optimal for producing a product, which may have costly consequences such as increased consumption of resources (e.g., energy, coolant, gas, etc.), increased product production time, increased component failures, increased number of defective products, etc. By inputting sensor data 142 into training model 190, receiving output of prediction data 168, and performing (e.g., based on prediction data 168) corrective actions to update manufacturing parameters (e.g., set optimal manufacturing parameters), system 100 may have the technical advantage of using optimal manufacturing parameters (e.g., hardware parameters, process parameters, optimal design) and / or healthy equipment to avoid the costly consequences of non-optimal manufacturing parameters.

[0057] Corrective actions may be associated with one or more of: Computational Process Control (CPC), Statistical Process Control (SPC) (e.g., SPC on electronic components used to determine the controlled process, SPC used to predict component life, SPC compared with a 3σ chart, etc.), Advanced Process Control (APC), model-based process control, preventive operational maintenance, design optimization, updating of manufacturing parameters, updating of manufacturing recipes, feedback control, machine learning modifications, etc.

[0058] In some embodiments, the corrective action includes providing a warning (e.g., providing an alert to stop or not perform a manufacturing process on an additional substrate if the prediction data 168 indicates a predicted anomaly, such as an anomaly of the product, component, or manufacturing equipment 124). In some embodiments, the corrective action includes providing a feedback control (e.g., modifying a manufacturing parameter in response to the prediction data 168 indicating a predicted anomaly). In some embodiments, the taking of the corrective action includes causing an update to one or more manufacturing parameters.

[0059] The manufacturing parameters may include hardware parameters (e.g., information indicating components included in the manufacturing equipment, an indication of a recently replaced component, an indication of a firmware version or update, etc.) and / or process parameters (e.g., temperature, pressure, flow rate, current and / or voltage, airflow, lifting speed, etc.). In some embodiments, the corrective action includes causing preventive operational maintenance (e.g., replacing, treating, cleaning components of the manufacturing equipment 124, etc.). In some embodiments, the corrective action includes causing design optimization (e.g., updating manufacturing parameters for an optimized product, updating a manufacturing process, updating the manufacturing equipment 124, etc.). In some embodiments, the corrective action includes updating a recipe (e.g., changing the timing of instructions for the manufacturing equipment 124 to be in an idle mode, a sleep mode, a preheat mode, etc., adjusting set points for temperature, airflow, plasma generation, etc.).

[0060] The prediction server 112, the server machine 170, and the server machine 180 may each include one or more computing devices, such as a rack server, a router computer, a server computer, a personal computer, a mainframe computer, a laptop computer, a tablet computer, a desktop computer, a graphics processing unit (GPU), an accelerator application-specific integrated circuit (ASIC) (e.g., a tensor processing unit (TPU)), etc.

[0061] Prediction server 112 may include prediction component 114. In some embodiments, prediction component 114 may receive current sensor data 146 and / or current manufacturing parameters 154 (e.g., received from client device 120, retrieved from data storage 140), and generate output (e.g., prediction data 168) for taking corrective action associated with manufacturing equipment 124 based on the current data. In some embodiments, prediction component 114 may use one or more training models 190 to determine output for taking corrective action based on the current data.

[0062] In some embodiments, the model 190 may include a trained physics-based digital twin model. The physics-based model may be able to solve a set of equations describing physical phenomena that may occur in a manufacturing chamber, such as equations governing heat flow, energy balance, gas conduction, mass balance, fluid dynamics, current, etc. In some embodiments, the physics-based model performs calculations of gas conduction in the manufacturing chamber. The manufacturing parameters 150 may be provided to the trained physics-based model. The trained physics-based model may provide as output a modeled property value indicating a condition within the chamber, which corresponds to a sensor 126 disposed within the manufacturing chamber (e.g., manufacturing equipment 124). The output of the physics-based model may be stored in a data storage 140.

[0063] The prediction component 114 of the prediction server 112 may receive summary data 162 generated from data collected by the sensors 126. The prediction component 114 may utilize the summary data 162 to generate prediction data 168. The prediction data 168 may include indications of failures, aging, drift, etc. in the manufacturing equipment 124, the sensors 126, etc. In some embodiments, the summary data may be used to generate an index, for example, a metric indicating the health and / or performance of a tool, a chamber, a group of manufacturing equipment 124, etc. This index may be used for corrective action, for example, may be displayed to a user on a GUI, may cause further analysis to be performed, etc. This index may indicate that further investigation of the source of the failure, aging, drift, etc. is to be conducted. In some embodiments, the index may be generated by providing the summary data to a trained machine learning model (e.g., model 190). In some embodiments, the prediction component 114 may utilize summary data associated with an index indicating that further investigation is to be conducted (e.g., summary data associated with an irregular index value) to isolate a subsystem related to tool performance. In some embodiments, the prediction component 114 uses summary data associated with an index indicating further investigation to perform analysis indicating unexpected drift in sensor data. In some embodiments, trace data associated with an index indicating further investigation may be utilized to isolate subsystems contributing to tool performance. FIG. 5A to FIG. 5G Provides further discussion of methods related to generating summary data, generating indices, and drill-down analysis.

[0064] Historical sensor data 144 and historical manufacturing parameters 152 can be used to train one or more models 190, including physics-based models, digital twin models, machine learning models, etc. In some embodiments, digital twin models can be used to generate summary data. Digital twin models can be generated based on physical principles. In some embodiments, the generated digital twin model may not be able to retrieve some complexities of the manufacturing system, for example, parts may be manufactured slightly differently (for example, within manufacturing tolerances), parts may have aged or drifted, etc. Historical data can be used to refine physics-based or digital twin models to improve accuracy. Historical data can also be used to train one or more machine learning models. In some embodiments, machine learning models can be used to generate indexes, generate data visualizations, generate predictions of metrology data for finished products, etc.

[0065] In some embodiments, sensor data 142 may include data collected from sensors 126 (e.g., measured by metrology equipment 128) during a manufacturing run that produced an acceptable product. A manufacturing run that produced an acceptable product may be referred to as a golden run. Sensor data associated with such a manufacturing run may be stored in data storage 140 as part of historical sensor data 144. Prediction component 114 of prediction server 112 may compare the golden run sensor data, current sensor data 146, and expected sensor data (e.g., as output of a trained physics-based model) to determine whether a component failure, drift, etc. has occurred. In some embodiments, some or all of these operations may alternatively be performed by a different device, such as client device 120, server machine 170, server machine 180, etc.

[0066] In some embodiments, the prediction component 114 may determine which components cause differences between expected sensor data (e.g., historical sensor data, sensor data from different tools, sensor data reflecting manufacturing parameters, etc.), current sensor data 146, and / or golden run sensor data. This may be done, for example, by examining (e.g., through a machine learning model, a statistical model, etc.) historical data of systems that perform as expected and systems that exhibit non-optimal performance. Tagging sensor trace data with an indication of the cause of a failure (e.g., a subsystem responsible for non-optimal performance) may enable a processing device to associate sensor differences (e.g., a particular pattern or feature in the data from a particular sensor) with a failure. The processing logic may be configured to associate certain features or patterns in the sensor data with certain types of failures, certain component aging, drift, or failure, etc. A metric value indicative of performance may be assigned to a sensor, for example, a metric value that aggregates the likelihood that a sensor indicates a failure across multiple operations or runs. Sensor data may be grouped into subsystems. Sensor metric values ​​or indices may be aggregated into subsystem metric values ​​or indices. Aggregated subsystem values ​​(e.g., metric values, indices, etc.) may be used to identify subsystem performance, etc.

[0067] Machine learning models (e.g., one or more of the models 190) can be trained using historical sensor data 144, historical manufacturing parameters 152, metrology data 160 corresponding to operations that measure historical sensor data 142, and the like. One type of machine learning model that can be used to perform some or all of the above tasks is an artificial neural network, such as a deep neural network. An artificial neural network generally includes a feature representation component with a classifier or regression layer that maps features to a target output space. For example, a convolutional neural network (CNN) hosts multiple layers of convolutional filters. Pooling is performed at the lower layers, and nonlinear problems can be solved. A multi-layer perceptron is usually attached above the lower layers to map the top features extracted by the convolutional layers to decisions (e.g., classification outputs). Deep learning is a class of machine learning algorithms that uses a cascade of multiple layers of nonlinear processing units for feature extraction and transformation. Each successive layer uses the output of the previous layer as input. Deep neural networks can learn in a supervised (e.g., classification) and / or unsupervised (e.g., pattern analysis) manner. A deep neural network includes a hierarchy of layers, where different layers learn different levels of representation corresponding to different levels of abstraction. In deep learning, each layer learns to transform the input data of each layer into a slightly more abstract and complex representation. For example, in an image recognition application, the original input may be a matrix of pixels; the first representation layer may abstract the pixels and encode the edges; the second layer may form and encode the arrangement of the edges; the third layer may encode higher-level shapes (e.g., teeth, lips, gums, etc.); and the fourth layer may recognize the scanned character. It is worth noting that the deep learning process can learn which functions can best be placed at which layer. The "depth" in "deep learning" refers to the number of layers through which the data is transformed. More precisely, deep learning systems have a considerable credit assignment path (CAP) depth. CAP is the chain of transformations from input to output. CAP describes the potential causal relationship between input and output. For a feedforward neural network, the depth of CAP may be the depth of the network, and may be the number of hidden layers plus 1. For a recurrent neural network, where a signal may propagate through a layer more than once, the CAP depth may be infinite.

[0068] Using a physics-based digital twin model in conjunction with manufacturing equipment has significant technical advantages over operating manufacturing equipment without such a model. Multiple components (sensors, hardware, processing, etc.) can be linked together in a closed-loop control loop. If there is a problem with any equipment part in the closed-loop control loop, other components may adjust the setpoints, etc., and the readings may return to the target values, but the actual conditions in the chamber may have changed. This leads to less consistent substrate production; loss of productivity; inefficiency in energy, time and materials; and may lead to premature component failures, increased maintenance costs, etc. In some embodiments, multiple data streams indicating the same properties can be monitored to provide additional evidence of the root cause of the offset of sensor data, chamber conditions, etc. In some embodiments, a sensor indicating chamber pressure is monitored. The sensor may include sensors that measure chamber pressure, actuator position of actuators that affect chamber pressure, RF matching voltage, RF matching current, RF matching capacitor position, etc. Some of these properties can be measured by more than one sensor. For example, a manufacturing chamber may be equipped with multiple pressure sensors, a subset of which is involved in the closed-loop control loop, and another subset of which is free to provide measurements of chamber conditions. Physics-based models can capture and analyze subtle changes, variations, drifts, etc. in sensor readings, sensor performance, etc.

[0069] In some embodiments, the prediction component 114 receives the current sensor data 146 and / or the current manufacturing parameters 154, performs signal processing to decompose the current data into a current data set, provides the current data set as an input to the training model 190, and obtains an output from the training model 190 indicating prediction data 168. In some embodiments, the prediction data 168 indicates the metrology data 160 (e.g., a prediction of substrate quality). In some embodiments, the prediction data 168 indicates component health. In some embodiments, the prediction data 168 indicates component performance.

[0070] In some embodiments, the various models discussed in conjunction with model 190 (e.g., physics-based digital twin models, predictive machine learning models, etc.) may be combined in one model (e.g., an integrated model), or may be separate models. Prediction component 114 may receive current sensor data 146 and current manufacturing parameters 154, provide the data to training model 190, and receive information indicating how much several components in the manufacturing chamber have drifted from the previous performance of several components. Data may be passed between several different models included in model 190 and prediction component 114. In some embodiments, some or all of these operations may be performed by different devices, such as client device 120, server machine 170, server machine 180, etc. Those skilled in the art will understand that changes in data flows, which components perform which processes, which models are provided with which data, etc. are all within the scope of the present disclosure.

[0071] The data storage 140 may be a memory (e.g., random access memory), a drive (e.g., a hard drive, a flash drive), a database system, or other type of component or device capable of storing data. The data storage 140 may include multiple storage components (e.g., multiple drives or multiple databases) that may span multiple computing devices (e.g., multiple server computers). The data storage 140 may store sensor data 142, manufacturing parameters 150, metrology data 160, summary data 162, and prediction data 168. The sensor data 142 may include historical sensor data 144 and current sensor data 146. The sensor data may include a temporal trace of sensor data over the duration of a manufacturing process, an association of data with physical sensors, pre-processed data (such as averages and synthetic data), and data indicating changes in sensor performance over time (i.e., many manufacturing processes). The manufacturing parameters 150 and the metrology data 160 may contain similar features. The historical sensor data 144 and the historical manufacturing parameters 152 may be historical data (e.g., at least a portion of the training model 190). The current sensor data 146 may be current data (e.g., at least a portion to be input into the learning model 190 following the historical data) for which the prediction data 168 is to be generated (e.g., for use in taking corrective actions). The summary data 162 may include processed sensor data 142. The summary data may contain information indicating corrective actions to be performed in association with the manufacturing equipment 124. The summary data may be less operationally intensive than the trace sensor data.

[0072] In some embodiments, the prediction system 110 also includes a server machine 170 and a server machine 180. The server machine 170 includes a data set generator 172, which is capable of generating a data set (e.g., a set of data inputs and a set of target outputs) to train, validate, and / or test the model 190, which includes a physics-based digital twin model and, in some embodiments, one or more machine learning models. Figure 3 , Figure 4 and Figure 5ASome operations of the data set generator 172 are described in detail. In some embodiments, the data set generator 172 may divide the historical data (e.g., historical sensor data 144, historical manufacturing parameters 152, summary data 162 stored in the data storage 140) into a training set (e.g., 60 percent of the historical data), a validation set (e.g., 20 percent of the historical data), and a test set (e.g., 20 percent of the historical data). In some embodiments, the prediction system 110 (e.g., via the prediction component 114) generates multiple sets of features. For example, a first set of features may correspond to a first set of sensor data types corresponding to each data set (e.g., a training set, a validation set, and a test set) (e.g., from a first set of sensors, a first combination of values ​​from the first set of sensors, a first pattern in values ​​from the first set of sensors), and a second set of features may correspond to a second set of sensor data types corresponding to each data set (e.g., from a second set of sensors different from the first set of sensors, a second combination of values ​​different from the first combination, a second pattern different from the first pattern).

[0073] The server machine 180 includes a training engine 182, a validation engine 184, a selection engine 185, and / or a test engine 186. An engine (e.g., a training engine 182, a validation engine 184, a selection engine 185, and a test engine 186) may refer to hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, a processing device, etc.), software (such as instructions running on a processing device, a general purpose computer system, or a dedicated machine), firmware, microcode, or a combination of the above. The training engine 182 may be capable of training a model 190 using one or more sets of features associated with a training set from the data set generator 172. The training engine 182 may generate multiple training models 190, where each training model 190 corresponds to a different set of features of the training set (e.g., sensor data from a different set of sensors). For example, a first training model may be trained using all features (e.g., X1-X5), a second training model may be trained using a first subset of features (e.g., X1, X2, X4), and a third training model may be trained using a second subset of features (e.g., X1, X3, X4, and X5) that may partially overlap the first subset of features. The dataset generator 172 may receive the output of a training model (e.g., a physics-based digital twin model, a machine learning model), collect the data into training, validation, and test datasets, and use the datasets to train a second model (e.g., a machine learning model configured to output prediction data, corrective actions, etc., etc.).

[0074] The validation engine 184 may be able to validate the training models 190 using a corresponding feature set of a validation set from the dataset generator 172. For example, a first training model 190 trained using a first set of features of the training set may be validated using a first set of features of the validation set. The validation engine 184 may determine the accuracy of each training model 190 based on the corresponding feature set of the validation set. The validation engine 184 may discard training models 190 having an accuracy that does not meet a threshold accuracy. In some embodiments, the selection engine 185 may be able to select one or more training models 190 having an accuracy that meets a threshold accuracy. In some embodiments, the selection engine 185 may be able to select a training model 190 having the highest accuracy among the training models 190.

[0075] The testing engine 186 may be able to test the training model 190 using a corresponding feature set of the test set from the dataset generator 172. For example, a first training model 190 trained using a first set of features of the training set may be tested using a first set of features of the test set. The testing engine 186 may determine the training model 190 with the highest accuracy among all the training models based on the test set.

[0076] In the case of a machine learning model, the model 190 may refer to an artifact model created by the training engine 182 using a training set that includes data inputs and corresponding target outputs (correct answers for the corresponding training inputs). Patterns in the data set that map the data inputs to the target outputs (correct answers) may be found, and a mapping that obtains these patterns is provided to the machine learning model 190. The machine learning model 190 may use one or more of a support vector machine (SVM), a radial basis function (RBF), a clumping method, supervised machine learning, semi-supervised machine learning, and unsupervised machine learning, a k-Nearest Neighbor algorithm (k-NN), a linear regression, a random forest, a neural network (e.g., an artificial neural network), and the like.

[0077] The prediction component 114 may provide the current sensor data 146 and / or summary data 162 to the model 190, and may run the model 190 on the inputs to obtain one or more outputs. The prediction component 114 may be able to determine (e.g., extract) prediction data 168 from the outputs of the model 190, and may determine (e.g., extract) confidence data from the outputs indicating a confidence level that the prediction data 168 is an accurate predictor of a process associated with the input data of a product produced or to be produced using the manufacturing equipment 124 at the current sensor data 146 and / or current manufacturing parameters 154. The prediction component 114 or the corrective action component 122 may use the confidence data to determine whether to cause a corrective action associated with the manufacturing equipment 124 based on the prediction data 168.

[0078] The confidence data may include or indicate a confidence level that the prediction data 168 is an accurate prediction of a product or component associated with at least a portion of the input data. In one example, the confidence level is a real number between 0 and 1 (inclusive), where 0 represents no confidence that the component health prediction data 168 of the component of the manufacturing equipment 124 is an accurate prediction of the processed product, and 1 represents an absolute confidence that the component health prediction data 168 of the component of the manufacturing equipment 124 accurately predicts the property of the processed product. In response to the confidence data indicating that the confidence level is below a threshold level for a predetermined number of instances (e.g., a percentage of instances, a frequency of instances, a total number of instances, etc.), the prediction component 114 may cause the training model 190 to be retrained (e.g., based on the current sensor data 146, the current manufacturing parameters 154, etc.).

[0079] For purposes of illustration and not limitation, aspects of the present disclosure describe training one or more machine learning models 190 using historical data (e.g., historical sensor data 144, historical manufacturing parameters 152) and inputting current data (e.g., current sensor data 146, current manufacturing parameters 154, etc.) into one or more trained machine learning models to determine prediction data 168. In other embodiments, a heuristic model or a rule-based model is used to determine prediction data 168 (e.g., without using a trained machine learning model). The prediction component 114 can monitor the historical sensor data 144, the historical manufacturing parameters 152, the summary data 162, and the metrology data 160. Figure 3 Any information described by data input 310 may be monitored or otherwise used in a heuristic or rule-based model.

[0080] In some embodiments, the functionality of client device 120, prediction server 112, server machine 170, and server machine 180 may be provided by a smaller number of machines. For example, in some embodiments, server machines 170 and 180 may be integrated into a single machine, while in some other embodiments server machine 170, server machine 180, and prediction server 112 may be integrated into a single machine. In some embodiments, client device 120 and prediction server 112 may be integrated into a single machine.

[0081] In general, the functions described in one embodiment as being performed by client device 120, prediction server 112, server machine 170, and server machine 180 may also be performed on prediction server 112 in other embodiments, as appropriate. In addition, the functions attributed to a particular component may be performed by different or multiple components operating together. For example, in some embodiments, prediction server 112 may determine a corrective action based on prediction data 168. In another example, client device 120 may determine prediction data 168 based on output from a trained machine learning model, a physics-based model, etc.

[0082] Furthermore, the functionality of a particular component may be implemented by different or multiple components operating together. One or more of the prediction server 112, server machine 170, or server machine 180 may be accessed as a service provided to other systems or devices via an appropriate application programming interface (API).

[0083] In an embodiment, a "user" may be represented as a single individual. However, other embodiments of the present disclosure encompass a "user" that is an entity controlled by multiple users and / or automation sources. For example, a group of individual users united as a group of administrators may be considered a "user."

[0084] The embodiments of the present disclosure may be applied to data quality assessment, feature enhancement, model evaluation, virtual metrology (VM), predictive maintenance (PdM), extreme optimization, etc.

[0085] Figure 2A cross-sectional view of a fabrication chamber 200 (e.g., a semiconductor wafer fabrication chamber) according to some aspects of the present disclosure is depicted. The fabrication chamber 200 may be one or more of an etch chamber (e.g., a plasma etch chamber), a deposition chamber (including atomic layer deposition, chemical vapor deposition, physical vapor deposition, or plasma enhanced versions of the foregoing), an annealing chamber, and the like. For example, the fabrication chamber 200 may be a chamber for a plasma etcher, a plasma cleaner, and the like. Examples of chamber components may include a substrate support assembly 204, an electrostatic chuck, a ring (e.g., a process kit ring), a chamber wall, a pedestal, a showerhead 206, a gas distribution plate, a liner, a liner set, a shield, a plasma shield, a flow equalizer, a cooling pedestal, a chamber viewport, a chamber lid, a nozzle, and the like.

[0086] In one embodiment, the manufacturing chamber 200 includes a chamber body 208 and a showerhead 206 that encloses an interior volume 210. In some chambers, the showerhead 206 may be replaced by a cover and nozzle. The chamber body 208 may be made of aluminum, stainless steel, or other suitable materials. The chamber body 208 generally includes sidewalls 212 and a bottom 214.

[0087] An exhaust port 216 may be defined in the chamber body 208 and may couple the interior volume 210 to a pump system 218. The pump system 218 may include one or more pumps and valves for evacuating and regulating the pressure of the interior volume 210 of the fabrication chamber 200. An actuator for controlling the flow of gas out of the chamber and / or the pressure in the chamber may be disposed at or near the exhaust port 216.

[0088] The showerhead 206 may be supported on a sidewall 212 of the chamber body 208 or on the top of the chamber body. The showerhead 206 (or a lid in some embodiments) may be opened to allow access to the interior volume 210 of the manufacturing chamber 200 and may provide a seal for the manufacturing chamber 200 when closed. A gas panel 220 may be coupled to the manufacturing chamber 200 to provide a process gas or a cleaning gas to the interior volume 210 via the showerhead 206 (or a lid and a nozzle). The showerhead 206 may include a plurality of gas delivery holes extending through the showerhead 206. Examples of process gases that may be used to process substrates in the manufacturing chamber 200 include halogen-containing gases such as C2F6, SF6, SiCl4, HBr, NF3, CF4, CHF3, F2, Cl2, CCl4, BCl3, and SiF4, among others, and other gases such as O2 or N2O. Examples of carrier gases include N2, He, Ar, and other gases that are inert to the process gas (e.g., non-reactive gases).

[0089] The substrate support assembly 204 is disposed in the interior volume 210 of the manufacturing chamber 200 below the showerhead 206. In some embodiments, the substrate support assembly 204 includes a pedestal 222 and a shaft 224. The substrate support assembly 204 supports the substrate during processing. In some embodiments, one or more heaters 226 and reflectors 228 are also disposed within the manufacturing chamber 200.

[0090] Sensors are used to measure property values ​​in the manufacturing chamber 200. Sensors associated with the manufacturing chamber 200 may include temperature sensors, pressure sensors (in some embodiments, there may be at least two pressure sensors), flow sensors, etc. In some embodiments, using a physics-based digital twin model of the manufacturing chamber 200, faulty components may be identified and corrective actions may be taken.

[0091] In some embodiments, chamber 200 may include a plurality of sensors that collect time trace data. The time trace data may be sent to a data storage device (e.g., Figure 1 The data storage 140 of the embodiment of the present invention can be used to store the summary data of the time trace sensor data. Summary data can be generated based on the time trace sensor data. Processing of the summary data and / or the time trace data can generate data indicating chambers, subsystems, components, etc. that have problems (e.g., failures, drift, aging, etc.). The data processing can be performed by one or more models, including physics-based models, machine learning models, digital twin models, etc. The summary data can be used to generate an index. The value of the index can indicate the performance of the tool, component, fleet, etc.

[0092] Figure 3 For example, a dataset generator 372 according to some embodiments (e.g., Figure 1 172) for creating a dataset for use in applying an input to a model (e.g., Figure 1 The data set generator 372 may be a Figure 1 Part of the server machine 170. In some embodiments, the model 190 may include multiple machine learning models, physics-based models, digital twin models, etc. Each model may have its own dataset generator, or multiple models may share a dataset generator. Figure 3 The operation of one dataset generator (associated with a model for mapping sensor data to performance data) is described in detail. Any other model may be associated with a similar dataset generator. For other models, the input and output data may be different from the combined Figure 3 Describe the input and output.

[0093] A data set generator 372 (e.g., Figure 1 The system 300 of the dataset generator 172 of the present invention is a machine learning model (e.g. Figure 1The data set generator 372 may use the historical manufacturing parameters 352 (e.g., Figure 1 The system 300 may be used to generate a data set (e.g., data input 310) using historical manufacturing parameters 152 of the manufacturing process, historical sensor data, summary sensor data, etc. The system 300 may be used to generate a data set to train, test, and validate a machine learning model. In some embodiments, performance data 322 is provided to the machine learning model as a target output 320. The performance data may indicate properties of a finished product, such as metrology data of a processed substrate. In some embodiments, the performance data may indicate a condition of one or more components of a manufacturing device, such as failure, aging, drift, recommended maintenance, etc. In some embodiments, the data set generator 372 may be configured to generate a data set of a physics-based model of a component of a manufacturing device. The model may be trained to refine the output of the physics-based model by using the sensor data as the target output. In some embodiments, sensor data associated with a successful process (e.g., golden run data from a process that produces a product within manufacturing tolerances, golden run data from a recently qualified chamber, etc.) may be used as input 310 or target output 320.

[0094] In some embodiments, the data set generator 372 generates a data set (e.g., a training set, a validation set, a test set) including one or more data inputs 310 (e.g., training inputs, validation inputs, test inputs), and may include one or more target outputs 320 corresponding to the data inputs 310. The data set may also include mapping data that maps the data inputs 310 to the target outputs 320. The data inputs 310 may also be referred to as "features," "attributes," or "information." In some embodiments, the data set generator 372 may provide the data set to the training engine 182, the validation engine 184, or the testing engine 186, where the data set is used to train, validate, or test one or more models 190 (e.g., one of the models included in the model 190, the ensemble model 190, etc.). In some embodiments, the model 190 may include a physics-based digital twin model and a model for analyzing the output of the physics-based model. In some embodiments, the dataset generator 372 may provide inputs to the physics-based model as training inputs (e.g., manufacturing parameters), and may provide target output data associated with the output of the second model (e.g., component contribution data from the machine learning model), and the model 190 may be trained using these associated datasets. Figure 5A Some embodiments of generating a training set are further described.

[0095] In some embodiments, the data set generator 372 generates data input 310 without generating target output 320 to provide a training data set for an unsupervised machine learning model, such as a clustering model, an autoencoder, a dimensionality reduction model, etc. Figure 7The operation of such a model is further described. In some embodiments, the data set generator 372 generates data input 310 and target output 320 (e.g., to train an unsupervised or semi-supervised model). In some embodiments, the data input 310 may include one or more sets of data. As an example, the system 300 may generate a sensor data set that may include one or more sensor data from one or more types of sensors, a combination of sensor data from one or more types of sensors, a pattern of sensor data from one or more types of sensors, a manufacturing parameter from one or more manufacturing parameters, a combination of some manufacturing parameter data and some sensor data, summary data, etc.

[0096] In some embodiments, the dataset generator 372 may generate a first data input corresponding to a first set of historical sensor data 344A and / or historical manufacturing parameters 352A to train, validate and / or test a first machine learning model, and the dataset generator 372 may generate a second data input corresponding to a second set of historical sensor data 344B and / or historical manufacturing parameter set 352B to train, validate and / or test a second machine learning model.

[0097] The data input 310 and target output 320 used to train, validate and / or test the machine learning model may include information of a specific manufacturing chamber (e.g., a specific substrate manufacturing tool). For example, historical manufacturing parameters 352, historical sensor data 344, summary data, and performance data 322 may be associated with the same manufacturing chamber.

[0098] In some embodiments, the information used to train the machine learning model may come from a particular type of manufacturing equipment in a manufacturing facility with particular characteristics (e.g., Figure 1 The information used to train the model may be components from two or more manufacturing facilities, and may allow the trained machine learning model to determine results for a particular group of manufacturing facilities 124 based on input from current sensor data (e.g., current sensor data 146) associated with one or more components that share the characteristics of the particular group. In some embodiments, the information used to train the model may be components from two or more manufacturing facilities, and may allow the trained machine learning model to determine results for components based on input from one manufacturing facility.

[0099] In some embodiments, after a data set is generated and used to train, validate, or test the machine learning model 190, the model 190 may be further trained, validated, tested, or adjusted (e.g., adjusting weights or parameters associated with the input data of the model 190, such as parameters in a physics-based model or connection weights in a neural network).

[0100] Figure 4A To illustrate a method for generating output data (e.g., Figure 1In some embodiments, the system 400 can be used in conjunction with a machine learning model (e.g., Figure 1 In some embodiments, a system similar to system 400 can be used in conjunction with a physics-based model to determine correlations between sensors and performance of a tool, chamber, or set of manufacturing equipment, which can affect the performance of corrective actions (e.g., corrective actions can be taken in light of the simulated sensor data). In some embodiments, a system similar to system 400 can be used in conjunction with a physics-based model to determine corrective actions associated with manufacturing equipment.

[0101] At block 410, the system 400 (e.g., Figure 1 Components of the forecasting system 110) perform historical data 464 (e.g., for Figure 1 The data partitioning (e.g., via Figure 1 The data set generator 172 of the server machine 170 of the embodiment of the present invention is used to generate a training set 402, a validation set 404 and a test set 406. For example, the training set may be 60% of the historical data, the validation set may be 20% of the historical data, and the test set may be 20% of the historical data.

[0102] The generation of training set 402, validation set 404, and test set 406 may be customized for a particular application. For example, the training set may be 60% of the historical data, the validation set may be 20% of the historical data, and the test set may be 20% of the historical data. System 400 may generate multiple sets of features for each of the training set, validation set, and test set. For example, if historical data 464 includes data from 20 sensors (e.g., Figure 1 For example, if the sensor data of sensor 126 of the embodiment of the present invention is a first set of features including sensors 1-10 and a second set of features including sensors 11-20, the sensor data can be divided into a first set of features including sensors 1-10 and a second set of features including sensors 11-20. The manufacturing parameters can also be divided into multiple groups, such as a first set of manufacturing parameters including parameters 1-5 and a second set of manufacturing parameters including parameters 6-10. The target input, the target output, both, or neither can be divided into multiple sets. Multiple machine learning models, physics-based models, etc. can be trained on different data sets.

[0103] At block 412, system 400 performs model training using training set 402 (e.g., via Figure 1Training engine 182 of the machine learning model and / or physics-based model (e.g., digital twin) can be implemented in a supervised learning manner, which involves feeding a training dataset including labeled inputs through the model, observing its output, defining an error (by measuring the difference between the output and the labeled value), and using techniques such as deep gradient descent and backpropagation to tune the model's weights so that the error is minimized. In many applications, repeating this process over many labeled inputs in the training dataset produces a model that produces correct outputs when presented with inputs that are different from those present in the training dataset.

[0104] For each training data item in the training data set, the training data item can be input into a model (e.g., into a machine learning model). The model can then process the input training data item (e.g., a process recipe from a historical run) to produce an output. The output can include, for example, predicted performance data for a process product. The output can be compared to a label for the training data item (e.g., a measured actual performance metric).

[0105] The processing logic may then compare the generated output (e.g., predicted performance) to the labels (e.g., actual performance measurements) included in the training data items. The processing logic determines an error (i.e., classification error) based on the difference between the output and the labels. The processing logic adjusts one or more weights and / or values ​​of the model based on the error.

[0106] In the case of training a neural network, an error term or increment may be determined for each node in the artificial neural network. Based on this error, the artificial neural network adjusts one or more parameters of the artificial neural network (the weights of one or more inputs of the node) for one or more nodes of the artificial neural network. The parameters may be updated in a back-propagation manner, so that the nodes of the highest layer are updated first, followed by the nodes of the next layer, and so on. An artificial neural network contains multiple layers of "neurons", each of which receives input values ​​from the neurons of the previous layer. The parameters of each neuron include weights associated with the values ​​received from each neuron of the previous layer. Thus, adjusting the parameters may include adjusting the weights assigned to each input of one or more neurons of one or more layers in the artificial neural network.

[0107] The system 400 may train multiple models using multiple sets of features of the training set 402 (e.g., a first set of features in the training set 402, a second set of features in the training set 402, etc.). For example, the system 400 may train models to generate a first training model using a first set of features in the training set (e.g., sensor data from sensors 1-10), and to generate a second training model using a second set of features in the training set (e.g., sensor data from sensors 11-20). In some embodiments, the first training model and the second training model may be combined to generate a third training model (e.g., the third training model itself may be a better predictor than the first or second trained model). In some embodiments, the feature sets used to compare models may overlap (e.g., the first set of features is sensor data from sensors 1-15, and the second set of features is sensor data from sensors 5-20). In some embodiments, hundreds of models may be generated, including models and model combinations with various feature arrangements. In some embodiments, hundreds or more models may be stacked together to generate an ensemble model with greater predictive power than a single model alone.

[0108] At block 414, system 400 performs model validation using validation set 404 (e.g., via Figure 1 404). The system 400 may validate each training model using a corresponding set of features of the validation set 404. For example, the system 400 may validate a first training model using a first set of features in the validation set (e.g., sensor data from sensors 1-10), and validate a second training model using a second set of features in the validation set (e.g., sensor data from sensors 11-20). In some embodiments, the system 400 may validate hundreds of models generated at block 412 (e.g., models with various feature arrangements, combinations of models, etc.). At block 414, the system 400 may determine the accuracy of each of the one or more training models (e.g., via model validation), and may determine whether one or more of the training models have an accuracy that meets a threshold accuracy. In response to determining that none of the training models have an accuracy that meets the threshold accuracy, the process returns to block 412, where the system 400 performs model training using a different set of features from the training set. In response to determining that one or more training models have an accuracy that meets the threshold accuracy, the process continues to block 416. The system 400 may discard trained models (eg, based on a validation set) that have an accuracy below a threshold accuracy.

[0109] At block 416, the system 400 performs a model selection (e.g., via Figure 1The selection engine 185 of block 414 may be used to determine which of the one or more training models that meet the threshold accuracy has the highest accuracy (e.g., the selected model 408 based on the verification of block 414). In response to determining that the two or more training models that meet the threshold accuracy have the same accuracy, the process may return to block 412, where the system 400 uses a further refined training set corresponding to the further refined feature set to perform model training to determine the training model with the highest accuracy.

[0110] At block 418, system 400 performs model testing using test set 406 (e.g., via Figure 1 406). In response to the accuracy of the selected model 408 not meeting the threshold accuracy (e.g., the selected model 408 is overfit to the training set 402 and / or the validation set 404 and is not applicable to other data sets, such as the test set 406), the process continues to block 412, where the system 400 uses a different training set corresponding to a different feature set (e.g., sensor data from a different sensor) to perform model training (e.g., retraining). In response to determining that the selected model 408 has an accuracy that meets the threshold accuracy based on the test set 406, the process continues to block 420. At least in block 412 , the model may learn patterns in the historical data to make predictions, and in block 418 , the system 400 may apply the model to the remaining data (eg, the test set 406 ) to test the predictions.

[0111] At block 420, the system 400 receives current sensor data 446 (e.g., Figure 1 146 ) and determining (e.g., extracting) predicted performance data 468 (e.g., Figure 1 The predicted performance data 468 may be used to compare Figure 1 In some embodiments, the current sensor data 446 may correspond to the same type of features in the historical manufacturing parameter data. In some embodiments, the current sensor data 446 corresponds to the same type of features as a subset of the type of features in the historical manufacturing parameter data used to train the selected model 408.

[0112] In some embodiments, current data is received. The current data may include current performance data 460 (e.g., Figure 1The new model is trained based on the current sensor data 446 and the current performance data 460.

[0113] In some embodiments, one or more of actions 410-420 may occur in various orders and / or with other actions not presented and described herein. In some embodiments, one or more of actions 410-420 may not be performed. For example, in some embodiments, one or more of the data partitioning of block 410, the model validation of block 414, the model selection of block 416, or the model testing of block 418 may not be performed.

[0114] Similar to Figure 4A The system 400 of the system can be configured for training, validation, testing and use of a physics-based digital twin model. The physics-based model is configured to accept manufacturing parameters as input (e.g., set points provided to manufacturing equipment) and provide simulated sensor data (e.g., predicted sensor data) as output. Other models related to this physics-based model may follow similar data flows in training, validation, testing and use. In some embodiments, the model may receive simulated sensor data and measured sensor data, and is configured to output a list of components predicted to contribute to the difference between the two data sets. This model can provide an estimate of the contribution of each component to the change. For example, this model can provide an estimate of power transfer to the component. Sensor data describing power transfer can be provided to the physics-based model, or the output of the physics-based model and sensor data can be provided to another model. Differences between predicted data and measured data may indicate component aging, failure, drift, etc. The partitioning, training, validation, selection, testing and use blocks of the system 400 can be performed similarly to train additional models using data of different data types. Retraining can also be performed using current simulated sensor data and current measured sensor data. In some embodiments, golden operational data may also be provided for partitioning, training, validation, selection, testing, and use of machine learning or physics-based models.

[0115] Figure 4B 4 is a flow chart of a method 430 for performing TTTM analysis using trace sensor data according to some embodiments. In some embodiments, the operations of method 430 may be performed after training various machine learning models, statistical models, digital twin models, etc. to be used as part of the TTTM analysis module.

[0116] At block 431, a configuration operation is performed. In some embodiments, the configuration operation is performed by a user. In some embodiments, the user may interact with a graphical user interface to configure the TTTM system. In some embodiments, another method such as text input, command line control, etc. may be used to configure the TTTM system. In some embodiments, configuration settings (e.g., a file or archive containing data indicating configuration settings) may be referenced by a processing device that repeatedly performs TTTM analysis throughout the analysis process.

[0117] Configuration settings may include label information and conventions, including dates, recipe labels, product labels, equipment and / or sensor labels, etc. Configuration settings may include instructions for data collected using one or more tools, one or more sensors in conjunction with one or more processing runs or products, etc. Configuration settings may include instructions for the type of analysis to be performed (or not performed) on the processing device. Configuration settings may indicate what metrics to use, what types of summary data to generate, what models to use in the analysis (e.g., machine learning models), what features of the TTTM software module to use in the analysis (e.g., whether to perform normal value drift detection or time trace analysis), etc. Configuration settings may include instructions for conditional thresholds for performing further analysis, such as control limits for classifying data points as outliers, the number of thresholds for triggering drill-down analysis, etc. Configuration settings may include instructions for metric aggregation and / or index aggregation to be performed by the TTTM module. In some embodiments, a user may be able to store one or more "favorite" configuration settings for future TTTM analysis. In some embodiments, configuration settings may include options for visualization, including the type of plot to be generated (e.g., box plot, histogram, scatter plot, etc.), the number of plots to be generated, a threshold metric or index value that triggers the generation of a plot, design elements of the generated plot, whether to calculate and / or display a best fit line or trend line on the plot, etc.

[0118] Once the configuration settings are determined (or default settings, favorite settings, etc. are selected), the automatic processing procedure 432 is performed. In some embodiments, the TTTM module (guided by the configuration settings) independently performs many operations to produce an analysis of the sensor data. FIG. 5B to FIG. 5I The operations of automatic processing program 432 are discussed in more detail, and a brief overview of these operations is included here.

[0119] At block 433, a data collection operation is performed by the processing device. Based on the configuration settings, data is retrieved by the processing device for analysis. The retrieved data may include trace sensor data, pre-processed sensor data, manufacturing data (e.g., hardware or processing parameters, recipe information, etc.), summary data, etc.

[0120] At block 434, a data processing operation is performed by a processing device. In some embodiments, data processing is performed according to configuration settings. Data processing may include generating metrics, indices, aggregated metrics and indices, trace analysis, drift analysis, etc. Data processing may include providing data (e.g., data collected at block 433) to one or more models to generate outputs. Data processing models may include machine learning models, digital twin models, statistical models, etc. Figure 5D Data processing operations are discussed in more detail.

[0121] At block 435, processing logic performs visualization operations. Visualization operations may be performed according to configuration settings. In some embodiments, generation of visualizations may be triggered by conditions satisfied during data processing, such as a metric value exceeding a threshold value, a portion of data points being classified as outliers, etc. In some embodiments, visualizations may be generated at multiple stages of the analysis, for example, TTTM index values ​​may be plotted, and one or more sets of TTTM index values ​​may also be plotted. Any convenient plotting scheme may be used for visualization, such as box plots, histograms, scatter plots, pie charts, etc. In combination FIG. 6A to FIG. 6G Discuss visualization in more detail.

[0122] At block 436, processing logic performs analysis and reporting operations. In some embodiments, the operations of block 436 are provided to a user. In some embodiments, the operations of block 436 may be triggered automatically (e.g., a case evaluation process for evaluating the performance of the TTTM module may occur periodically). In some embodiments, the operations of block 436 may be triggered by a user. In some embodiments, the results of the operations of block 436 may affect configuration settings for future analysis (e.g., Figure 4B 431).

[0123] Figure 4C 4 is a flow chart describing a method 450 for generating summary data according to some embodiments. The generation of summary data may begin with the collection of trace data. As an example, method 450 operates on a complete trace from a single run and a single sensor. In other embodiments, data from multiple sensors, multiple runs, etc. may be used to generate summary data. In some embodiments, trace data may be separated into processing operations for storage, and the separated data may be used to generate summary data. In some embodiments, pre-processing may be performed on the data, such as smoothing, interpolation, normalization, etc. Pre-processing may be performed at various stages of the analysis.

[0124] At block 451, the complete trace data from a single run is segmented into processing operations by processing logic. In some embodiments, the complete trace data and the data segmented into individual processing operations (e.g., steps of a processing recipe) may be provided to processing logic to perform basic summary operations. The data may be segmented into processing operations based on timestamps of data points, behavior of trace data, characteristics of reference trace data, etc. The data segmented into processing operations may be provided to a processing device for further segmentation into transient and steady-state portions. In some embodiments, the processing logic may be coupled to a processor to perform a basic summary operation. Figure 4C It is possible to perform various operations in an order different from that shown and to be within the scope of the present disclosure.

[0125] At block 452, the trace data segmented into processing operations is further segmented into transient and steady-state portions. Fig. 6A A visualization of the splitting of the transient portion is depicted in FIG. In some embodiments, a process parameter such as temperature or voltage may have an operating set point. It may take a period of time to reach such a set point, at which time the property value changes rapidly. Periods in which the property value changes rapidly can be separated as the transient portion of the trace data, while periods in which the property value is more stable can be separated as the steady-state portion of the trace data. In some embodiments, a process recipe may require that a property (e.g., temperature) change in a controlled manner (e.g., a ramp up) over a period of time. Such periods that are neither steady-state nor transient can also be separated and analyzed. In some embodiments, the ramp data can be analyzed as steady-state data. In some embodiments, the ramp data can be analyzed as transient data. In some embodiments, the ramp data can be analyzed in a different manner than steady-state or transient data. In some embodiments, features from transient data analysis and / or steady-state analysis can be used to analyze the ramp data.

[0126] The transient and steady-state portions of the trace data may be separated by the processing device. In some embodiments, the separation of the transient and steady-state portions may be performed according to configuration settings. The identification of the transient portion of the trace data may include determining the slope of the trace data, one or more statistical measures of the data (e.g., standard deviation, range, etc.), based on prior knowledge (e.g., steady-state time estimates derived from golden run data), etc. In some embodiments, a sliding window is used to separate the transient and steady-state portions of the trace data. A window that is smaller (e.g., containing fewer data points) than the trace data associated with the operation may be analyzed. In some embodiments, the points within the window may be determined as part of the steady-state portion or part of the transient portion. In some embodiments, if the standard deviation of the points within the window is greater than a threshold value, the points in the window may be determined as part of the transient portion. The window may then be moved to include different subsets of the data points of the processing operation trace data. The size of the window, the measure used to determine whether a portion of the trace data is transient (e.g., slope, standard deviation, quartile, range, etc.), the threshold value for determining the transient, etc. may all be performed according to configuration settings. In some embodiments, the interface between the portion designated as steady state and the portion designated as transient may be subject to further processing, such as changing the window size, changing the determination metric, changing the threshold metric value, etc. The data divided into transient and steady state (and ramp or another type) portions is provided to processing logic to generate summary data. In some embodiments, the steady state portion and the transient portion are provided to processing logic to perform basic data summarization operations (block 454). In some embodiments, the transient portion is provided to processing logic to perform digital twin summary operations (block 453).

[0127] At block 453, processing logic performs a digital twin summary operation. In some embodiments, the digital twin summary operation is performed in conjunction with the transient portion of the trace data. In some embodiments, a digital twin model of a physical asset is generated. The physical asset may include one or more components of a manufacturing system. The performance of the physical asset may be modeled by the digital twin, and the modeled behavior may be compared to the behavior recorded by one or more sensors associated with the manufacturing system. The visualization of the digital twin data used to generate the summary data may be performed at Figure 6B Found in.

[0128] Various features of the transient portion can be compared with the digital twin output to generate summary data. In some systems, the property value may exceed (e.g., overshoot) the set point before settling to the set value. The properties of the overshoot region of the trace data (e.g., overshoot value, overshoot time length, time to reach the maximum or minimum overshoot value, etc.) can be used to generate summary data. In some embodiments, the property value may show the rise time to reach the set point. The rise time property of the sensor data can be compared with the digital twin data to generate summary data, such as the time required for the property value to reach 50% of the indicated value, 90% of the indicated value, etc. In some embodiments, the property value may take some time to settle to the set point (e.g., after overshoot). The settling time can be used to generate summary data (e.g., until the value remains within 5% of the set point, within 2% of the set point, etc.). In some embodiments, the final property value reached may be different from the set point. Summary data can be generated based on the difference. In some embodiments, the properties of the closed-loop control system can be modeled by the digital twin model. Properties such as gain, phase, and bandwidth of the closed-loop control system can also be monitored and used to generate summary data. In some embodiments, the property value may oscillate (e.g., when the property value approaches a set point, the value may go above and below the set point several times before settling). Detection of such oscillations, as well as the nature of the oscillations (such as frequency, amplitude, etc.), may be included in the summary data. In some embodiments, a maximum value, minimum value, slope, average value, or other metric may be included in the summary data associated with the transient portion.

[0129] At block 454, the processing logic performs basic summary operations. In some embodiments, several components are provided to the basic summary operation logic (e.g., both transient data and steady-state data may be provided to the basic summary operation logic). In some embodiments, the basic summary data may include metadata, such as data associated with the processing run. The metadata may include the processing run start time, run ID, tool ID, tool name, batch name, batch ID, processing recipe name, product ID, product information (e.g., product design), product number, product count (e.g., number of products since the last maintenance event), etc.

[0130] In some embodiments, the basic summary data may include contextual data, such as data associated with a sensor, a processing operation, etc. The contextual data may include a channel or sensor name or ID, a processing operation number (e.g., a step number), a timestamp of the processing operation, an operation name or ID, an operation type (e.g., an etch operation, a deposition operation, etc.), a manufacturing subsystem name or ID, etc.

[0131] In some embodiments, basic summary data may include statistical information associated with the trace data. The statistical information may include data mean, median, minimum, maximum, range, quartiles and other percentile information, standard deviation, skew, kurtosis, flags indicating whether the data is within a control range, etc. Basic summary data may be generated according to configuration settings. At block 455, the summary data is stored, for example, by the TTTM module for future use. The summary data may be stored in a data storage, such as Figure 1 in the data storage device 140.

[0132] FIG. 5A to FIG. 5I Flowchart of method 500A-I associated with generating data to cause corrective action according to certain embodiments. Method 500A-I may be performed by processing logic, which may include hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, processing device, etc.), software (such as instructions running on a processing device, a general purpose computer system, or a dedicated machine), firmware, microcode, or a combination thereof. In some embodiments, method 500A-I may be performed in part by prediction system 110. Method 500A may be performed in part by prediction system 110 (e.g., Figure 1 The server machine 170 and the data set generator 172, Figure 3 According to an embodiment of the present disclosure, the prediction system 110 may use method 500A to generate a data set for at least one of training, validating, or testing a machine learning model. Methods 500B to 500I may be performed by the prediction server 112 (e.g., the prediction component 114). In some embodiments, a non-transitory storage medium stores instructions that, when executed by a processing device (e.g., the prediction system 110, the server machine 180, the prediction server 112, etc.), cause the processing device to perform one or more of methods 500A-I.

[0133] For simplicity of description, the method 500A-I is described as a series of operations. However, the operations according to the present disclosure may occur in various orders and / or simultaneously with other operations not presented and described herein. In addition, not all illustrated operations may be performed to implement the method 500A-I according to the disclosed subject matter. In addition, those skilled in the art will understand and appreciate that the method 500A-I may alternatively be represented as a series of interrelated states via a state diagram or events.

[0134] Figure 5A is a flow chart of a method 500A for generating a data set of a model for generating predictive data (e.g., Figure 1 Prediction data 168).

[0135] refer to Figure 5AIn some embodiments, at block 501 , the processing logic implementing method 500A initializes the training set T to an empty set.

[0136] At block 502, processing logic generates a first data input (eg, a first training input, a first validation input), which may include sensor data (eg, Figure 1 Historical sensor data 144; Figure 3 of historical sensor data 344), manufacturing parameters (e.g., Figure 1 In some embodiments, the first data input may include a first set of features for the data type, and the second data input may include a second set of features for the data type (e.g., Figure 4A described).

[0137] In some embodiments, at block 503, processing logic generates a first target output for one or more data inputs (e.g., a first data input). In some embodiments, the first target output is performance data (e.g., metrology data of a processed product, a failed component, etc.). In some embodiments, the input data may be in the form of sensor data, and the target output may be a list of components that may be failed, such as in the case of a machine learning model configured to be performed in conjunction with a physics-based digital twin model. In some embodiments, no target output is generated (e.g., an unsupervised machine learning model that is capable of grouping or finding correlations in input data rather than requiring a target output).

[0138] At block 504, processing logic optionally generates mapping data indicating an input / output mapping. The input / output mapping (or mapping data) may refer to a data input (e.g., one or more data inputs described herein), a target output for the data input, and an association between the data input and the target output. In some embodiments, such as associated with a machine learning model that does not provide a target output, block 504 may not be performed.

[0139] In some embodiments, at block 505 , processing logic adds the mapping data generated at block 504 to data set T.

[0140] At block 506, processing logic branches based on whether the dataset T is sufficient for at least one of training, validating, and / or testing the model 190. If so, execution continues to block 507, otherwise, execution returns to block 502. It should be noted that in some embodiments, the sufficiency of the dataset T may be determined simply based on the number of inputs in the dataset, which in some embodiments are mapped to outputs, while in some other embodiments may be determined based on one or more other criteria in addition to or in lieu of the number of inputs (e.g., a measure of diversity of data instances, accuracy, etc.).

[0141] At block 507, processing logic provides a data set T (e.g., to a server machine 180) to train, validate, and / or test the machine learning model 190. In some embodiments, the data set T is a training set and is provided to the training engine 182 of the server machine 180 for training. In some embodiments, the data set T is a validation set and is provided to the validation engine 184 of the server machine 180 for validation. In some embodiments, the data set T is a test set and is provided to the testing engine 186 of the server machine 180 for testing. In the case of a neural network, for example, the input values ​​of a given input / output mapping (e.g., the values ​​associated with the data input 310) are input to the neural network, and the output values ​​of the input / output mapping (e.g., the values ​​associated with the target output 320) are stored in the output nodes of the neural network. The connection weights in the neural network are then adjusted according to a learning algorithm (e.g., back propagation, etc.), and this process is repeated for other input / output mappings in the data set T. After block 507, the model (e.g., model 190) may be at least one of trained using the training engine 182 of the server machine 180, validated using the validation engine 184 of the server machine 180, or tested using the testing engine 186 of the server machine 180. The trained model may be implemented by the prediction component 114 (of the prediction server 112) to generate prediction data 168 for performing signal processing or for performing corrective actions associated with the manufacturing equipment 124.

[0142] Figure 5B is a flow chart of a method 500B for utilizing one or more models to determine corrective actions to take, according to some embodiments. Figure 5B The operation may include combining Figure 4B The automatic processing operation described.

[0143] refer to Figure 5B , at block 510, processing logic performs preparation operations associated with generating tool-to-tool match (TTTM) information indicating corrective action. Preparation operations may be triggered by manual input (e.g., upon identification of non-optimal performance, in response to non-optimal metrology data for one or more finished products, etc.), at scheduled time intervals, etc. TTTM analysis preparation may include data collection logic. Data (e.g., sensor trace data) may be retrieved from a data store. Data collection may be performed according to configuration settings. The collected data may include the date the process was performed, the tool ID of the manufacturing equipment, the recipe name, data indicating key operations, sensors or parameters, etc. The configuration settings may include indications of these types of data to be collected and pre-processing procedures (e.g., data cleaning, resampling, averaging, smoothing, etc.). Processing logic may perform data pre-processing as part of the preparation operations.

[0144] The preparation operations may further include processing logic to generate summary data, for example, related to the manufacturing process. Figure 4C The data summary operation is discussed in detail. Data summarization can be performed according to configuration settings. In some embodiments, summary data is generated according to the trace sensor data. The trace sensor data can be divided into multiple parts, and summary data is extracted separately from each part. For example, the trace sensor data can be divided into one or more transient parts and one or more steady-state parts. In some embodiments, a sliding window is used to provide statistics of the data values ​​of the part of the trace. If a specific statistical mark (e.g., standard deviation, variance, etc.) of a point within the window is greater than (or less than) a threshold value, it can be determined that the point in this window is included in the steady-state part of the trace data, the transient part of the trace data, etc. Other metrics (e.g., slope of the data, time threshold, etc.) can be used to specify the steady-state and transient parts of the trace data. In some embodiments, the summary logic processes steady-state data and transient data differently. Configuration settings may include adjustments to how to identify steady-state and transient parts, such as window size, values ​​or identifiers of metrics that determine which part the window belongs to, etc. In some embodiments, different data are included in the summary data of different parts of the trace data. In some embodiments, a physics-based (e.g., digital twin) model of a component is used to generate behavior predictions. In some embodiments, summary data is generated using the difference between the digital twin prediction and the measured sensor data. For example, the digital twin model may predict that the voltage supplied to a component (e.g., an electrostatic chuck) may have a specific rise time (e.g., time to reach 90% of the target value), characteristic overshoot (e.g., percentage of the target value of the peak voltage), and settling time (e.g., oscillation time before reaching a stable value). The summary logic may include recording the difference between the measured transient portion of the sensor data and the prediction of these metrics by the digital twin model.

[0145] Summary data may include metadata, contextual data, statistics, and digital twin-assisted summary data. Metadata may include data describing the manufacturing process (e.g., conditions for processing semiconductor wafers). Metadata may include run start time, recipe ID, tool ID associated with manufacturing equipment, tool name, batch name and / or ID, product ID, product information, product number, product count produced by the tool, etc. Contextual data may include data describing trace data, such as sensor ID, recipe operation number, timestamp data, subsystem, operation type, etc. Statistics may include mean, minimum and maximum values, percentile (e.g., quartile) values, variance, skew, kurtosis, range, interquartile range, lower and upper control limits (e.g., calculated based on other statistics, such as three standard deviations from the mean or a multiple of the interquartile range outside the percentile value, etc.), etc. Digital twin auxiliary summary data may include overshoot (e.g., the percentage difference between the target value of a property and the peak value of the property), rise time (e.g., the time for the property to adjust to 90% of the target value), settling time (e.g., the time the measured value is within a certain threshold of the target value), steady-state error (e.g., the percentage difference between the final value of the measurement and the target value), closed-loop control system parameters, such as gain margin (e.g., the gain difference between no gain and system gain at a frequency where the phase offset between input and output is 180°), phase margin (e.g., the phase difference between the system phase and 180° at a frequency where there is no gain) and bandwidth; oscillation detection, oscillation frequency, oscillation amplitude, etc.

[0146] At block 511, processing logic checks for one or more existing models that are compatible with the data to be analyzed. An existing model may be a model trained using historical data. The compatibility of an existing model may be determined based on training on the same set of manufacturing equipment, the same type of manufacturing equipment, the same or similar processes, etc. If an existing compatible model does not exist, the process continues to block 512. If an existing compatible model does exist, the process continues to block 513.

[0147] At block 512, processing logic performs training operations associated with performing a TTTM analysis. Figure 5C The training operation is discussed in more detail. The training operation may include, for example, Figure 1 The data storage 140 retrieves data associated with one or more manufacturing processes. The training operation may include retrieving and / or generating golden run sensor data, retrieving summary data, retrieving trace data, etc. The training operation may be performed according to the configuration settings.

[0148] At block 513, processing logic performs inference operations associated with TTTM analysis. Figure 5DInference operations are discussed in more detail. Inference may include analysis using one or more models. Models may include physics-based models, digital twin models, machine learning models, etc. Inference may be performed based on configuration settings. Inference operations may be performed in response to previous inference operations, for example, further drill-down analysis may be performed in response to the output of previous operations that meet one or more criteria. Inference results may include corrective actions, instructions for systems or components to further follow up, schedule maintenance, etc. Inference operations may include visualization of results.

[0149] At block 514, the processing logic performs a case evaluation operation associated with the TTTM analysis. The case evaluation program can be used to generate information indicating the performance of the TTTM analysis program. The performance of various models and algorithms can be evaluated. The case evaluation operation can include comparing the predicted performance metric with the measured performance metric. In some embodiments, the case evaluation can be triggered by a user. In some embodiments, the case evaluation can be triggered by a result, such as the result of the TTTM analysis is outside the threshold range, the result of the measurement is outside the similarity threshold range predicted by the TTTM analysis, etc. In some embodiments, the case evaluation can be performed according to the case evaluation schedule (e.g., every nth process, at a selected frequency, etc.). Data related to the manufacturing process (e.g., sensor data, summary data, etc.) can be provided to the processing logic using the output of the TTTM analysis (e.g., one or more predicted faulty components, predicted drift or aging subsystems, predicted metrology data of finished products, visualization of component or subsystem health, etc.). Measured performance data related to the output of the TTTM analysis (e.g., maintenance results of predicted faulty components, measured metrology of products, etc.) can also be provided to the processing logic performing the case evaluation.

[0150] The case evaluation operation may include and / or be intended to determine whether the TTTM module (e.g., one or more models included in the TTTM analysis) is to be retrained. The processing logic may evaluate whether to retrain based on one or more of a plurality of metrics. In some embodiments, retraining may be performed based on a specific portion of the prediction that does not meet the accuracy threshold (e.g., the portion of the prediction within a period of time). In some embodiments, retraining may be performed based on the number of products in the column that do not meet the accuracy threshold. In some embodiments, retraining may be performed based on known changes to the processing conditions, such as recipe updates, input material updates, chamber maintenance performed, etc. The processing logic may use data from cases with sufficiently accurate TTTM predictions and cases with unsatisfactory TTTM predictions for evaluation. The processing logic may use both positive cases (cases where the TTTM analysis predicts failures, aging, drift, etc., such as true positives and false positives) and negative cases (cases where the TTTM analysis does not predict failures, such as true negatives and false negatives) for evaluation.

[0151] The processing logic may further evaluate false cases (e.g., false positives, false negatives). The root causes of false cases may be evaluated, such as potential modifications to the TTTM model to reduce false cases. In some embodiments, the effect of adjusting the weights of operations, sensors, etc. may be evaluated. In some embodiments, the effect of adjusting control limits may be evaluated. In some embodiments, the effect of removing sensors, operations, etc. from future TTTM modeling may be evaluated. The processing logic may store the results of the case evaluation. The processing logic may generate recommendations, such as updating or retraining all or part of the TTTM analysis module.

[0152] At block 515, processing logic retrains the TTTM module. Retraining may be performed based on recommendations generated during the case evaluation at block 514. For example, during the case evaluation operation, processing logic may generate recommendations to retrain sensor weights. During the retraining operation, processing logic may make recommendations by providing additional data to train one or more models used for TTTM analysis. In some embodiments, retraining may include model validation and / or testing. In some embodiments, the retraining operation may include using different sensor data (e.g., as combined with Figure 4A The new model may be trained based on the different sensor sets described above. In some embodiments (e.g., based on recommendations generated during the case evaluation operation), a retraining operation may not be performed. After retraining, the process may return to the inference phase (block 513) to perform TTTM analysis on additional products, processes, manufacturing equipment, etc.

[0153] Figure 5C Flowchart of method 500C for performing training operations of a TTTM analysis method according to the description of some embodiments. These operations may be directed to preparing a TTTM analysis for inference and are different from training a specific machine learning model, a physics-based model, etc. At block 521, data for the TTTM analysis is retrieved. The retrieved data may include trace data, pre-processed trace data, summary data, etc. The retrieved data may include data identified, generated, stored, etc. in preparation operations for the TTTM analysis. Data may be retrieved according to configuration settings, such as settings indicating which analysis techniques, which training models, etc. will be used in the TTTM analysis.

[0154] At block 522, processing logic determines whether one or more golden data sets are to be used in the TTTM analysis. Golden data may include trace data associated with successfully manufactured products, equipment that is operating normally, etc., clean or pre-processed trace data, and summary data, etc. The use of one or more golden data sets may be based on configuration settings, based on which model or models are to be used as part of the TTTM analysis, etc. If no golden data sets are to be used, the flow continues to block 526. If one or more golden data sets are to be used, the flow continues to block 523. At block 523, processing logic determines whether one or more golden data sets are available (e.g., data designated as golden data is stored in the data storage 140). If golden data is available (e.g., golden data for each sensor, process, system, etc. for which golden data is to be used), the flow continues to block 525 and retrieves one or more golden data sets. If golden data sets are not available (e.g., if one or more golden data sets to be used in the TTTM analysis are not available), the flow continues to block 524.

[0155] At block 524, processing logic generates one or more golden data sets. Data may be selected by processing logic (e.g., from data storage 140) to serve as one or more golden data sets. Data may be selected as golden data based on one or more indications that the data corresponds to acceptable processing parameters (e.g., processing parameters that facilitate production of a product within manufacturing tolerances). In some embodiments, a tool, chamber, and / or set of manufacturing equipment that was recently qualified (e.g., recently maintained, recently installed, etc.) may be designated as a golden tool. Data from a process performed by a golden tool may be designated as golden data. In some embodiments, data indicating production of an acceptable product may be associated with sensor data (e.g., metrology data, classification or quality data, etc.). Sensor data related to production of an acceptable product (including pre-processed sensor data, summary sensor data, etc.) may be designated as golden operating data. In some embodiments, one or more sensors may have associated golden operating data. In some embodiments, one or more processing operations may have associated golden operating data.

[0156] At block 526, processing logic defines control limits. The control limits may be set according to configuration settings. In some embodiments, the control limits may be established by user input (e.g., an acceptable process range may be selected by the user). In some embodiments, the control limits may be calculated by processing logic (e.g., three standard deviations from the mean). In some embodiments, a hybrid system may be used (e.g., three standard deviations from the mean, but not exceeding user defined limits).

[0157] Figure 5DFlowchart of method 500D for performing inference operations of a TTTM analysis method according to some embodiments. At block 530, one or more models to be used in the TTTM analysis are imported. These may include trained machine learning models, physics-based models, digital twin models, statistical models, etc. Models may be imported according to configuration settings.

[0158] At block 531, processing logic performs an outlier detection operation. Summary data may be compared to control limits to determine whether a data point (e.g., summary data associated with a set of trace data) is an outlier. Each sensor in each processing operation (e.g., recipe step) may be considered individually. A data point may be considered an outlier if any one summary data is outside of a control limit, if multiple summary data exceed a control limit, if some of the total weights of the summary data (e.g., summaries with more predictive power are given more weight) exceed a control limit, etc. In some embodiments, the individual outlier signals may be aggregated to generate an index (e.g., a TTTM index) that indicates the performance and / or quality of a chamber, tool, process, facility, product, group of manufacturing equipment, etc.

[0159] In some embodiments, the generation of the TTTM index may depend on sensor and process operation weights and control limits. It may be found that some sensor operation combinations have higher predictive power than other sensor operation combinations. For example, for a process operation of a semiconductor substrate utilizing plasma, a sensor indicating a plasma parameter may have a high predictive power of chamber performance. For a process operation not utilizing plasma processing, the same sensor may not provide information that is easily correlated to tool performance. The weighting factor may indicate the importance of the contribution of the relevant data to the performance.

[0160] In some embodiments, data points classified as high outliers (e.g., above an upper control limit), low outliers (e.g., below a lower control limit), and normal values ​​(e.g., between an upper control limit and a lower control limit) may be treated differently. In some embodiments, one or more weighting factors may be applied to sensor data, summary data, and the like. Weighting factors (e.g., weights) may indicate the impact of a property value measured in a processing chamber on performance. In some embodiments, weights may be applied to sensors (e.g., sensors closer to a substrate may produce data more closely related to performance). In some embodiments, weights may be applied to processing operations (e.g., sensor data during an etching operation may be more closely related to the performance of an etching recipe than operations where etching does not occur (e.g., preheating or cooling operations). In some embodiments, weights may be applied to processing runs (e.g., processing runs with known problems may be given lower weights). In some embodiments, weighting factors may be applied to data types (e.g., sensor types such as temperature sensors, summary data types such as averages may be given more weight than standard deviations, summary data from transient portions may be given different weights than steady-state portions from trace data, etc.). In some embodiments, weighting factors may be applied to combinations of conditions, such as specific sensors during specific operations (e.g., a sensor measuring the voltage applied to produce a plasma may produce results that are highly correlated with performance in processing operations involving a plasma, but may not correlate well with performance in operations without a plasma).

[0161] In some embodiments, weights may be used to generate a TTTM index. In some embodiments, weights may be selected so that the sum of weights from all sensors for each processing operation (e.g., processing step) may be normalized (e.g., equal to 1). An outlier sensor for an operation may contribute zero to the TTTM index value, while a normal value contributes its weight. If all sensors have normal values, the TTTM index may be 1 (or may be scaled to a different value). A TTTM index value below 1 (or a scaled value) may indicate that further analysis of this manufacturing process, operation, tool, facility, etc. is recommended.

[0162] In some embodiments, weights may be assigned (or the assignment of weights may be assisted) by a weight finder model. In some embodiments, the weight finder model may include a machine learning model. The machine learning model may be trained using data associated with the sensor (e.g., summary data) as training input and performance data (e.g., metering data) as target output. The trained machine learning model adjusts the weights that connect the input data to the target output data. The machine learning model may output weights that connect the operation / step combination to the performance. The weights may be used in TTTM analysis, as outputs of the machine learning model, adjusted by the user, etc. In some embodiments, a partial least squares model may be used to generate a weight function. In some embodiments, a stacked partial least squares model (e.g., multiple partial least squares models arranged as an ensemble model) may be used.

[0163] At block 532, processing logic performs sensor contribution calculation operations. In some embodiments, the performance of operations of blocks 532 to 535 may depend on the value of the TTTM index. For example, processing logic may be configured to perform further analysis (e.g., drill-down analysis to explore the root cause) if the TTTM index meets a threshold value (e.g., below a threshold value established in a configuration setting). At block 532, sensor contribution information indicating "how much" an outlier one or more data points are may be generated. Previous operations include marking whether a data point is within a control limit. The sensor contribution value indicates the degree to which the outlier exceeds the control limit. An exemplary sensor contribution metric is a severity distance. The severity distance is a measure of the degree to which a data point (e.g., a summary data point) exceeds a control limit. The severity distance indicates the relationship between a data point (or a set of data points) and an associated control range. In some embodiments, a severity distance may be generated for one or more sensor / operation data points of a manufacturing run (e.g., a single product). The severity distance may be based on one or more summary data (e.g., one or more types of summary data). In some embodiments, the severity distance calculation may include a calculation of the degree to which one or more data points exceed a control limit. In some embodiments, the severity distance calculation may include a calculation of the degree to which a data point exceeds a control limit relative to the size of a control range (e.g., a temperature sensor may be assigned a control range between 20 degrees and 40 degrees, and a data point of 50 degrees may be calculated as being 50% above the control upper limit size). For example, a data point below a lower control limit (e.g., below a control range) may have a severity distance calculated as

[0164]

[0165] Where SD is the severity distance, LCL and UCL are the lower and upper control limits, x is the value of the relevant data point (or aggregation of data points), the denominator of the fraction calculates the control range, w is the weight factor, and 100 is used to convert the severity distance to percentage form. Corresponding formulas can be used for data points above the upper control limit. Different calculations can be used for the sensor contribution calculation - for example, the sensor contribution calculation can depend on the square of the difference between the control limit and the data point value to give further weight to data points farther away from the control limit (e.g., the relationship between the sensor contribution and the difference between the data point and the control limit is steeper than linear). The severity distance of a normal point can be zero. In some embodiments, the severity distance can be related to the distance away from the center of the control range (e.g., the mean or an ideal value that is not necessarily at the mathematical center).

[0166] An aggregate metric value or index may be generated based on one or more severity distances. An exemplary aggregate metric is a severity score. The severity score may include an aggregation of severity distances for a number of process runs (e.g., a number of products produced). The severity score may indicate the likelihood that a particular sensor / operation pair (e.g., a sensor measuring a voltage applied to generate plasma in a plasma etch process operation) will generate data indicative of chamber failure, chamber drift, etc. over a number of runs. The calculation of an aggregate severity distance (such as a severity score) may include a factor indicating the frequency of outlier data points. For example, a severity score may be calculated as

[0167]

[0168] Where SS is the severity score, FF is a frequency factor calculated by dividing the number of outliers in a set of data (e.g., the number of runs for a particular sensor / operation pair) by the total number of data points in the set, r is the index of the process run, q is the number of process runs to be included in the aggregation, and SD is the severity distance from run r (e.g., the severity distance associated with the sensor / operation pair for a single process run). The severity score can indicate the degree to which a sensor / operation pair performs beyond expectations over several processes or runs (e.g., a product produced). Similar metrics can be generated to generate supplemental data, for example, a similar metric of severity scores summed over sensors rather than runs can be used to indicate whether a particular operation is likely to be performed correctly for a particular product.

[0169] In some embodiments, the metrics may be further aggregated to aid in the analysis. For example, the processing logic may aggregate the severity scores of the recipe operations (in some embodiments, corresponding to a single sensor and a single operation) to find a severity index for each sensor (e.g., by summing the product of the operational weights and severity scores of the processing operations). Such analysis may be triggered by the severity score reaching a threshold (e.g., a quality index that meets a standard, a quality score, a severity score, etc.), the number or portion of the severity scores that meet one or more thresholds, etc. The trigger conditions for aggregating data to generate a new index may be stored as configuration settings. Aggregating severity scores in this manner may provide additional information to problematic or faulty sensors. Similar processing may be performed to sum the sensors to determine whether one or more operations of the manufacturing process are not optimally performed. In some embodiments, the TTTM index (in some embodiments, associated with a run or product and a processing operation) may be aggregated, for example, by adding the product of the TTTM index (e.g., each run / operation) to the operational weights on multiple operations (e.g., preheat, etch, deposition, and cooling operations) to generate a TTTM index for each run. This can help determine if a particular operation of a manufacturing process is not operating optimally. Similar procedures can be performed to isolate, for example, a failure traced back to a single run (e.g., a single semiconductor wafer). Various sets of indices can be used by processing logic to recommend components or subsystems for maintenance or replacement, various sets of indices can be provided as input to machine learning models to generate corrective action recommendations, etc. Corrective actions can include scheduling preventive maintenance, updating process recipes, sending alerts to users, etc. In some embodiments, cluster analysis can be performed on the indices and / or metrics. In combination Figure 5H Cluster analysis is discussed in more detail. Cluster visualization is discussed in more detail in conjunction with block 535 .

[0170] At block 533, the processing logic performs a normal value drift detection. Fig.5I Methods for detecting normal value drift are described in detail. TTTM index analysis, severity distance, severity score, etc. may be associated with outlier data. Normal value data (e.g., data within control limits) may also provide valuable information for analysis. Normal value analysis may be performed using summary data. In some embodiments, summary data from a run (e.g., one product, such as one semiconductor wafer) may be compared with golden summary data (e.g., summary data records during the production of products that meet performance thresholds). In some embodiments, summary data from a series of manufacturing runs (e.g., one day, one week, one month, etc. of processing) may be compared with golden summary data from a series of production runs.

[0171] The comparison between the golden data and the data to be analyzed may be performed using any technique for comparing data sets. In some embodiments, techniques for quantifying time series similarity may be used. One metric that may be used is to pre-process and / or clean the data so that each data point of the data set to be analyzed has a corresponding data point of the golden run data (e.g., data with the same timestamp, data associated with the same processing operation, etc.). The differences between the values ​​of the corresponding data points may then be aggregated (e.g., sum of squares of differences, absolute values ​​of differences, etc.). The metric may be further processed, e.g., square root of the sum of squares divided by the number of data points to be normalized, etc. A higher value of the metric may indicate that the two data sets are less similar. In some embodiments, a discrete Fourier transform may be performed on the two data sets, and the corresponding Fourier coefficients may be compared in a manner similar to the analysis of comparing corresponding data points described above. In some embodiments, an autoregressive model may be used to compare the data sets. An autoregressive model is a representation of a random process in which the value of a data point of a process depends on the value of a previous data point. An autoregressive model may be used to determine the likelihood that the data set to be analyzed and the golden data set are associated with the same process (e.g., processing conditions, chamber quality, etc. are the same between the two sets).

[0172] In some embodiments, an elastic dissimilarity metric may be used to determine a similarity value between two data sets. In some embodiments, dynamic time warping may be used for normal value drift analysis. Dynamic time warping works by aligning or warping data sets in the time domain, thereby minimizing the cumulative cost of the alignment. The cost function may be determined according to configuration settings. The elastic dissimilarity metric may be considered to adjust the spacing between data points in one data set so that the one data set is better aligned with the second data set. The elastic dissimilarity metric does not adjust the order of the data points, only the spacing between the data points. The elastic dissimilarity metric may return a metric or index value that indicates how much spacing adjustment was made to achieve optimal alignment of the two data sets (e.g., determined by minimizing the cost function). In some embodiments, each point of the first data set may be associated with a point of the second data set after elastic warping. For example, the data to be analyzed may be warped so that the data to be analyzed is optimally aligned with the golden data. Each point of the data to be analyzed may be assigned to a "nearest neighbor" point in the golden running data set after warping (e.g., several points in the analyzed set may be assigned to the same nearest neighbor point in the golden data set). In some embodiments, each data point of the two data sets may be linked to one or more nearest neighbor points of the other data set. For example, each point of the warped dataset being analyzed may be assigned a nearest neighbor point of the golden dataset. This assignment may include some points of the golden dataset being associated with multiple points of the dataset to be analyzed. This assignment may also include some points of the golden dataset not being assigned to any points in the analysis dataset as nearest neighbors. Each data point in the golden set may then be assigned a nearest neighbor point in the time-warped dataset to be analyzed. In the case where the two datasets are aligned (e.g., the warp cost function is minimized when each data point 1...n in the dataset to be analyzed is closest to the corresponding point 1...n in the golden dataset), the number of nearest neighbor associations may be equal to the number of data points in the set. In the case where the two datasets are significantly misaligned, one or more points in each dataset may have several points designated as nearest neighbors in the other dataset. In this case, the number of nearest neighbor associations may be higher than the number of data points in the set. The increased number of associations may be used as a metric indicating how many warps produce the best alignment of the two datasets (in some embodiments, with preprocessing such as normalization, subtracting the number of data points, etc.). Combined Fig.5I and FIG. 6D to FIG. 6E The detection of normal value drift and the measurement of elastic dissimilarity are further discussed.

[0173] In some embodiments, the operation of each tool, chamber, manufacturing equipment group, etc. may be different. Each tool, chamber, equipment group, etc. may have its own associated one or more golden data sets for each tool, chamber, equipment group. Each type of process (e.g., each target product design, each recipe, etc.) may have its own associated one or more golden data sets for each type of process.

[0174] The normal value drift detection operation can be used to perform corrective actions. In some embodiments, a normal value drift detection metric above a certain threshold value (in some embodiments, according to a configuration setting) can trigger a corrective action. The corrective action can include providing an alert to a user, scheduling maintenance, updating a recipe, etc.

[0175] At block 534, additional features of the TTTM module may be used by the processing logic for further analysis. Additional features may be used according to configuration settings. In some embodiments, values ​​of various metrics may be used to determine whether to utilize additional features. Additional features may be helpful for drill-down analysis to isolate causes of chamber failures, drift, unexpected results, insufficient product quality, etc. Additional features may be used to determine corrective actions to be taken. Additional features may include providing data associated with the manufacturing process to one or more models (e.g., machine learning, physics-based, etc.) to provide additional analysis.

[0176] In some embodiments, additional features of the TTTM module may include trace data analysis, such as a trace data exploratory analysis module. Trace data analysis may include using a trained model (e.g., a machine learning model). In some embodiments, trace data analysis may be performed by providing trace data to a trained machine learning model and receiving output from the machine learning model indicating a corrective action, indicating a device component to be maintained or replaced, indicating a fault in a subsystem of a manufacturing device, etc. In some embodiments, an autoencoder model may be used to perform trace data analysis. Figure 7 Discuss the autoencoder operation in more detail. FIG. 6F to FIG. 6G Examples of visualizations associated with the results of trace data analysis are discussed in more detail.

[0177] The trace data analysis may include data preparation operations such as data cleaning, reformatting, normalization, interpolation, etc. The trace data may then be provided to a trained model. The trained model may generate an output indication of a sensor, subsystem, product, etc. that is not performing in an expected manner. In some embodiments, trace data associated with a number (e.g., all) of sensors associated with a product processed by a manufacturing device may be provided to the trained model. A data sequence associated with a product sequence may be provided to the trained model. This model may generate data indicating a non-optimal predicted product performance, measurement, property, etc. based on the trace data analysis of the trace sensor data.

[0178] At block 535, processing logic performs a result visualization operation. Any metrics, analyses, etc. performed or generated by the TTTM analysis process may be incorporated into one or more visualization processes, and in some embodiments, processing logic may perform visualization operations according to configuration settings. Indices, metrics, etc. may be visualized in any convenient or appropriate manner. For example, scatter plots, histograms, box plots, line plots, etc.

[0179] In some embodiments, data can be clustered for visualization. In some embodiments, severity distance can be used for clustering and visualization of outliers. Data from multiple runs and multiple sensors can be used for clustering and visualization. In some cases, most sensor data will be normal value data (e.g., within control limits). Data can be preprocessed to remove redundant data (e.g., all normal value data can be removed). In some embodiments, dimensionality reduction can be applied to simplify the remaining outlier data. Dimensionality reduction can help data visualization, for example, reducing data to two or three dimensions for more convenient analysis. In some embodiments, nonlinear dimensionality reduction models can be used. Nonlinear dimensionality reduction models can capture correlations that linear dimensionality reduction models (e.g., principal component analysis) will not capture. Dimensionality reduction models can convert correlations or similarities between data points into the probability that data points are close together in dimensionality reduction space (e.g., by Euclidean distance, Minkowski distance, correlation distance, or another distance function). In some embodiments, dimensionality reduction models can include t-random nearest neighbor embedding algorithms. In some embodiments, parameters controlling the operation of the dimensionality reduction model may be stored as configuration settings, such as perplexity (a metric that determines the number of nearest neighbors expected in a cluster), distance metrics in high-dimensional space, and the like.

[0180] In some embodiments, a clustering model may be used to cluster the reduced dimension data. In some embodiments, a clustering model may be selected that self-selects the number of clusters in the data set. In some embodiments, density-based spatial clustering with application of a noisy clustering algorithm may be utilized. Parameters that determine the operational details of the clustering model may be stored as configuration settings (e.g., a metric that affects cluster distances, cluster size, distance metrics, etc.).

[0181] In some embodiments, data points (e.g., clustered, dimensionally reduced outlier points) may contribute to a severity score (e.g., a sensor's severity distance contributes to a severity score associated with an outlier point). Top sensors that contribute to a cluster (e.g., a sensor represented in multiple data points in a cluster, a sensor that contributes to one cluster but not another, etc.) may be used to label the cluster. In some embodiments, sensors and their associated data may be assigned to subsystems of a manufacturing device (e.g., a heater power subsystem, a chamber temperature subsystem, a chamber pressure subsystem, etc.). Clusters may be labeled by the subsystem that contributes most to the outlier state of the points associated with the cluster. In this way, clusters may be grouped according to root cause (e.g., subsystem failure). Outlier clusters may be used to take corrective actions, such as providing alerts to users, scheduling maintenance, adjusting process recipes, etc.

[0182] The processing logic may generate visualizations related to normal value drift detection. The results of the elastic dissimilarity measurement model may be visualized. The displacement may be represented by a scatter plot, bar graph, line plot, etc., the data point association may be plotted on a graph of dataset 1 and dataset 2, or other methods of visualizing normal value drift detection data may be used.

[0183] The processing logic may generate visualizations related to the detection of anomaly traces. In some embodiments, sensors may be grouped into subsystems, and metrics describing deviations from expected sensor results may be aggregated into subsystem metrics. Subsystem analysis metrics may be displayed for each subsystem per run (product produced), for example as a heat map.

[0184] In some embodiments, visualization may be provided to the user as part of a graphical user interface. In some embodiments, additional data may be obtained through the interaction of the user with the graphical user interface. In some embodiments, additional analysis may be performed based on the interaction of the user with the graphical user interface. For example, a heat map showing the results of an abnormal trace detection analysis may be based only on a single metric (e.g., each subsystem runs each time), but when selecting a visualized data point, more data may be displayed (e.g., details of the difference between the analyzed trace data and the expected trace data, details of the output of the machine learning model, details of the recommended corrective actions, etc.). In another example, a visualization tool may display representations of the TTTM index for a variety of tools, products, etc. A user may select a specific representation (e.g., one replaced from a set of other points), and additional analysis may be performed on the data associated with this data point (e.g., normal value drift detection, abnormal trace detection, etc.).

[0185] Figure 5EFlowchart of method 500E for causing corrective action to be taken based on metrics indicative of process chamber performance according to some embodiments. At block 540, processing logic receives trace sensor data associated with a first manufacturing process of a manufacturing chamber. The trace sensor data may be collected during processing of a product such as a semiconductor wafer by the manufacturing chamber.

[0186] At block 542, processing logic processes the trace sensor data to generate summary data associated with the trace sensor data. In some embodiments, generating the summary data includes identifying at least one steady-state portion and at least one transient portion of the trace sensor data. The transient portion includes a set of trace data associated with a time period when the property value in the chamber is changing (e.g., a variance above a threshold value). The steady-state portion includes a set of trace data associated with a time period when the property value measured by the sensor does not change (e.g., a variance below a threshold value). In some embodiments, a sliding window is used to distinguish between the steady-state portion and the transient portion. In some embodiments, determining whether a set of data points (e.g., points within a window) of the trace sensor data belongs to the transient or steady-state portion may include determining whether a statistical measure (e.g., a standard deviation) of the set of data points meets a threshold value. The transient portion may include a set of points whose standard deviation is above a threshold value, while the steady-state portion may include a set of points whose standard deviation is below a threshold value. Determining whether a set of points belongs to the transient portion or the steady-state portion may include considering a statistical measure of a window of the data points that is nearby (e.g., overlaps with the set of points). In some embodiments, the processing recipe includes a plurality of processing operations. In some embodiments, trace data from a sensor of a process operation (e.g., one or more process operations of a process recipe) may include a transient header (e.g., when conditions in a manufacturing chamber are adjusted to a set point), a steady-state portion (e.g., when conditions remain at the set point), and a transient tail.

[0187] In some embodiments, summary data is generated for each section (e.g., each steady-state section, each transient section, etc.) respectively. The summary data of the steady-state section may include a statistical description of the data, such as an average, maximum value, minimum value, range, standard deviation, kurtosis, etc. The summary data of the transient section may include a statistical description. The summary data of the transient section may also include a metric generated taking into account the output of the digital twin model. A digital twin model of one or more components of a manufacturing chamber associated with one or more sensors may be generated (e.g., a sensor may measure the power supplied to the component, and a digital twin model of the component and a power supply of the component may be constructed). The digital twin model may receive input information indicating a processing recipe (e.g., a set point, a value of a neighboring environmental property, etc.), and generate output time trace data predicting sensor readings. In some embodiments, the output of the digital twin model (e.g., simulated sensor data) may be compared with measured sensor data. In some embodiments, the difference between the features in the simulated and measured sensor data (e.g., a difference greater than a threshold value) may be included in the summary data of the transient section. Characteristics of transient data may include metrics such as overshoot above a target or final value, rise time, settling time, parameters of detected oscillations, differences between set point and final value, and the like.

[0188] At block 544, a quality index score is generated based on the summary data. The quality index score may be associated with a particular run (e.g., a product produced by a manufacturing facility), a process operation, a sensor, or some combination of these features (e.g., a quality index score may be associated with a sensor and an operation across multiple runs, a quality index score may be associated with a group of sensors (such as all temperature sensors, etc.)). In some embodiments, a quality index is generated that indicates the quality of the runs and process operations. In some embodiments, the generated quality index may be used to perform actions to increase consistency of process parameters between runs, between chambers, etc., such as tool-to-tool matching (TTTM).

[0189] In some embodiments, a quality index is generated in view of one or more classification conditions. In some embodiments, data (e.g., summary data) may be classified as normal value data or abnormal value data. In some embodiments, data may be classified as normal value data, low abnormal value data (e.g., below a lower control limit), or high abnormal value data (e.g., above an upper control limit). Classification may be performed by comparing data values ​​to upper and lower control limits, for example, to determine whether the data value is within a control range, to determine whether the data meets a threshold value, etc. Control limits may be assigned by a user, may be statistical in nature (e.g., three standard deviations from the mean of historical data), etc. In some embodiments, normal values ​​may not contribute to a quality index score, and the quality index may be based only on abnormal values.

[0190] In some embodiments, a quality index is generated in view of a plurality of weighting factors. The weighting factors may represent the contribution of the data to the performance, such as the contribution to the properties of the finished product. For example, in certain processes, the data from one sensor may have a stronger correlation with the performance than the data from another sensor. For such processes, the data from a first sensor with a weaker correlation with the product performance may have a smaller weight in the calculation of the quality index score than the data from a second sensor with a stronger correlation with the product performance. The weighting factors may be associated with sensors, processing operations, sensor / operation pairs (e.g., during plasma operation, a sensor measuring the voltage provided to the plasma generating device may be assigned a high weight, while during operation without plasma, a low weight or zero weight may be assigned), features of the data (e.g., one aspect of the summary data may be more relevant to the product performance than another aspect, one part of the trace data may be more relevant to the product performance than other parts, etc.). In some embodiments, the weighting factors may be extracted from a trained machine learning model. The machine learning model may be trained to associate process data (e.g., sensor data, summary sensor data, etc.) with performance data (e.g., metrology data). The machine learning model may be configured to map one or more relationships between sensor data and metrology data. The weight factors may be extracted from a machine learning model (e.g., a stacked partial least squares model), may be determined in view of factors extracted from a machine learning model, may be calculated based on data including data extracted from a machine learning module (e.g., in addition to process knowledge), etc.

[0191] In some embodiments, a quality index score may be generated as an aggregation of calculated metrics. For example, a TTTM index score may be generated by aggregating data associated with many sensors (e.g., via a weighted sum) to calculate a quality index score that indicates the quality of an operation of a processing run (e.g., a product). Other index aggregations may also be applicable to the TTTM process, for example, aggregating multiple runs to indicate the quality of the process over time, aggregating operations and sensors to provide a general quality value for the processing run, etc. The processing logic may generate many indices and provide information (e.g., provided to a user, further processing logic, etc.) associated with index values ​​that may indicate problems with manufacturing equipment (e.g., all index values ​​that meet a threshold value, a specific percentage or number of index values ​​that are most likely associated with a problem or failure, etc.). Index values ​​that indicate problems or failures can be used to initiate further analysis, for example, drill-down analysis to determine the root cause, corrective actions to be taken, etc.

[0192] In some embodiments, the quality index score may include information related to the extent to which the data point exceeds the control range, such as the severity of the outlier. In some embodiments, a severity distance may be assigned to the data point. Calculating a severity distance (e.g., a severity distance score) may include calculating the extent to which the data point exceeds the control range, such as as a fraction of the size of the control range (e.g., an outlier below a lower control range may have a severity distance calculated using the ratio of the extent to which the data point is below a lower control limit value and the difference between an upper control limit value and a lower control limit value). Control limits and control ranges may be associated with sensors, process operations, chambers, product designs, etc. Calculation of severity distances may include weighting factors. Severity distances may be associated with sensors, process operations, and process runs (e.g., products).

[0193] In some embodiments, the severity distances may be aggregated (e.g., aggregated into a severity score). The aggregated severity distance score may be used for drill-down analysis. As an example, generating a severity score (e.g., an aggregated severity distance) may include summing the severity distances of multiple runs (e.g., generating a severity score associated with a sensor and processing operation). In some embodiments, the severity score may include a factor related to how many points are outliers, such as the frequency (e.g., percentage) of outlier points for each sensor and processing operation.

[0194] In some embodiments, one or more metrics indicating quality (e.g., TTTM index, severity score, etc.) may be provided to the processing logic for visualization. In some embodiments, the data points (e.g., severity distance) may be provided to a dimensionality reduction model. In some embodiments, the dimensionality reduction model may be a nonlinear model, such as a t-stochastic nearest neighbor embedding model. In some embodiments, the data points (e.g., dimensionality reduction severity distance) may be provided to a clustering model. In some embodiments, the clustering model may be a nonlinear clustering model, such as density-based spatial clustering with application of a noise model. In some embodiments, the data points of the clusters may be assigned to subsystems (e.g., based on the most significant contribution to the severity score of the data point). In some embodiments, the subsystem root causes of the manufacturing equipment may be assigned to each cluster.

[0195] At block 546, processing logic provides an alert to a user based on the quality index score, wherein the alert includes an indication that the fabrication chamber performance does not meet a first threshold. In some embodiments, a metric (e.g., a TTTM index score) may be provided to the user. In some embodiments, a visualization of the metric (e.g., a bar graph, a histogram, a box plot, etc.) may be provided to the user. In some embodiments, the user may use the visualization to initiate, influence, or continue a drill-down analysis, for example, to identify a root cause of a failure or a corrective action to be taken.

[0196] Fig. 5FFlowchart of method 500F for performing normal value analysis according to some embodiments. In one embodiment, method 500F is performed after method 500E is performed on data in which no outliers are identified. At block 550, processing logic receives first data associated with measurements made by a sensor during a first manufacturing procedure of a manufacturing chamber. In some embodiments, the data may include summary data. The data may include normal value data points, wherein the normal value data points are points that meet a standard (e.g., a threshold value). At block 552, processing logic receives second data. The second data includes reference data associated with the first data. The reference data may be data associated with a process that produces an acceptable product, a process that achieves acceptable conditions, etc. The reference data may be referred to as golden data. The reference data may be associated with the same process, sensor, chamber, equipment, etc. as the first data.

[0197] At block 554, the first data and the reference data (e.g., golden run data) are provided to a comparison model. In some embodiments, the comparison model is an elastic dissimilarity model, e.g., a model that manipulates the space between two data points to achieve the closest possible match without changing the order of the data points of the two data sets. In some embodiments, the comparison model may be a dynamic time warping model. In some embodiments, data (e.g., summary data) over multiple processing runs (e.g., days, weeks, or more of product manufacturing) may be provided to the comparison model. A metric indicating how different the measured data and the golden data are may be provided (e.g., by the elastic dissimilarity model). For example, the model may note the best match between each data point of the two data sets and the data point of the opposite set. Two identical data sets may have the same number of matching pairs as points in one data set. Two unmatched sets may have more pairs. The metric may be related to the number of matching pairs between the two sets, e.g., to the ratio of data points to matching pairs.

[0198] At block 556, processing logic receives a similarity score associated with the first data and the reference data from the comparison model. In some embodiments, the similarity score is related to the number of matching pairs between the two data sets. At block 558, processing logic causes execution of a corrective action based on the similarity score. The corrective action may include sending an alert to a user. The alert may include a visualization, such as a visualization of multiple similarity scores, a visualization of the closest point matches between the two sets of data, etc. The corrective action may include scheduling preventive or corrective maintenance. The corrective action may include updating a process recipe.

[0199] Figure 5GFlowchart of a method 500G for performing trace sensor data analysis according to some embodiments. At block 560, processing logic receives trace sensor data associated with a first manufacturing process of a processing chamber. In some embodiments, trace data may be provided to processing logic in response to values ​​of one or more metrics satisfying a threshold value. For example, a TTTM index may satisfy a threshold value. In response to the index satisfying the threshold value (e.g., satisfying a criterion), trace data associated with the TTTM index (e.g., trace data from a sensor having a high severity score) may be further analyzed. In some embodiments, processing logic performs preprocessing on the trace data. Preprocessing may include smoothing, cleaning (e.g., resizing, removing empty data, etc.), interpolation, etc.

[0200] At block 562, processing logic processes the trace sensor data using one or more trained machine learning models. The trained machine learning models are trained to generate a representation of the trace sensor data (e.g., a reduced dimensionality representation). The one or more trained machine learning models are trained to reconstruct the sensor data based on the representation of the trace sensor data. The one or more trained machine learning models output the reconstructed data. In some embodiments, the trace sensor data may be provided to an autoencoder model. In some embodiments, the trace sensor data may be provided to a trained encoder model to generate a representation of the trace sensor data. Subsequently, this representation may be provided to a trained decoder model to generate the reconstructed sensor data. The autoencoder model may be an encoder-decoder model. In some embodiments, the machine learning model may be trained by providing data (e.g., preprocessed sensor trace data) to the model. This model may be configured to generate a representation of the input data (e.g., a reduced dimensionality representation). This model may be configured to generate reconstructed data based on the representation of the input data. The model may then be trained by minimizing the difference between the input data and the reconstructed data (e.g., according to a configuration setting, to avoid not changing the trace sensor data to generate the representation).

[0201] At block 564, processing logic compares the trace sensor data to the reconstructed sensor data. In some embodiments, the machine learning model is trained using data that meets the thresholds, e.g., data within control limits, data where a specific portion is within control limits, data associated with a process that produces a product that meets performance thresholds, golden run data, etc. The trace sensor data provided to the trained machine learning model may or may not be associated with an irregular process run, e.g., a process run where the data is outside of the control limits.

[0202] At block 566, processing logic determines one or more differences between the reconstructed sensor data and the trace sensor data. The differences may be determined based on a comparison performed in association with block 564. In some embodiments, the trace sensor data may differ from the training data in some way, for example, the trace sensor data may be associated with a process run that does not produce an acceptable product, the trace sensor data may be outside one or more control limits, etc. The difference between the training set and the trace sensor data may cause the trace sensor data and the reconstructed data to differ significantly. The reconstruction error may be used to determine whether the input data significantly exceeds the range of the training data. Since the machine learning model is trained using data from a healthy tool (e.g., a tool with relatively new parts and producing products that meet quality standards), when the trace data comes from a healthy tool, the machine learning model produces an accurate reproduction of the trace data. However, when the reconstructed data differs from the trace data, this indicates that the trace data differs from the trace data of the healthy tool in some way. In some embodiments, the reconstruction error (e.g., the error in each data point of the trace data) may be aggregated, for example, to generate an error metric associated with the sensor and the process run (e.g., a product).

[0203] At block 568, processing logic determines whether to recommend a corrective action associated with the processing chamber. This determination may be based on one or more differences between the trace sensor data and the reconstructed sensor data. This determination may be based on aggregated error metrics. In some embodiments, sensors may be associated with subsystems. Error metrics may be further aggregated (e.g., as root mean square error) into subsystems. In some embodiments, visualizations may be generated. Visualizations may be organized by run, by subsystem, by sensor, etc.

[0204] Figure 5H Flowchart of method 500H for performing outlier cluster analysis according to some embodiments. Details of triggering and processing of outlier clustering may be configured. In some embodiments, outlier clustering may be triggered by multiple outlier points meeting a threshold in an earlier analysis. At block 570, sensor contribution values ​​are extracted by processing logic. Combined Figure 5D Block 532 of discusses the sensor contribution calculation in more detail. In some embodiments, each sensor contribution index (e.g., severity index, severity distance, severity score, etc.) used for clustering can be associated with a single run and sensor. In some embodiments, a clustering approach can be used to analyze multiple data points (e.g., from multiple sensors in multiple processing runs). Due to the large number of sensors associated with the tool, this data may occupy a high-dimensional space. In some embodiments, most of the data will be normal value data, such as data within control limits. In some embodiments, the sensor contribution value of the normal value can be defined as zero. In some embodiments, only data points with sensor contribution values ​​that meet the threshold value can be used for cluster analysis.

[0205] At block 572, data preprocessing for cluster analysis is performed. Data preprocessing may include cleaning the data set, for example, to include only the operation of sensors that exhibit outlier behavior. Other preprocessing may be performed, including normalization, etc. At block 574, a dimensionality reduction operation is performed by a processing device. In some embodiments, a high-dimensional data set may be simplified to a space that is more easily visualized to improve analytical capabilities to adapt to clustering models, etc. In some embodiments, the data set may be simplified to a two-dimensional data set. In some embodiments, an algorithm may place data points that are close together in a high-dimensional space close together in a low-dimensional space. Proximity may be determined by Euclidean distance, cosine distance, or other distance metrics. In some embodiments, a nonlinear dimensionality reduction model may be used. In some embodiments, a t-random nearest neighbor embedding method may be used for dimensionality reduction. Dimensionality reduction may be performed according to configuration settings (e.g., distance metrics, the expected number of nearest neighbors, etc.).

[0206] At block 576, a clustering operation is performed by the processing device. The clustering operation can be performed on the data reconstructed in the low-dimensional space. In some embodiments, a clustering model that selects the number of clusters in the data can be used. In some embodiments, density-based spatial clustering with the application of noise methods can be used for clustering operations. Clustering can identify multiple groups of data points that are closely represented together in the low-dimensional space, which indicates that the probability of the data points being closely represented in the high-dimensional space is high. These data points may be related to the same root cause. The clustering operation can be performed according to configuration settings. The configuration settings may include distance metrics, the minimum number of samples in a cluster, the maximum distance between two samples that are considered adjacent, etc.

[0207] At block 578, processing logic ranks contributions of sensor systems to one or more clusters according to some embodiments. The data points of a cluster may be analyzed by examining the contribution values ​​of the sensors that contributed to the data points. In some embodiments, the sensors associated with the data points may be ranked by sensor contribution values ​​(e.g., severity distance). In some embodiments, the sensors with the highest contributions (e.g., the three sensors with the highest contribution values) may be used to assign major subsystems of the manufacturing system to data points, clusters, etc. In some embodiments, the clustering results may be visualized by processing logic. Figure 5D and Figure 6C Block 535 discusses outlier cluster visualization in more detail.

[0208] Fig.5IFlowchart of a method 500I for performing normal value drift detection according to some embodiments. Normal value drift detection may be triggered and / or performed according to configuration settings. Some operations of the TTTM analysis module utilize outlier data points (e.g., TTTM index, sensor contribution / severity metrics, etc.). In some embodiments, analysis of normal value data points (e.g., data points within control limits) may be used to determine system faults, drifts, corrective actions to be taken, etc.

[0209] At block 580, reference data is set for normal value drift detection. In some embodiments, normal value drift detection includes comparing data (e.g., data collected over a period of time) to reference data. In some embodiments, a golden data set is provided or selected. The golden data set may include data from a process run associated with a product manufactured within specification. The golden data set may include data from a process run performed shortly after a successful tool installation or preventive maintenance.

[0210] At block 582, a warp path search is performed. In some embodiments, outlier drift detection includes using an elastic dissimilarity model. Such a model changes the spacing between data points of a data set (without changing the order of the data points) to produce a best match between the two sets, for example, as measured by summing the differences between the data values ​​(or interpolations of the data values) of the two sets. In some embodiments, a dynamic time warping algorithm is applied to a reference data set and a data set to be analyzed. In some embodiments, the data set includes summary data. In some embodiments, the data set includes summary data collected over a time span (e.g., many processing runs). The warp path determines how to stretch one data set (e.g., the data to be analyzed) to match a pattern in another data set (e.g., golden data).

[0211] At block 584, a distortion point count is determined. The distortion point count is a measure of how far a data set must be distorted to provide the best match to another data set. Other metrics may also be used to indicate the degree of distortion, such as an aggregate measure of how far the data points of one set must be offset. The distortion point count is a measure of how many pairs of matching comparison points there are between two data sets. In some embodiments, each point of a target data set (e.g., the data set being analyzed) is matched to a "nearest neighbor" point of a reference data set (e.g., a golden data set). In some embodiments, each point of the reference data set is matched to a nearest neighbor point of the target data set. If the two data sets are well aligned, then each data point of one set may be matched to another data point in the opposite set, and the number of matches is the same as the number of data points. If the two data sets are not well aligned, some data points in one or both sets may be matched to multiple points of the opposite set, and the number of matches may be greater than the number of data points. The matching point count can be extended to situations where the reference data and the target data have different numbers of data points. In some embodiments, since each data point will have at least one matching point, the count of matching points may remove the number of data points (e.g., the number of guaranteed matches), leaving only the number of additional matches due to the distortion of one data set, e.g., the distortion point count. At block 586, a distortion rate calculation is performed by processing logic. The distortion rate is a comparison indicating how distorted the data set is overall (e.g., corrected for the size of the data set). In some embodiments, the distortion rate may be calculated as

[0212]

[0213] Where WR is the distortion rate, and 100 is used to convert the distortion rate calculation into a percentage. In some embodiments, a corresponding similarity rate may be calculated, for example, 100% minus the distortion rate. In some embodiments, a corrective action may be triggered based on the similarity rate or the distortion rate. In some embodiments, a corrective action may be triggered based on whether the similarity rate, the distortion rate, etc. meet a threshold value. The corrective action may include sending an alert to a user, performing additional drill-down analysis, updating a recipe, scheduling maintenance, etc.

[0214] FIG. 6A to FIG. 6G Depicted are example visualizations associated with aspects of TTTM analysis, according to some embodiments. Fig. 6A 6A depicts an operation for segmenting a trace into transient and steady-state portions according to some embodiments. The visualization 600A includes a time trace 602. The time trace 602 may be associated with a sensor and a processing operation (e.g., a complete runtime trace has been segmented, such as by Figure 4CThe operation segmenter 451 of the embodiment of the present invention is segmented to generate a time trace 602. The window 604 is used to determine whether a group of data points (e.g., data points within the window) belong to the transient or steady-state part of the data. The metric value of the points within the window can be used to classify the points in the window as transient or steady-state. In some embodiments, if the standard deviation of the points in the window is higher than a threshold value, the points of the window are determined to be in the transient part.

[0215] In some embodiments, window 604 moves in the time trace to classify the data points of the trace as transient or steady state, as shown by the arrow and dashed window to the right of trace 602. In some embodiments, time trace 602 can be divided into two or more parts. In some embodiments, time trace 602 can be separated into a transient head 606, a steady state part 607, and a transient tail 608.

[0216] Figure 6B A visualization 600B of a digital twin output for summary data generation operations according to some embodiments is depicted. One or more components of a manufacturing facility can be modeled by a digital twin. A digital twin includes a digital representation of one or more physical assets. Using a digital twin, system responses (e.g., control adjustments to meet set points) can be modeled. Various features of the digital twin response can be compared to sensor data to generate summary data.

[0217] Visualization 600B includes a time trace 610. In some embodiments, the time trace 610 may be associated with a transient portion of a processing operation. The digital twin model may include a target set point 611 associated with a property value measured by a sensor in a physical manufacturing system. Features of the digital twin time trace include an overshoot 612. The time trace 610 may exceed the set point (or, in some embodiments, drop below the set point) before settling to the set point value. One or more of the values ​​of the overshoot 612 (or a normalized or otherwise processed version of the overshoot 612) or the peak time 613 may be used to generate summary data. In some embodiments, the rise time 614 may be used to generate summary data. In some embodiments, the rise time 614 may be measured at the first time that the time trace 610 meets the set point 611. In some embodiments, the rise time 614 may be measured at the first time that the time trace 610 reaches a threshold value (e.g., 80% of the difference between the set point and the starting value, as shown in visualization 600B).

[0218] In some embodiments, the period of oscillation (e.g., oscillation 615) can be used to generate summary data. Properties such as oscillation frequency, amplitude, etc. can be used to generate summary data. Settling time 617 (e.g., the time that time trace 610 remains within error window 616 of set point 611) can be used to generate summary data. In some embodiments, the set point error (e.g., the difference between the final value of time trace 610 and set point 611) can be used to generate summary data.

[0219] Figure 6C Depicts a visualization of the results of a clustering operation according to some embodiments, such as combining Figure 5H Those discussed. Outlier data points (e.g., data points indicating abnormal or non-optimal performance) can be used for clustering operations. Clustering operations can be used to identify the root causes (e.g., subsystems) that cause outlier data (e.g., causing non-optimal performance of manufacturing equipment). In some embodiments, high-dimensional sensor data is converted to a low-dimensional space, for example, by a dimensionality reduction model. In some embodiments, the high-dimensional space is reduced to a two-dimensional space. In some embodiments, the dimensionality reduction algorithm places points that are close together in the high-dimensional space (e.g., by Euclidean distance, cosine distance, etc.) close together in the low-dimensional space. In some embodiments, data points with the same (or related) root causes (e.g., subsystems that cause the data points to be outliers) can be represented as close together in the low-dimensional space. In some embodiments, the contribution of sensors to data points can be used to assign subsystems to clusters.

[0220] Visualization 600C depicts outlier points in a low-dimensional space. A clustering algorithm (e.g., a density-based spatial clustering application) can be used to separate data points into groups that may be associated with the same root cause. Exemplary visualization 600C is divided into four clusters 640, 641, 642, and 643. The highest contributing sensor for each cluster (e.g., the sensor with the highest severity metric) can be used to label each cluster with a manufacturing subsystem. For example, cluster 640 can be found to be associated with a heater power subsystem, cluster 642 can be found to be associated with an electrostatic chuck temperature, cluster 643 can be found to be associated with airflow in certain parts of the manufacturing system, etc.

[0221] FIG. 6D to FIG. 6E Depicted is a visualization of normal value drift detection elastic dissimilarity model results according to some embodiments. Fig.6D The results 600D of the elastic dissimilarity model applied to the target time trace 650 and the reference time trace 651 are depicted. Fig.6D (and Fig. 6E) is the original sequential data, e.g., summary data collected in multiple processing runs, with the result of distortion (e.g., application of an elastic dissimilarity model) shown by the dashed lines indicating matching point pairs. Result 600D is produced by two very similar traces. As highlighted by the circles, only a few points in either trace match multiple points in the opposite trace, and in each case the number of matching points is small. Target trace 650 and reference trace 651 are similar in time, as indicated by the fairly ordered (e.g., mostly parallel) dashed lines.

[0222] Fig. 6E The results 600E of the elastic dissimilarity model applied to a target time trace 660 and a reference time trace 661 are depicted. Fig.6D 660. Many metrics can be used to quantify the degree of distortion of the target trace 660, for example, to trigger corrective action. The result 600E includes a few data points that match a large number of points in the opposing trace. One method of indicating the degree of distortion associated with a pair of temporal traces includes comparing the number of matches present in a pair of traces that match well to the number of matches present in the analysis. Other methods include characterizing the aggregate displacement of matches in time (e.g., Fig. 6E The degree to which the dotted line is stretched from left to right), the slope of the matching line, etc.

[0223] FIG. 6F to FIG. 6G Depicted are example visualizations 600F and 600G of trace data analysis results according to some embodiments. Fig. 6F Includes heat map 670 and legend 671. The reconstruction error of the autoencoder model can be used to generate heat map 670. In some embodiments, the reconstruction error (e.g., related to the difference between the input data and the reconstructed data at each timestamp of the time trace) can be aggregated into a single error number for each trace. In some embodiments, the reconstruction error can be further aggregated by the sensor subsystem. In some embodiments, a single value can be generated for a processing run and a manufacturing subsystem. Heat map 670 can be generated from multiple subsystems (e.g., subsystems A, B, C and D, such as vacuum subsystems, temperature subsystems, clamping subsystems, etc.) and multiple processing runs (e.g., substrates produced). Legend 671 can visually indicate subsystem / run pairs with high reconstruction errors, for example, subsystem / run pairs that may have been performed non-optimally. Example visualization 600F depicts a particularly high reconstruction error associated with run number 5, subsystem B. In some embodiments, visualization 600F can be generated as a corrective action, for example, an alarm that can be viewed as provided to a user. In some embodiments, a visualization such as visualization 600F may be used to take corrective action, for example, visualization 600F may indicate that maintenance on subsystem B is appropriate.

[0224] Figure 6G An example visualization 600G of the output of an autoencoder model associated with trace data analysis according to some embodiments is depicted. The visualization 600G includes an input data pattern 680 and a reconstructed data pattern 681. In some embodiments, the visualization 600G may be generated (or displayed to a user) in response to detecting a high reconstruction error, for example, in response to detecting a high reconstruction error in a scene such as Fig. 6F 600G). In some embodiments, non-optimal performance may be associated with high reconstruction errors (e.g., large differences between input trace data pattern 680 and reconstructed data pattern 681), as shown in visualization 600G.

[0225] Figure 7 A diagram of the operation of a model 700 (e.g., a machine learning model) capable of reducing the dimensionality of input data (e.g., an autoencoder model) according to some embodiments.

[0226] The input data 710 of the model 700 is data associated with the production of the substrate. In some embodiments, the input data 710 includes one or more of metrology data, manufacturing parameters, sensor data, or a combination thereof. The input data 710 may be pre-processed data. In some embodiments, the input data 710 is metrology data associated with the substrate. The metrology data may be of any (or many) types, including thickness, in-plane displacement, chemical properties, electronic properties, optical properties, etc. In some embodiments, the input data 710 includes trace sensor data and / or a pre-processed version of the trace sensor data associated with product manufacturing.

[0227] The model 700 includes a first part 720 (e.g., an encoder) and a second part 740 (e.g., a decoder). In some embodiments, the model is one or more of an autoencoder, a neural network model, a convolutional neural network model, a deep belief network, a feedforward neural network, a multi-layer neural network, etc. The first part 720 reduces the dimension of the input data 710 (e.g., metering data) to a compressed form (e.g., compressed data 730). The input layer of the machine learning model can be separated from the compressed data by multiple hidden layers ( Figure 7Two are depicted in FIG, but any number of hidden layers may be used). During training of the machine learning model 700, the first portion 720 may generate one or more functions to fit the input data 710 into a lower dimensional representation without user guidance. The reduction (e.g., compression, encoding) may be performed in several stages (i.e., first converting the input data 710 into partially compressed data and then further converting into compressed data 730), or the reduction (e.g., compression, encoding) may be completed in a single stage.

[0228] The second part 740 takes the compressed data 730 as input and produces output data 750. During training, the model 700 is trained to minimize the difference between the input data 710 and the output data 750, where the output 750 is the reconstruction of the input data 710 from the compressed data 730. The minimization function used to train the model 700 may also impose a penalty on the dimensionality of the compressed data 730 to avoid returning a function that under-compresses (e.g., an identity function that perfectly reconstructs the input data 710 but does not compress the data to a reduced dimension). The model 700 may be trained so that the output data 750 of the model 700 approximately matches the input data 710.

[0229] The functions used by the first portion 720 and the second portion 740 may be nonlinear in nature. All of the processes of the model 700 (i.e., both dimensionality reduction and reconstruction, both encoding and decoding, etc.) may be used in some applications. In other applications, only some of the functions may be used. For example, during training, the model 700 may pass the input data 710 through the first portion 720 to form compressed data 730, then pass it through the second portion 740 to determine the output data 750, and then compare the output data 750 to the input data 710 to determine whether the output data 750 is substantially similar to the input data 710.

[0230] In some embodiments, the model 700 can be used for anomaly trace analysis. The model 700 can be trained based on historical data. The historical training data may be limited to successful runs, for example, runs where the properties of the product produced were within a threshold range, runs where chamber conditions were achieved within a threshold range, etc. Subsequently, the model 700 can be configured to generate compressed data for future successful manufacturing runs. In some embodiments, the model 700 can be trained and used for data associated with a set (or a limited number) of equipment, recipes, product designs, etc.

[0231] The model 700 may generate output data 750 with some reconstruction error (e.g., difference from the input data 710). In some embodiments, input data 710 that is similar in some way to the training input used to train the model 700 may generate output data 750 with relatively low reconstruction error. A reconstruction error index (e.g., sum of errors from data point to data point, squared error, normalized sum, root mean square error, etc.) may be generated to represent a measure of the total reconstruction error of the data set input into the model 700. In some embodiments, the magnitude of the error index may indicate the similarity between the input data 710 and the data used to train the model 700. In some embodiments, a high reconstruction error (e.g., substantial difference between the output data 750 and the reconstructed data generated from the compressed data 730) may indicate that the input data 710 is substantially different from the data used to train the model 700. In some embodiments, the model 700 is trained strictly on "good" data (e.g., data associated with a manufacturing run that meets a performance threshold), and a large reconstruction error index may indicate that the input data 710 is non-optimal. Reconstruction errors can be calculated per run (e.g., per product produced), per sensor, per tool, per subsystem, etc., and can be visualized in a convenient format to facilitate corrective action.

[0232] Figure 8 800 is a block diagram showing a computer system 800 according to certain embodiments. In some embodiments, the computer system 800 may be connected to other computer systems (e.g., via a network such as a local area network (LAN), an intranet, an inter-network, or the Internet). The computer system 800 may operate in the capacity of a server or client computer in a client-server environment, or as a peer computer in a peer-to-peer or distributed network environment. The computer system 800 may be provided by a personal computer (PC), a tablet PC, a set-top box (STB), a personal digital assistant (PDA), a cellular phone, a network device, a server, a network router, a switch or a bridge, or any device capable of executing a set of instructions (sequentially or otherwise), wherein the set of instructions specifies the actions to be taken by this device. In addition, the term "computer" shall include any computer collection that executes a set (or multiple sets) of instructions individually or jointly to perform any one or more methods described herein.

[0233] On the other hand, the computer system 800 may include a processing device 802, a volatile memory 804 (e.g., a random access memory (RAM)), a non-volatile memory 806 (e.g., a read-only memory (ROM) or an electrically erasable programmable ROM (EEPROM)), and a data storage device 818. The processing device 802, the volatile memory 804 (e.g., a random access memory (RAM)), the non-volatile memory 806 (e.g., a read-only memory (ROM) or an electrically erasable programmable ROM (EEPROM)), and the data storage device 818 can communicate with each other via a bus 808.

[0234] The processing device 802 may be provided by one or more processors, such as a general-purpose processor (e.g., a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a microprocessor that implements other types of instruction sets, or a microprocessor that implements a combination of various types of instruction sets) or a special-purpose processor (e.g., an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), or a network processor).

[0235] The computer system 800 may also include a network interface device 822 (e.g., coupled to the network 874). The computer system 800 may also include a video display unit 810 (e.g., LCD), an alphanumeric input device 812 (e.g., keyboard), a cursor control device 814 (e.g., mouse), and a signal generating device 820.

[0236] In some embodiments, the data storage device 818 may include a non-transitory computer-readable storage medium 824 (e.g., a non-transitory machine-readable medium) on which instructions 826 encoding any one or more of the methods or functions described herein may be stored, including instructions for Figure 1 The components (e.g., prediction component 114, corrective action component 122, model 190, etc.) of the system are encoded and used to implement instructions for the methods described herein.

[0237] During execution of the instructions 826 by the computer system 800 , the instructions 826 may also reside completely or partially within the volatile memory 804 and / or the processing device 802 , and thus, the volatile memory 804 and the processing device 802 may also constitute machine-readable storage media.

[0238] Although computer-readable storage medium 824 is shown as a single medium in the illustrative example, the term "computer-readable storage medium" shall include a single medium or multiple media (e.g., a centralized or distributed database, and / or associated caches and servers) that store one or more sets of executable instructions. The term "computer-readable storage medium" shall also include any tangible medium that can store or encode a set of instructions for execution by a computer, the instructions causing the computer to perform any one or more of the methods described herein. The term "computer-readable storage medium" shall include, but is not limited to, solid-state memories, optical media, and magnetic media.

[0239] The methods, components, and features described herein may be implemented by discrete hardware components, or may be integrated into the functionality of other hardware components, such as ASICS, FPGAs, DSPs, or similar devices. Furthermore, the methods, components, and features may be implemented by firmware modules or functional circuits within a hardware device. Furthermore, the methods, components, and features may be implemented in any combination of hardware devices and computer program components, or in a computer program.

[0240] Unless expressly specified otherwise, terms such as "receive," "perform," "provide," "obtain," "make," "access," "determine," "add," "use," "train," "reduce," "generate," "correct," and the like refer to operations and processes performed or implemented by a computer system that manipulate and transform data represented as physical (electronic) quantities within computer system buffers and memories into other data similarly represented as physical quantities within computer system memories or buffers or other such information storage, transmission, or display devices. In addition, the terms "first," "second," "third," "fourth," and the like used herein are intended to be labels for distinguishing between different elements, and may not have ordinal meanings according to the numerical names of the different elements.

[0241] The examples described herein also relate to an apparatus for performing the methods described herein. This apparatus may be specially constructed for performing the methods described herein, or the apparatus may comprise a general purpose computer system selectively programmed by a computer program stored in the computer system. Such a computer program may be stored in a computer readable tangible storage medium.

[0242] The methods and illustrative examples described herein are not inherently related to any particular computer or other device. Various general-purpose systems may be used in accordance with the teachings described herein, or it may prove convenient to construct more specialized equipment to perform the methods described herein and / or their individual functions, routines, subroutines, or operations. Structural examples of various such systems are set forth in the description above.

[0243] The above description is intended to be illustrative rather than limiting. Although the present disclosure has been described with reference to specific illustrative examples and embodiments, it should be appreciated that the present disclosure is not limited to the described examples and embodiments. The scope of the present disclosure should be determined with reference to the following claims and the full scope of equivalents to which the claims are entitled.

Claims

1. A method comprising: receiving trace sensor data associated with a first manufacturing process in a processing chamber; processing the trace sensor data using one or more trained machine learning models, the one or more trained machine learning models generating a representation of the trace sensor data, and subsequently generating reconstructed sensor data based on the representation of the trace sensor data, wherein the one or more trained machine learning models output the reconstructed sensor data; comparing the trace sensor data to the reconstructed sensor data; determining one or more differences between the reconstructed sensor data and the trace sensor data based on the comparison; and A determination is made based on the one or more differences between the trace sensor data and the reconstructed sensor data whether a corrective action associated with the processing chamber is recommended.

2. The method of claim 1, further comprising: receiving a quality index score associated with the processing chamber; determining whether the quality index score satisfies one or more criteria; and In response to the value of the quality index score failing to satisfy the one or more criteria, providing the trace sensor data to the one or more trained machine learning models.

3. The method of claim 1, wherein the one or more trained machine learning models comprise an encoder-decoder model, wherein an encoder of the encoder-decoder model generates the representation of the trace sensor data, and wherein a decoder of the encoder-decoder model generates the reconstructed sensor data.

4. The method of claim 1, further comprising: The historical trace sensor data is provided as training input to the one or more trained machine learning models, wherein the one or more trained machine learning models include a first part and a second part, the first part is configured to generate a representation of the input data by reducing the dimension of the input data, and the second part is configured to reconstruct the input data based on the representation of the input data, wherein after training, the reconstructed data output by the second part of the one or more trained machine learning models approximately matches the input data.

5. The method of claim 1, further comprising: The one or more differences between the trace sensor data and the reconstructed data of the sensor are aggregated to produce a measure of an overall error of the sensor.

6. The method of claim 5, further comprising: generating a first set of sensors, wherein the sensors of the first set each contribute to a subsystem of the processing chamber; and The measure of total error of the sensor and the measure of total error of the sensors of the first set of sensors are aggregated to produce an indication of total error of the subsystem.

7. The method of claim 1, wherein the corrective action comprises one or more of the following: Schedule preventive maintenance; Schedule corrective maintenance; Update the treatment recipe; or Provide an alert to the user. 8 . The method of claim 7 , wherein providing an alert to a user comprises generating a visualization depicting the one or more differences between the trace sensor data and the reconstructed sensor data.

9. A system comprising a memory and a processing device coupled to the memory, wherein the processing device is configured to: receiving trace sensor data associated with a first manufacturing process in a processing chamber; processing the trace sensor data using one or more trained machine learning models, the one or more trained machine learning models generating a representation of the trace sensor data, and subsequently generating reconstructed sensor data based on the representation of the trace sensor data, wherein the one or more trained machine learning models output the reconstructed sensor data; comparing the trace sensor data to the reconstructed sensor data; determining one or more differences between the reconstructed sensor data and the trace sensor data based on the comparison; and A determination is made based on the one or more differences between the trace sensor data and the reconstructed sensor data whether a corrective action associated with the processing chamber is recommended.

10. The system of claim 9, wherein the processing device is further configured to: receiving a quality index score associated with the processing chamber; determining whether the quality index score satisfies one or more criteria; and In response to the value of the quality index score failing to satisfy the one or more criteria, providing the trace sensor data to the one or more trained machine learning models.

11. The system of claim 9, wherein the one or more trained machine learning models comprise an encoder-decoder model, wherein an encoder of the encoder-decoder model produces the representation of the trace sensor data, and wherein a decoder of the encoder-decoder model produces the reconstructed sensor data.

12. The system of claim 9, wherein the processing device is further configured to: The historical trace sensor data is provided as training input to the one or more trained machine learning models, wherein the one or more trained machine learning models include a first part and a second part, the first part is configured to generate a representation of the input data by reducing the dimension of the input data, and the second part is configured to reconstruct the input data based on the representation of the input data, wherein after training, the reconstructed data output by the second part of the one or more trained machine learning models approximately matches the input data.

13. The system of claim 9, wherein the processing device is further configured to: One or more differences between the trace sensor data and the reconstructed data of the sensor are aggregated to produce a measure of the total error of the sensor.

14. The system of claim 13, wherein the processing device is further configured to: generating a first set of sensors, wherein the sensors of the first set each contribute to a subsystem of the processing chamber; and The measure of total error of the sensor and the measures of total error of the sensors of the first set of sensors are aggregated to produce an indication of total error of the subsystem.

15. The system of claim 9, wherein the corrective action comprises providing an alert to a user, and wherein providing an alert to a user comprises generating a visualization depicting the one or more differences between the trace sensor data and the reconstructed sensor data.

16. A non-transitory machine-readable storage medium storing instructions that, when executed, cause a processing device to perform operations comprising: receiving trace sensor data associated with a first manufacturing process in a processing chamber; processing the trace sensor data using one or more trained machine learning models, the one or more trained machine learning models generating a representation of the trace sensor data, and subsequently generating reconstructed sensor data based on the representation of the trace sensor data, wherein the one or more trained machine learning models output the reconstructed sensor data; comparing the trace sensor data to the reconstructed sensor data; determining one or more differences between the reconstructed sensor data and the trace sensor data based on the comparison; and A determination is made based on the one or more differences between the trace sensor data and the reconstructed sensor data whether a corrective action associated with the processing chamber is recommended.

17. The non-transitory machine-readable storage medium of claim 16, wherein the operations further comprise: receiving a quality index score associated with the processing chamber; determining whether the quality index score satisfies one or more criteria; and In response to the value of the quality index score failing to satisfy the one or more criteria, providing the trace sensor data to the one or more trained machine learning models.

18. The non-transitory machine-readable storage medium of claim 16, wherein the one or more trained machine learning models include an encoder-decoder model, wherein an encoder of the encoder-decoder model generates the representation of the trace sensor data, and wherein a decoder of the encoder-decoder model generates the reconstructed sensor data.

19. The non-transitory machine-readable storage medium of claim 16, wherein the operations further comprise: The historical trace sensor data is provided as training input to the one or more trained machine learning models, wherein the one or more trained machine learning models include a first part and a second part, the first part is configured to generate a representation of the input data by reducing the dimension of the input data, and the second part is configured to reconstruct the input data based on the representation of the input data, wherein after training, the reconstructed data output by the second part of the one or more trained machine learning models approximately matches the input data.

20. The non-transitory machine-readable storage medium of claim 16, wherein the corrective action comprises providing an alert to a user, and wherein providing the alert to the user comprises generating a visualization depicting the one or more differences between the trace sensor data and the reconstructed sensor data.

Images