Protective tape in substrate processing system

Abstract

A method includes identifying tracking data including a plurality of data points, the tracking data associated with producing a substrate having an attribute value satisfying a threshold via a substrate processing system. The method further includes determining a guard band violation data point of the plurality of data points of the tracking data based on the guard band. The method further includes determining a guard band violation shape characterization based on the guard band violation data point. Classification of additional guard band violation data points for additional tracking data will be based on guard band violation shape characterization. A corrective action associated with the substrate processing system is performed based on the classification.

Description

Technical Field

[0001] The present disclosure relates to guardbands, and in particular to guardbands in substrate processing systems. Background Art

[0002] A product may be produced by performing one or more manufacturing processes using a manufacturing device. For example, a substrate processing device may be used to produce a substrate through a substrate processing operation. A product having specific properties will be produced. Sensor data is monitored in association with the substrate manufacturing process. Summary of the invention

[0003] Below is a simplified summary of the disclosure to provide a basic understanding of some aspects of the disclosure. This summary is not a broad overview of the disclosure. This summary is neither intended to identify the key or important elements of the disclosure, nor is it intended to describe any scope of a specific embodiment of the disclosure or any scope of the claims. The sole purpose of this summary is to present some concepts of the disclosure in a simplified form as a preface to a more detailed description presented later.

[0004] In one aspect of the present disclosure, a method includes: identifying tracking data including a plurality of data points, the tracking data associated with producing a substrate having a property value satisfying a threshold value via a substrate processing system. The method further includes: determining a guard band violation data point for the plurality of data points of the tracking data based on the guard bands. The method further includes: determining a guard band violation shape characterization based on the guard band violation data point. Classifying additional guard band violation data points of the additional tracking data based on the guard band violation shape characterization. Performing a corrective action associated with the substrate processing system based on the classification.

[0005] In one aspect of the present disclosure, a method includes: identifying tracking data including a plurality of data points, the tracking data associated with production of a substrate via a substrate processing system. The method further includes: determining a guard band violation data point for the plurality of data points of the tracking data based on the guard bands. The method further includes: determining a classification of the guard band violation data point based on a guard band violation shape characterization. Performing a corrective action associated with the substrate processing system based on the classification of the guard band violation data point.

[0006] In one aspect of the present disclosure, a non-transitory computer-readable storage medium stores thereon instructions that, when executed, cause a processing device to perform operations, the operations comprising: identifying tracking data comprising a plurality of data points, the tracking data associated with producing a substrate having a property value satisfying a threshold value via a substrate processing system. The operations further comprising: determining a guard band violation data point for the plurality of data points of the tracking data based on the guard bands. The operations further comprising: determining a guard band violation shape characterization based on the guard band violation data point. Classifying additional guard band violation data points of the additional tracking data based on the guard band violation shape characterization. Performing a corrective action associated with the substrate processing system based on the classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] The present disclosure is illustrated by way of example and not by way of limitation in the figures of the accompanying drawings.

[0008] FIG. 1 is a block diagram illustrating an exemplary system architecture, according to certain implementations.

[0009] 2A-2D are flow diagrams of methods associated with generating guard bands according to certain embodiments.

[0010] 3A-3D are flow diagrams of methods associated with guard band violation analysis according to certain embodiments.

[0011] 4A-4D are flow diagrams of methods associated with dynamically acceptable regions outside of guard band limits according to certain embodiments.

[0012] 5A-5E illustrate graphs of varying allowed types according to certain implementations.

[0013] 6A-6E illustrate guard band violation analysis according to certain embodiments.

[0014] 7A-7F illustrate dynamically acceptable regions outside of guard band limits according to certain embodiments.

[0015] 8A-8B illustrate guard band adaptation according to certain embodiments.

[0016] 9 is a block diagram illustrating a computer system, according to some implementations. DETAILED DESCRIPTION

[0017] Described herein are techniques related to guard bands in substrate processing systems (e.g., guard band enhancement, guard band violation analysis, and dynamic regions outside of guard bands). Guard bands may be upper and lower thresholds around target values ​​(e.g., an allowable error range). Sensor data may be compared to the guard bands to determine whether the sensor data is within the threshold (e.g., a good substrate) or outside the threshold (e.g., a bad substrate).

[0018] Products are produced by performing one or more manufacturing processes using manufacturing equipment. Products are produced to have specific attributes. For example, substrate processing equipment is used to produce substrates through substrate processing operations. Substrates that meet specific attribute values ​​(e.g., dimensions verified by metrology data) will be used, and substrates that do not meet specific attribute values ​​will be discarded. Sensor data associated with substrate processing operations are collected over time. There are many purposes for monitoring sensor data, including trying to produce substrates that meet specific attribute values, trying to verify that the equipment is operating properly, trying to determine when the equipment needs to be repaired or replaced now or in the future, and trying to determine adjustments to equipment parameters to make the process more efficient (e.g., in terms of yield or quality, throughput or quantity, cost or life). For the purpose of simplicity of explanation, this discussion will focus on the goal of trying to produce substrates that meet specific attribute values.

[0019] Traditionally, sensor data is aggregated across a specific recipe or recipe operation associated with a piece of equipment producing a substrate. This aggregation is defined using a set of statistics, such as mean and variance. These aggregate statistics are then compared to set limits, such as the mean being between a lower and upper limit. If the sensor data is within the limits, the substrate is estimated to meet the specific attribute value, and if the sensor data is outside the limits, the substrate is estimated to not meet the specific attribute value. Limits that are too narrow can result in false positives (e.g., incorrectly predicting that the substrate will not meet the attribute value). Limits that are too wide can result in false negatives (e.g., incorrectly predicting that the substrate meets the attribute value). Variations in sensor data and variations in substrate handling equipment can result in many false positives. Expanding the limits to allow for variations in sensor data and substrate handling equipment can result in many false negatives. Traditional systems that incorrectly flag substrates as meeting or not meeting attribute values ​​can result in wasted materials, reduced yields, product defects, increased user time, increased equipment downtime, and more. Attempting to fix incorrect flags requires additional processing overhead, bandwidth, energy consumption, metrology operations, user time, and more.

[0020] The methods, apparatus, and systems of the present disclosure provide guard band improvements in substrate processing systems that address these and other deficiencies of conventional solutions.

[0021] In some embodiments, a processing device identifies tracking data associated with the production of substrates (e.g., good substrates) having property values ​​that meet a threshold. The tracking data may include a collection of sensor data sets associated with the production of different substrates and from different types of sensors. In some embodiments, the data is analyzed at a trace level by using guard bands (e.g., as opposed to providing only summary statistics of sensors for an entire recipe or recipe operation). The guard bands may provide upper and lower limits over the length of the entire tracking data. This provides many advantages over summary statistics (e.g., the present disclosure may identify and analyze specific violations in the tracking data). The processing device generates an initial guard band based on the tracking data (e.g., 3 times the standard deviation (3 sigma) around the mean of the tracking data for each time window in the data, where 3 sigma is determined by analyzing multiple data runs for a specific time window). The processing device determines an allowable type of variation of the guard band based on the tracking data. Since the tracking data is for good wafers, the allowable type of variation of the guard band may be a time shift of one or more of the sensor data sets (e.g., a shift in the x direction) to meet the mean of the sensor data. The allowable type of variation may include a guard band upper limit that is different from the mean of the tracking data than a guard band lower limit. The allowed type of change may include a wider guard band limit and a narrower guard band limit at a particular portion of the guard band. The processing device generates the guard band based on the tracking data and the allowed type of change. The processing device compares the additional tracking data (e.g., which may belong to a good or bad substrate) to the guard band, and in response to one or more data points not being within the guard band, the processing device causes a corrective action to be performed (e.g., discarding the substrate, interrupting a substrate processing operation, initiating a substrate inspection, etc.).

[0022] In some embodiments, tracking data for both good and bad substrates is used. Guard bands are determined or updated based on a better understanding of the differences between good and bad tracking data, where the bad tracking data may be a single category or multiple categories representing different modes or degrees of bad substrates.

[0023] In some embodiments, a processing device identifies tracking data associated with production of substrates having property values ​​that satisfy a threshold (e.g., good substrates). The processing device determines a guard band violation data point of the tracking data based on the guard bands. The processing device determines a guard band violation shape representation based on the guard band violation data point. The processing device compares additional tracking data (e.g., that may belong to good or bad substrates) to the guard bands to determine the guard band violation data point. The processing device determines a classification of the guard band violation data point (e.g., whether the guard band violation data point corresponds to a good wafer or a bad wafer) based on the guard band violation shape representation.

[0024] In some embodiments, a processing device identifies tracking data associated with the production of substrates (e.g., good substrates) having property values ​​that meet a threshold. The processing device determines a dynamic acceptable region outside of a guard band limit based on the tracking data. The dynamic acceptable region is a region outside of the guard band that corresponds to data points of a good substrate resulting from acceptable noise, acceptable drift, and the like. The processing device compares additional tracking data (e.g., that may belong to a good or bad substrate) to the acceptable region outside of the guard band limit. In response to one or more data points being outside of the acceptable region, the processing device causes a corrective action to be performed (e.g., discarding the substrate, and the like). In response to one or more data points being within the acceptable region, the processing device causes the acceptable region to be updated. For example, the updated acceptable region may allow for additional drift or noise based on an average of the additional tracking data that has been processed.

[0025] In some embodiments, a processing device (e.g., performing a guardband method) presents the results of a trace data run that is determined to be within acceptable guardband limits, the results of a trace data run that has violations (including shape characteristics of those violations), and / or areas where the processing device (e.g., performing a guardband method) cannot definitively determine that a violation has occurred. The processing device (e.g., performing a guardband method) may allow a user (e.g., a subject matter expert) to verify or overturn the conclusions and positions of the guardband method. The processing device (e.g., performing a guardband method) may use feedback from a user (e.g., a subject matter expert) to update guardband limits, other violation assessments, and / or guardband characteristics.

[0026] Various aspects of the present disclosure provide technical advantages over conventional solutions. The present disclosure has fewer false positives and fewer missed negatives than conventional solutions. This provides less material waste, higher yields, fewer defective products, less user time, less equipment downtime, etc. The present disclosure has fewer repairs for substrate mis-markings than conventional solutions. This reduces processing processor overhead, bandwidth used, energy consumption, metrology operations, user time, etc.

[0027] Some embodiments of the present disclosure are described with respect to substrate processing. In some embodiments, the present disclosure is applicable to other types of manufacturing processes.

[0028] Some embodiments of the present disclosure are described with respect to monitoring sensor data for the purpose of producing substrates that meet specific property values. In some embodiments, the present disclosure can monitor sensor data for other purposes, such as verifying that equipment is operating properly, determining when to perform repair or replacement of equipment or equipment parts (e.g., preventive maintenance), determining adjustments to equipment parameters to make processing more efficient (e.g., in terms of aspects or metrics such as yield or quality, processing or quantity, cost or lifetime), and / or the like.

[0029] Some implementations of the present disclosure are described with respect to causing corrective actions to be performed. In some implementations, causing corrective actions to be performed may include identifying features of the tracking data as degraded, different, and / or similar.

[0030] 1 is a block diagram illustrating an exemplary system 100 (exemplary system architecture) according to certain embodiments. System 100 includes client device 120, manufacturing equipment 124 (e.g., substrate processing equipment), sensor 126, metrology equipment 128, prediction server 112, and data storage device 140. Prediction server 112 may be part of prediction system 110. Prediction system 110 may also include server machines 170 and 180. Prediction system 110 may be used to predict whether an anomaly has occurred, detect whether an anomaly has occurred, etc. (e.g., using guard band technology).

[0031] In some embodiments, the manufacturing equipment 124 (e.g., a cluster tool) is part of a substrate processing system (e.g., an integrated processing system). The manufacturing equipment 124 includes a controller, a housing system (e.g., a substrate carrier, a front opening standard container (FOUP), an automatic teaching FOUP, a process kit closed system, a substrate closed system, a box, etc.), a side storage container (SSP), an aligner device (e.g., an aligner chamber), a factory interface (e.g., an equipment front end module (EFEM)), a load lock, a transfer chamber, one or more processing chambers, a robot (e.g., disposed in the transfer chamber, disposed in the front end interface, etc.), and / or the like. The housing system, the SSP, and the load lock mounted to the factory interface and the robot disposed in the factory interface are used to transfer contents (e.g., substrates, process kit rings, carriers, verification wafers, etc.) between the housing system, the SSP, the load lock, and the factory interface. The aligner device is disposed in the factory interface for aligning the contents. The load lock and process chamber are mounted to the transfer chamber, and a robot disposed in the transfer chamber is used to transfer contents (e.g., substrates, process kit rings, carriers, verification wafers, etc.) between the load lock, process chamber, and transfer chamber. In some embodiments, the fabrication equipment 124 includes components of a substrate processing system. In some embodiments, the fabrication equipment 124 is used to produce one or more products (e.g., substrates, semiconductors, wafers, etc.). In some embodiments, the fabrication equipment 124 is used to produce one or more components to be used in a substrate processing system.

[0032] The sensor 126 may be coupled to the manufacturing equipment 124. The sensor 126 may provide sensor data associated with the manufacturing equipment 124 (e.g., associated with the production of a corresponding product (e.g., substrate) by the manufacturing equipment 124). The sensor data may be stored as a measurement value as a function of time (e.g., tracking data 142). The tracking data 142 may include historical tracking data 144 and current tracking data 146. The tracking data 142 may be used for equipment health and / or product health (e.g., product quality). The manufacturing equipment 124 may produce a product according to a recipe or by performing a run over a period of time. In some embodiments, the tracking data 142 may include values ​​of one or more of temperature (e.g., heater temperature), spacing (SP), pressure, high frequency radio frequency (HFRF), voltage of an electrostatic chuck (ESC), current, flow rate, power, voltage, etc. The tracking data 142 may be associated with or indicate manufacturing parameters such as hardware parameters (e.g., settings or components (e.g., size, type, etc.)) of the manufacturing equipment 124 or processing parameters of the manufacturing equipment. Data associated with some hardware parameters may be stored as manufacturing parameters instead or in addition. The manufacturing parameters may indicate input settings of a manufacturing device (e.g., heater power, airflow, etc.). Tracking data 142 and / or manufacturing parameters (e.g., equipment readings while processing a product) may be provided while the manufacturing equipment 124 performs a manufacturing process. Tracking data 142 may be different for each product (e.g., each substrate).

[0033] The metrology equipment 128 may be used to measure properties of a product such as a substrate (e.g., a processed substrate, a partially processed substrate, etc.). The metrology equipment may be combined with analysis to estimate or better determine metrology values. The metrology data may be included in the performance data 150 along with other performance indicators such as equipment maintenance, yield, etc. The performance data 150 may include historical performance data 152 and current performance data 154. The metrology data and / or performance data 150 may include virtual metrology data, non-virtual metrology data, a mixture of virtual and non-virtual metrology data, etc.

[0034] In some embodiments, the tracking data 142, the performance data 150, and / or the manufacturing parameters can be processed (e.g., by the client device 120 and / or by the prediction server 112). The processing of the tracking data 142 can include generating features. In some embodiments, a feature is a pattern in the tracking data 142 or the performance data 150 (e.g., slope, width, height, peak, etc.) or a combination of values ​​from the tracking data 142 or the performance data 150 (e.g., power from voltage and current, etc.). The tracking data 142 can include features, which can be used by the prediction component 114 and / or the client device 120 to perform signal processing and / or to obtain the prediction data 168 to perform corrective actions. The prediction component 114 can be used to predict whether an anomaly has occurred, detect whether an anomaly has occurred, etc. (e.g., using guard band techniques).

[0035] Each instance (e.g., set) of tracking data 142 may correspond to a product (e.g., a substrate), a set of manufacturing equipment 124, a type of substrate produced by the manufacturing equipment 124, etc. Each instance of performance data 150 or manufacturing parameters may likewise correspond to a product, a set of manufacturing equipment, a type of substrate produced by the manufacturing equipment, etc. The data storage device 140 may further store information associated with different sets of data types, such as indicating a set of tracking data, sensor data, a set of metrology data, and / or a set of manufacturing parameters associated with the same product, manufacturing equipment, substrate type, etc.

[0036] In some embodiments, the prediction system 110 may generate the prediction data 168 using supervised machine learning (e.g., a supervised dataset, the performance data 150 includes metrology data, the tracking data 142 used to train the model 190 is associated with good substrates and bad substrates, etc.). In some embodiments, the prediction system 110 may generate the prediction data 168 using semi-supervised machine learning (e.g., a semi-supervised dataset, the performance data 150 is a prediction percentage, the tracking data 142 used to train the model 190 is associated with only good substrates, etc.). In some embodiments, the prediction system 110 may generate the prediction data 168 using unsupervised machine learning (e.g., an unsupervised dataset, clustering, clustering based on the tracking data 142, etc.). In some embodiments, the prediction system 110 may generate the prediction data 168 using a model that is one or more of a machine learning model, a statistical model, etc.

[0037] Client devices 120 , manufacturing equipment 124 , sensors 126 , metrology equipment 128 , prediction server 112 , data storage device 140 , server machine 170 , and server machine 180 may be coupled to one another via network 130 to generate prediction data 168 to perform corrective actions.

[0038] In some embodiments, the network 130 is a public network that provides access to the prediction server 112, the data storage device 140, and other publicly available computing devices to the client device 120. In some embodiments, the network 130 is a private network that provides access to the manufacturing equipment 124, the sensor 126, the metering equipment 128, the data storage device 140, and other private computing devices to the client device 120. The network 130 may include one or more wide area networks (WANs), local area networks (LANs), wired networks (e.g., Ethernet networks), wireless networks (e.g., 802.11 networks or Wi-Fi networks), cellular networks (e.g., long term evolution (LTE) networks), routers, hubs, switches, server computers, cloud computing networks, and / or combinations thereof.

[0039] Client device 120 may include a computing device such as a personal computer (PC), a laptop, a mobile phone, a smart phone, a tablet computer, a netbook computer, a network-connected television ("smart TV"), a network-connected media player (e.g., a Blu-ray player), a set-top box, an OTT streaming device, an operator box, etc. Client device 120 may include a corrective action component 122. Corrective action component 122 may receive user input (e.g., via a graphical user interface (GUI) displayed by client device 120).

[0040] In some embodiments, the corrective action component 122 obtains tracking data 142 (e.g., current tracking data 146) associated with the manufacturing equipment 124 (e.g., from a data storage device 140, etc.) and provides the tracking data 142 (e.g., current tracking data 146) associated with the manufacturing equipment 124 to the prediction system 110. In some embodiments, the corrective action component 122 stores the tracking data 142 in the data storage device 140, and the prediction server 112 retrieves the tracking data 142 from the data storage device 140. In some embodiments, the prediction server 112 can store the output (e.g., prediction data 168) of the trained machine learning model 190 in the data storage device 140, and the client device 120 can retrieve the output from the data storage device 140. In some embodiments, the corrective action component 122 receives an indication of a corrective action from the prediction system 110 and causes the corrective action to be implemented. The client device 120 may include an operating system that allows a user to perform one or more of generating, viewing, or editing data (eg, instructions associated with the manufacturing equipment 124 , corrective actions associated with the manufacturing equipment 124 , etc.).

[0041] In some embodiments, the historical performance data 152 corresponds to historical attribute data of a product (e.g., generated using manufacturing parameters associated with the historical tracking data 144 and stored manufacturing parameters), and the predictive data 168 is associated with predicted attribute data (e.g., of a product produced (or produced) under the conditions recorded by the current tracking data 146 and / or manufacturing parameters). In some embodiments, the predictive data 168 is predicted metering data (e.g., virtual metering data) of a product to be produced or produced according to the conditions recorded as the current tracking data 146 and / or manufacturing parameters. In some embodiments, the predictive data 168 is an indication of an anomaly (e.g., an abnormal product, an abnormal component, an abnormal manufacturing equipment 124, abnormal energy usage, etc.) and one or more causes of the anomaly. In some embodiments, the predictive data 168 is an indication of changes or drifts over time in some components of the manufacturing equipment 124, sensors 126, metering devices 128, etc. In some embodiments, the predictive data 168 is an indication of the end of life of a component of the manufacturing equipment 124, sensors 126, metering devices 128, etc.

[0042] Performing manufacturing processes that result in defective products can be expensive in terms of time, energy, products, parts, manufacturing equipment 124, identifying defects, and discarding defective products, etc. By generating predictive data 168 based on tracking data 142 and prompting corrective actions to be performed based on predictive data 168, system 100 can have a technical advantage of avoiding the costs of producing, identifying, and discarding defective products.

[0043] Performing a manufacturing process that results in a component of the manufacturing equipment 124 failing can be costly in terms of downtime, damaged product, damaged equipment, quickly ordering replacement components, etc. By generating predictive data 168 based on tracking data 142 (e.g., manufacturing parameters that are being used or will be used to manufacture a product) and prompting corrective actions (e.g., predicted operational maintenance, such as replacing, treating, cleaning components, etc.) based on the predictive data 168, the system 100 can have a technical advantage of avoiding the cost of one or more of unexpected component failure, unplanned downtime, lost productivity, unexpected equipment failure, product scrapping, etc. Monitoring the performance of components (e.g., manufacturing equipment 124, sensors 126, metrology devices 128, etc.) over time can provide an indication of component degradation.

[0044] The manufacturing parameters may not be optimal for producing the product, which may result in costly consequences, such as increased consumption of resources (e.g., energy, coolant, gas, etc.), increased production time for the product, increased component failures, increased number of defective products, etc. By generating prediction data 168 based on features of the tracking data 142 and prompting corrective actions to update the manufacturing parameters (e.g., setting optimal manufacturing parameters) based on the prediction data 168, the system 100 may have the technical advantage of using optimal manufacturing parameters (e.g., hardware parameters, process parameters, optimized design) to avoid the costly consequences of poor manufacturing parameters.

[0045] The corrective action may be associated with one or more of computational process control (CPC), statistical process control (SPC) (e.g., SPC on electronic components used to determine a control process, SPC used to predict the useful life of a component, SPC compared to a 3-sigma chart, etc.), advanced process control (APC), model-based process control, preventive operational maintenance, design optimization, updating of manufacturing parameters or updating of manufacturing recipes for current or future manufacturing processes, feedback control, machine learning modifications, and the like.

[0046] In some embodiments, the corrective action includes providing an alert (e.g., an alert to stop or not perform a manufacturing process if the predicted data 168 indicates a predicted anomaly, such as an anomaly of a product, component, or manufacturing equipment 124). In some embodiments, the corrective action includes scheduling preventive maintenance. In some embodiments, the corrective action includes scheduling corrective maintenance. In some embodiments, the corrective action includes updating a process recipe to produce a subsequent substrate. In some embodiments, the corrective action may be determined based on an ongoing substrate processing process and may include updating a current process. In some embodiments, the corrective action includes correcting for chamber drift associated with a manufacturing equipment 124 (e.g., a substrate processing equipment). In some embodiments, the corrective action includes correcting for sensor drift of a sensor associated with a manufacturing equipment 124 (e.g., a substrate processing equipment). In some embodiments, the corrective action includes providing feedback control (e.g., modifying a manufacturing parameter in response to the predicted data 168 indicating a predicted anomaly). In some embodiments, the corrective action includes providing machine learning (e.g., modifying one or more manufacturing parameters based on the predicted data 168). In some embodiments, the performance of the corrective action includes causing an update to one or more manufacturing parameters. In some embodiments, one or more corrective actions will be performed in association with components of the substrate processing equipment.

[0047] Manufacturing parameters may include hardware parameters (e.g., replacing parts, using certain parts, replacing process wafers, updating firmware, etc.) and / or process parameters (e.g., temperature, pressure, flow, rate, current, voltage, gas flow, lifting speed, etc.). In some embodiments, the corrective action includes prompting preventive operational maintenance (e.g., replacement, treatment, cleaning, etc. of parts of the manufacturing equipment 124). In some embodiments, the corrective action includes prompting design optimization (e.g., updating manufacturing parameters, manufacturing processes, manufacturing equipment 124 for optimized products, etc.). In some embodiments, the corrective action includes updating a recipe (e.g., the manufacturing equipment 124 is in idle mode, sleep mode, preheating mode, etc.).

[0048] Prediction server 112, server machine 170, and server machine 180 may each include one or more computing devices, such as a rack server, a router computer, a server computer, a personal computer, a mainframe computer, a laptop computer, a tablet computer, a desktop computer, a graphics processing unit (GPU), an accelerator application specific integrated circuit (ASIC) (e.g., a tensor processing unit (TPU)), etc.

[0049] The prediction server 112 may include a prediction component 114. In some embodiments, the prediction component 114 may take current tracking data 146 (e.g., received from the client device 120, retrieved from the data storage device 140) and generate an output (e.g., prediction data 168) for performing a corrective action associated with the manufacturing equipment 124 based on the current data. In some embodiments, the prediction component 114 may use one or more training models 190 to determine an output for performing a corrective action based on the current data. In some embodiments, the prediction component 114 predicts that an anomaly has occurred or is about to occur. In some embodiments, the prediction component 114 predicts that another event has occurred or is likely to occur.

[0050] Model 190 may be a single model or may be a plurality of models. Models may be applied sequentially, multiple models may be used simultaneously, appropriate models may be selected based on some metrics, a combination of these practices may be used, and the like. Model 190 (or the models included in model 190) may be machine learning models, including supervised, unsupervised, or semi-supervised machine learning models. Model 190 may not be a machine learning model, such as a statistical model, a correlation model, and the like.

[0051] In some embodiments, a first model 190 is used to generate a guard band (e.g., see FIGS. 2A-2D ), a second model 190 is used for guard band violation analysis (e.g., see FIGS. 3A-3D ), and a third model 190 is used to generate a dynamic region outside the guard band (e.g., see FIGS. 4A-4D ).

[0052] In some embodiments, the data input to the model 190 may include tracking data 142 from a single sensor 126. In other embodiments, the data input to the model 190 may include tracking data 142 indicating values ​​of different properties from many sensors 126. The data input may include manufacturing parameters. Features extracted from the tracking data 142, methods of extracting features, corrective actions and / or predictive data 168 associated with the features, and methods of associating corrective actions and / or predictive data 168 may all be adjusted for the data provided as input.

[0053] In some embodiments, the prediction component 114 receives the current tracking data 146 and provides the current tracking data 146 as an input to the model 190, and obtains an output from the model 190 indicating the prediction data 168. In some embodiments, the prediction data 168 indicates the performance data 150 (e.g., metrology data, yield, etc.). In some embodiments, the prediction data 168 indicates a corrective action.

[0054] In some embodiments, model 190 takes tracking data 142 (e.g., and data indicating a recipe, manufacturing equipment component, etc. associated with tracking data 142) as input and produces prediction data 168 as output. Model 190 can be a single model or can include many models. Model 190 can determine which processing to perform based on the input data, or a user can indicate which analysis is appropriate for the input data, or a combination of these.

[0055] The data storage device 140 and / or may be a memory (e.g., random access memory), a drive (e.g., a hard drive, a flash drive), a database system, or another type of component or device capable of storing data. The data storage device 140 and / or may include multiple storage components (e.g., multiple drives or multiple databases) that may span multiple computing devices (e.g., multiple server computers). The data storage device 140 may store tracking data 142, performance data 150, and prediction data 168. The tracking data 142 may include historical tracking data 144 and current tracking data 146. The tracking data may include sensor data time tracking over the duration of a manufacturing process, association of data with physical sensors, pre-processed data (e.g., averages and composite data), and data indicating changes in sensor performance over time (e.g., many manufacturing processes). The manufacturing parameters and performance data 150 may contain similar features. The historical tracking data 144, the manufacturing parameters, and the historical performance data 152 may be historical data (e.g., at least a portion of which is used to train the model 190). Current tracking data 146 may be current data (eg, at least a portion to be input into model 190 , following historical data) for which prediction data 168 is to be generated (eg, for performing corrective action).

[0056] In some embodiments, the prediction system 110 also includes a server machine 170 and a server machine 180. The server machine 170 includes a data set generator 172, which is capable of generating a data set (e.g., a set of data inputs and a set of target outputs) to train, validate and / or test one or more models, such as machine learning models. The model 190 may include one or more machine learning models, or may be another type of model, such as a statistical model. Models that incorporate machine learning can be trained using input data and, in some cases, target output data. Models that do not incorporate machine learning can also be trained. In some embodiments, the data set generator 172 may divide historical data (e.g., historical tracking data 144, manufacturing parameters, or historical performance data 152 stored in the data storage device 140) into a training set (e.g., sixty percent of the historical data), a validation set (e.g., twenty percent of the historical data), and a test set (e.g., twenty percent of the historical data). In some embodiments, the prediction system 110 (e.g., via the prediction component 114) generates multiple sets of elements. For example, a first set of elements may correspond to a first set of sensor data of a type (e.g., from a first set of sensors, a first combination of values ​​from the first set of sensors, a first pattern in values ​​from the first set of sensors) corresponding to each of a data set (e.g., a training set, a validation set, and a test set), and a second set of elements may correspond to a second set of sensor data of a type (e.g., from a second set of sensors different from the first set of sensors, a second combination of values ​​different from the first combination, a second pattern different from the first pattern) corresponding to each of the data sets.

[0057] The server machine 180 includes a training engine 182, a validation engine 184, a selection engine 185, and / or a test engine 186. The engines (e.g., the training engine 182, the validation engine 184, the selection engine 185, and the test engine 186) may represent hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, processing devices, etc.), software (such as instructions running on a processing device, a general purpose computer system, or a dedicated machine), firmware, microcode, or a combination thereof. The training engine 182 may be capable of training the machine learning model 190, or various machine learning models included in the model 190, using one or more sets of elements associated with the training set from the data set generator 172. The training engine 182 may generate multiple trained machine learning models 190, where each trained machine learning model 190 corresponds to a different set of elements of the training set (e.g., sensor data from different groups and sensors). For example, a first trained machine learning model may have been trained using all elements (e.g., X1-X5), a second trained machine learning model may have been trained using a first subset of elements (e.g., X1, X2, X4), and a third trained machine learning model may have been trained using a second subset of elements (e.g., X1, X3, X4, and X5) that may partially overlap with the first subset of elements. The dataset generator 172 may receive the output of a trained machine learning model (e.g., a model trained to perform a first operation of tracking data processing), collect the data into training, validation, and test datasets, and use the datasets to train a second machine learning model (e.g., a model to be trained to perform a second operation of tracking data processing).

[0058] The validation engine 184 may be capable of validating the trained machine learning models 190 using a corresponding set of elements of the validation set from the dataset generator 172. For example, a first trained machine learning model 190 trained using a first set of elements of the training set may be validated using a first set of elements of the validation set. The validation engine 184 may determine the accuracy of each trained machine learning model 190 based on the corresponding set of elements of the validation set. The validation engine 184 may discard trained machine learning models 190 whose accuracy does not meet a threshold accuracy. In some embodiments, the selection engine 185 may be capable of selecting one or more trained machine learning models 190 whose accuracy meets a threshold accuracy. In some embodiments, the selection engine 185 may be capable of selecting a trained machine learning model 190 with the highest accuracy among the trained machine learning models 190. In some embodiments, the validation engine 184 and the selection engine 185 may repeat this process for each machine learning model included in the model 190.

[0059] In some embodiments, the verification engine 184 performs verification and / or validation (e.g., validation and verification (V&V)). Verification and verification can be independent procedures that are used together to determine whether a product, service, or system (e.g., a machine learning model) meets requirements and specifications and achieves its intended purpose. Verification can include ensuring that the machine learning model meets the needs of customers or other identified stakeholders (e.g., acceptance and applicability involving external customers). Validation can include evaluating whether the machine learning model meets specifications, requirements, specifications, or mandatory conditions (e.g., internal processes).

[0060] The testing engine 186 may be able to test the trained machine learning models 190 using corresponding sets of elements of the test set from the dataset generator 172. For example, a first trained machine learning model 190 trained using a first set of elements of the training set may be tested using a first set of elements of the test set. The testing engine 186 may determine, based on the test set, the trained machine learning model that has the highest accuracy among all the trained machine learning models included in the model 190. The testing engine 186 may repeat this process for each machine learning model included in the model 190.

[0061] Model 190 may represent a model artifact created by training engine 182 using a training set that includes data inputs and corresponding target outputs (correct answers for each training input). Patterns in the data set that map data inputs to target outputs (correct answers) may be found, and a mapping to obtain these patterns is provided for the machine learning model. The machine learning model may use one or more of support vector machines (SVMs), radial basis functions (RBFs), clustering, supervised machine learning, semi-supervised machine learning, unsupervised machine learning, k-nearest neighbor algorithms (k-NNs), linear regression, random forests, neural networks (such as artificial neural networks), and the like.

[0062] The prediction component 114 may provide the current tracking data 146 to the model 190, and may run the trained machine learning model 190 on the input to obtain one or more outputs. The prediction component 114 may be capable of determining (e.g., extracting) prediction data 168 from the output of the model 190, and may determine (e.g., extracting) confidence data from the output indicating a confidence level that the prediction data 168 is an accurate prediction of a process associated with the input data of a product produced or to be produced using the manufacturing equipment 124 under the current tracking data 146 and / or manufacturing parameters. The prediction component 114 may also be capable of determining a confidence range associated with the predicted event, such as a remaining useful life (RUL) window including an upper limit and a lower limit. The prediction component 114 or the corrective action component 122 may use the confidence data to determine whether to cause a corrective action and / or when to cause a corrective action associated with the manufacturing equipment 124 based on the prediction data 168.

[0063] The confidence data may include or indicate a confidence level that the prediction data 168 is an accurate prediction of a product associated with at least a portion of the input data. In one example, the confidence level is a real number between 0 and 1, where 0 indicates that the prediction data 168 is not believed to be an accurate prediction of a product processed according to the input data, and 1 indicates absolute confidence that the prediction data 168 accurately predicts the properties of the product processed according to the input data. In response to the confidence data indicating that the confidence level of a predetermined number of instances (e.g., a percentage of instances, a frequency of instances, a frequency of occurrence, a total number of instances, etc.) is below a threshold level, the prediction component 114 may cause the model 190 to be retrained (e.g., based on current tracking data 146, manufacturing parameters, current performance data 154, etc.).

[0064] For purposes of illustration and not limitation, aspects of the present disclosure describe using historical data (e.g., historical tracking data 144, historical performance data 152) to train one or more machine learning models 190, and inputting current data (e.g., current tracking data 146) into one or more trained machine learning models 190 to determine prediction data 168. In other embodiments, a heuristic model or a rule-based model is used to determine prediction data 168 (e.g., without using a trained machine learning model). The prediction component 114 can monitor the historical tracking data 144 and the historical performance data 152.

[0065] In some implementations, the functionality of client device 120, prediction server 130, server machine 170, and server machine 180 may be provided by a smaller number of machines. For example, in some implementations, server machines 170 and 180 may be integrated into a single machine, while in other implementations, server machine 170, server machine 180, and prediction server 112 may be integrated into a single machine. In some implementations, client device 120 and prediction server 112 may be integrated into a single machine.

[0066] In general, functions described in one embodiment as being performed by client device 120, prediction server 130, server machine 170, and server machine 180 may also be performed on prediction server 112 in other embodiments, if appropriate. In addition, the functions attributed to a particular component may be performed by different or multiple components operating together. For example, in some embodiments, prediction server 112 may determine a corrective action based on prediction data 168. In another example, client device 120 may determine prediction data 168 based on output from a trained machine learning model.

[0067] Additionally, the functionality of a particular component may be performed by different or multiple components operating together.One or more of the prediction server 112, server machine 170, or server machine 180 may be accessed as a service provided to other systems or devices through an appropriate application programming interface (API).

[0068] In an embodiment, a "user" may be represented as a single individual. However, other embodiments of the present disclosure include a "user" that is an entity controlled by multiple users and / or automated sources. For example, a group of individual users united as a group of administrators may be considered a "user".

[0069] Embodiments of the present disclosure may be applied to data quality assessment, feature enhancement, model assessment, virtual metering (VM), predictive maintenance (PdM), extreme optimization, anomaly or fault detection, anomaly or fault classification, etc.

[0070] 2A-2D, 3A-3D, and 4A-4D are flow diagrams of methods 200A-200D, 300A-300D, and 400A-400D associated with guard bands according to certain embodiments. In some embodiments, methods 200A-200D, 300A-300D, and 400A-400D are performed by processing logic that includes hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, processing devices, etc.), software (such as running on a processing device, a general-purpose computer system, or a dedicated machine), firmware, microcode, or a combination thereof. In some embodiments, methods 200A-200D, 300A-300D, and 400A-400D are performed at least in part by prediction system 110. In some embodiments, methods 200A-200D, 300A-300D, and 400A-400D are performed at least in part by prediction system 110 (e.g., prediction server 112, prediction component 114), client device 120 (e.g., corrective action component), manufacturing equipment 124, and / or metrology equipment 128. In some embodiments, a non-transitory storage medium stores instructions that, when executed by a processing device (e.g., a processing device of prediction system 110, a processing device of server machine 180, a processing device of prediction server 112, etc.), cause the processing device to perform one or more of methods 200A-200D, 300A-300D, and 400A-400D.

[0071] For simplicity of explanation, methods 200A-200D, 300A-300D and 400A-400D are described and illustrated as a series of operations. However, operations according to the present disclosure may occur in various orders and / or simultaneously and with other operations not presented and described herein. In addition, in some embodiments, not all illustrated operations are performed to implement methods 200A-200D, 300A-300D and 400A-400D according to the disclosed subject matter. In addition, those skilled in the art will understand and appreciate that methods 200A-200D, 300A-300D and 400A-400D may alternatively be represented as a series of interrelated states by state diagrams or events.

[0072] In some embodiments, FIGS. 2A-2D and 3A-3D relate to sequential data guardband analysis (e.g., sequential guardband analysis for fault detection and classification (FDC)) for improved fault diagnosis, fault classification, and prediction. In some embodiments, the present disclosure provides improved analysis of sequential data streams (e.g., trace data) using guardband fault diagnosis full trace analysis (FTA) to provide improved detection, classification, and prediction, including reducing false alarms (e.g., false positives) and missed alarms (e.g., missed negatives). Traditional sensor trace analysis may have false positives and missed negatives (e.g., inability to fully capture normal changes, inability to adjust models over time, inability to handle recipe endpoint determination, inability to resolve out-of-phase tracking, inability to transfer models to different fields, etc.), resulting in low adoption of traditional guardbands. Traditional summary statistics may miss portions of trace data (e.g., transient regions). The present disclosure addresses false positives and missed negatives, and adaptively provides robustness and flexibility over time to address different fields (e.g., see FIGS. 8A-8B). The present disclosure can use guard bands to analyze both transient and steady-state regions and extract complex features (eg, more complex than unique variable analysis (UVA)).

[0073] The guard bands of the present disclosure can be used to detect anomalies or faults that appear in trace data but may not conform to typical anomaly shapes (e.g., spikes or oscillations). Conventional guard bands may have output quality issues based on false positives (e.g., in transients), amplification of transition boundaries, trace-to-trace variability that is considered abnormal, all violations are treated equally and are not analyzed or quantified, etc. The guard bands of the present disclosure can address these shortcomings. The guard bands of the present disclosure can be provided alone or can be combined with other analysis capabilities such as semi-automatic feature extraction (SFE).

[0074] A guard band can be a channel defined for a data stream that is intended to identify areas of commonality (e.g., acceptable values) based on location in the data stream. A data stream is typically the values ​​of a sensor during some event (e.g., a production run). A data stream is typically trace data, which is a sequence of data values ​​presented and positioned as a function of time (e.g., the x-axis is time). The sorting may not be based on time (e.g., it may be an indication of a count of something, such as the number of products produced or the number of errors recorded). The term "acceptable" is a function of the application environment and the goal of the guard band analysis. For example, "acceptable" can be "no anomalies" or "no faults." A guard band channel can have upper and lower limits. In some embodiments, these limits can be calculated using statistical methods along the duration of the data stream. For example, a channel can represent the plus or minus 3-sigma of multiple trace values ​​of a particular sensor, where the change is calculated at each time value of the data stream. Smoothing techniques (e.g., time window averaging) can be employed to allow the channel to be more noise-resistant and smoother.

[0075] FTA guard bands can be used for anomalous traces that have slight oscillations in transient segments. FTA guard bands can be used to analyze multiple traces or trace partitions from a specific sensor over multiple runs. FTA guard bands can establish upper and lower limits that indicate the normal range or channel (e.g., 3-sigma) of sensor data over time. The solution identifies and analyzes excursions from the guard bands. The FTA functionality can be used in a complementary manner with SFE to provide comprehensive analysis and serve as input to a fingerprint library. Guard bands can be associated with a single sensor (UVA) or with multiple sensors (multivariate analysis (MVA)). For MVA, the sensor value (e.g., y-axis) can be a metric representing some combination of the sensors involved in the MVA.

[0076] In some embodiments, the guard band may include multiple guard bands (e.g., a warning guard band completely contained within an error guard band). In some embodiments, the first region is a normal region independent of other parameters, the second region is a region classified as good or bad based on analysis of other parameters, and the third region is a fault region independent of other parameters.

[0077] In some embodiments, the x-axis parameter of the guard band or the parameter defining the ordering aspect of the data is time (eg, closely related to the sensor reading rate). In some embodiments, the substrate number in the processing tool can correspond to the substrate process.

[0078] In some embodiments, the guard band violation definition can be an anomaly, a fault, a warning, an event trigger, and / or a prediction. The guard band violation determination and interpretation may be related to the purpose of the guard band (e.g., fault or anomaly detection). A single data point located outside the guard band channel may indicate a fault. A probability distribution function may be used to define the area, duration, and magnitude of the portion of the trace that includes the violation region. These violation regions (e.g., violation region attribute values) may be evaluated to interpret whether a fault exists.

[0079] Figures 2A-2D are flow diagrams of methods 200A-200D associated with generating guard bands according to certain embodiments. Figure 2A is a flow diagram of method 200A associated with generating guard bands, Figure 2B is a flow diagram of method 200B associated with using guard bands, Figure 2C is a flow diagram of method 200C associated with generating guard bands via machine learning, and Figure 2D is a flow diagram of method 200D associated with using guard bands via machine learning.

[0080] 2A, in some embodiments, at block 202, processing logic. Tracking data includes a set of individual sensor data for each substrate over time. In some embodiments, the tracking data includes sensor data from multiple different sensors (eg, different types of sensor data) for each substrate.

[0081] At block 204 , processing logic determines allowed types of changes (eg, of guard bands) based on the trace data.

[0082] In some embodiments, processing logic generates a guard band based on the trace data. The guard band may include upper and lower limits for fault detection (e.g., anomaly detection). In some embodiments, to form the guard band, an average of all trace data is generated, and then an offset (e.g., 3 sigma) of the average is used as the upper and lower limits. Typically, the upper and lower limits are equidistant from the average of the trace data.

[0083] Since the tracking data is for a good substrate, any variation in the guard band formed by the tracking data is an allowed type of variation.

[0084] In some embodiments, block 204 includes processing logic that determines trace-to-trace variability of trace data at block 210 (see Figures 5A-5B). In some embodiments, the allowed types of variation include trace-to-trace variability of time offset. The trace data may include sensor values ​​(e.g., y-axis) over time (e.g., x-axis). Due to different times when sensor data is recorded, some sensor data sets may shift over time (e.g., time offset on the x-axis). In some embodiments, processing logic tracks the differences between traces and autonomously adjusts the guard band changes horizontally when normal time offset changes are found. This prevents false positives when recipe steps or traces change slightly. This increases robustness and reduces false positives.

[0085] In some embodiments, block 204 includes processing logic at block 212 that determines an upper limit different from a lower limit based on the trace data (see FIG. 5C ). In some embodiments, the allowed types of changes include a first allowed type of change that forms an upper limit of the guard band and a second allowed type of change that forms a lower limit of the guard band. The first allowed type of change (e.g., amount) may be different from the second allowed type of change (e.g., amount). Changes on each side of the guard band will be calculated separately (e.g., during signal transitions) and reduce false positives.

[0086] In some embodiments, block 204 includes processing logic at block 214 that determines location-dependent changes in the trace data (see FIG. 5D ). Parameters and weights (e.g., duration, level, area, cascades of consecutive violations, etc.) and other guard band settings can be adjusted based on the location in the guard band (e.g., along the x-axis). Examples include: 1) adjusting along the x-axis corresponding to different operations in the process to make the guard band more (or less) sensitive or adaptive; 2) adjusting the guard band based on properties of the signal (e.g., less aggressive in areas where the guard band changes rapidly in the y-direction, and more aggressive in areas where the guard band value is relatively constant); 3) adjusting the guard band based on alarms detected along the guard band (e.g., adjusting the distribution parameters after the first spike is detected to be more (or less) sensitive to future spike features on this trace); and so on. This improves guard band performance and further allows subject matter expertise to be incorporated into guard band analysis.

[0087] In some embodiments, block 204 includes processing logic for determining a trace segmentation of the trace data at block 216 (see FIG. 5E ). In some embodiments, the allowed type of change is further based on the segmentation of the portion of the trace data (e.g., associated with a change in value exceeding a threshold change, a value within a threshold). For example, a feature (e.g., within a threshold) may trigger a wider guard band for the next segment of the guard band. In some embodiments, the allowed type of change is further based on a feature extracted from the trace data.

[0088] In some embodiments, block 204 includes processing logic that performs feature extraction (e.g., semi-automatic feature extraction (SFE)) of the trace data at block 218. For example, guard band parameters may be varied for different segments or for different features. This improves guard band performance and overall analysis by allowing different capabilities (e.g., guard bands, feature extraction, and / or trace segmentation) to be combined.

[0089] In some embodiments, the trace data includes sensor data from different types of sensors and the allowed types of changes are through MVA. The MVA metric combines the values ​​of multiple sensors into a single metric (e.g., the first principal component in a principal component analysis (PCA)). MVA techniques can be used to apply guard band parameters to multiple guard bands. Examples include: 1) determining violations and violation distributions as a function of violation characteristics of two or more traces that occur at the same time (or not at the same time); and / or 2) dynamics associated with a guard band for a signal based on one or more other signals. This improves guard band performance, further allows subject matter expertise to be incorporated into guard band analysis, and allows correlations between multiple signals to be resolved.

[0090] In some embodiments, these features include one or more of the following: continuous features (e.g., ramps and flats); user-defined features (e.g., cascades of ramps and slopes); intermittent features (such as ramps, flats, and FTA features); x-offset; y-offset; shape; length; warp, and the like.

[0091] At block 205, processing logic generates guard bands based on the allowed types of changes. For example, guard bands can allow for time offsets (e.g., on the x-axis), different upper and lower limits, multivariate (e.g., multiple variables) analysis, segmentation of trace data portions, extracting features of trace data, etc. In some embodiments, guard bands are generated by training a machine learning model, as shown in FIG2C.

[0092] At block 208, processing logic causes corrective action associated with the substrate processing system to be performed based on the guard band.Block 208 may include comparing the additional tracking data to the guard band to determine whether to perform corrective action (eg, see FIG. 2B).

[0093] Referring to FIG. 2B , in some embodiments, at block 222, processing logic identifies tracking data associated with production of substrates via a substrate processing system (e.g., the same substrate processing system as FIG. 2A ). The tracking data may be associated with substrates for which it is unknown whether attribute data of the substrates meets a threshold (e.g., a good substrate) or does not meet a threshold (e.g., a bad substrate). The tracking data includes a set of individual sensor data for each substrate that varies over time. In some embodiments, the tracking data includes sensor data from a plurality of different sensors (e.g., different types of sensor data) for each substrate.

[0094] At block 224, processing logic compares the trace data to guard bands that have been generated based on the allowed types of changes (eg, see block 206 of FIG. 2A). The guard bands may have upper and lower limits for data points over time.

[0095] At block 226, processing logic determines that one or more data points of the tracking data are not within the guard band. The one or more data points may include at least one data point above an upper limit and / or at least one data point below a lower limit (e.g., the data point does not match a data point of a substrate having a property value that satisfies a threshold).

[0096] At block 228, processing logic causes execution of a corrective action associated with the substrate processing system. In some embodiments, the corrective action includes providing an alarm, interrupting substrate processing equipment, initiating a substrate check, discarding a substrate, updating manufacturing parameters, etc. In some embodiments, execution of the corrective action is specific to the type or number of data points outside of the guard band.

[0097] Referring to FIG2C , in some embodiments, at block 242, processing logic identifies historical tracking data associated with production of substrates via a substrate processing system. In some embodiments, a substrate (e.g., a good substrate) has a property value that satisfies a threshold. Block 242 can be similar to block 202 of FIG2A . In some embodiments, a substrate (e.g., a bad substrate) has a property value that does not satisfy a threshold.

[0098] In some embodiments, processing logic identifies historical performance data associated with the historical tracking data at block 244. In some examples, the historical performance data indicates whether the substrate is a good substrate or a bad substrate (e.g., all substrates have a property value that meets a threshold, and all substrates have a property value that does not meet the threshold).

[0099] At block 246, processing logic trains a machine learning model using data inputs including historical tracking data (e.g., and target outputs including historical performance data) to generate a trained machine learning model indicating guard bands associated with the varying allowed types. Training of the machine learning model may enable the guard bands to be based on the varying allowed types described in block 204 of FIG. 2A.

[0100] The trained machine learning model can be used to determine whether additional tracking data satisfies the guard band (eg, see FIG. 2D ).

[0101] Referring to FIG2D , in some embodiments, at block 262, processing logic identifies tracking data associated with production of substrates via a substrate processing system (e.g., the same substrate processing system as FIG2C ). The tracking data may be associated with substrates for which it is not known whether attribute data of the substrate meets a threshold (e.g., a good substrate) or does not meet a threshold (e.g., a bad substrate). Block 262 may be similar to block 242 of FIG2B .

[0102] At block 264 , processing logic provides the trace data as input to a trained machine learning model associated with generating guard bands based on the allowed types of changes (eg, the trained machine learning model of 246 of FIG. 2C ).

[0103] At block 266 , processing logic receives output from the trained machine learning model indicating prediction data.

[0104] At block 268, processing logic determines that one or more data points of the tracking data are not within a guard band of the trained machine learning model based on the prediction data. The one or more data points may be above an upper limit of the guard band or may be below a lower limit of the guard band.

[0105] At block 270, processing logic causes corrective actions associated with the substrate processing system to be performed based on the predicted data. Block 270 may be similar to block 228 of Figure 2B.

[0106] 3A-3D are flow diagrams of methods associated with guard band violation analysis according to certain embodiments. FIG3A is a flow diagram of a method 300A associated with determining a guard band violation shape representation for classifying a guard band violation data point, FIG3B is a flow diagram of a method 300B associated with classifying a guard band violation data point based on the guard band violation shape representation, FIG3C is a flow diagram of a method 300C associated with training a machine learning model for classifying guard band violation data points, and FIG3D is a flow diagram of a method 300D associated with classifying a guard band violation data point using the trained machine learning model.

[0107] 3A, in some embodiments, processing logic identifies tracking data associated with substrates (eg, good substrates) produced by a substrate processing system having a property value that satisfies a threshold at block 302. Block 302 may be similar to block 202 of FIG. 2A.

[0108] At block 304, processing logic identifies a guard band associated with the trace data. The guard band may be generated based on the method 200A of FIG. 2A or the method 200C of FIG. 2C.

[0109] At block 306, processing logic determines a guard band violation data point of the tracking data based on the guard band. The guard band violation data point includes a data point of the tracking data that is above an upper limit of the guard band and / or a data point of the tracking data that is below a lower limit of the guard band.

[0110] In some implementations, the tracking data comes from different types of sensors and guard band violation data points are determined via multivariate (eg, multivariate) analysis.

[0111] In some implementations, the determination of the guard band violation data point includes segmenting a portion of the tracking data associated with a change in value that exceeds a threshold change. In some implementations, the determination of the guard band violation data point includes extracting features from the tracking data.

[0112] At block 308 , processing logic determines a guard band violation shape characterization based on the guard band violation data points (see FIGS. 6A-6E ).

[0113] In some embodiments, the guard band violation shape characterization is a weighted combination of one or more of guard band violation duration (e.g., the number of consecutive guard band violation data points that are outside the guard band limits), guard band violation magnitude (e.g., how far above an upper limit or below a lower limit a guard band violation data point is), guard band violation area (e.g., the area between a line passing through the guard band violation data points and the guard band limits), guard band violation location (e.g., the location of the guard band violation data points relative to the guard band limits), and / or guard band violation intermittency (e.g., how often a guard band violation data point exceeds an upper guard band limit and / or a lower guard band limit).

[0114] In some embodiments, the determination of the guard band violation shape characterization includes cascading consecutive violations into a single violation. By analyzing the behavior between consecutive violations, it is possible to better understand whether the violations are related. Typical analysis may include the time between violations, the degree of recovery between violations, and the similarity of factors that lead to consecutive violations (e.g., x-direction offset). The cascading of consecutive violations improves the performance of the guard band by identifying system problems rather than guard band violations. This reduces the noise (chatter) in the guard band violation report and allows subject matter expertise to be incorporated into the guard band analysis.

[0115] Guardband violation shape characterization can be used to characterize any guardband feature based on parameters related to duration, level, area, etc. These parameters can be weighted to better capture specific violation types.

[0116] At block 310, processing logic causes execution of a corrective action associated with the substrate processing system based on the guard band violation shape characterization. Block 310 may include classifying additional guard band violation data points of the additional tracking data based on the guard band violation shape characterization to determine whether to execute a corrective action (eg, see FIG. 3B).

[0117] Referring to FIG3B , in some embodiments, at block 322, processing logic identifies tracking data associated with production of substrates via a substrate processing system (e.g., the same substrate processing system as FIG3A ). The tracking data may be associated with a substrate for which it is not known whether the attribute data of the substrate meets a threshold (e.g., a good substrate) or does not meet the threshold (e.g., a bad substrate). Block 322 may be similar to block 222 of FIG2B .

[0118] At block 324, processing logic identifies a guard band associated with the trace data. Block 324 may be similar to block 304 of FIG. 3A.

[0119] At block 326, processing logic determines a guard band violation data point of the tracking data based on the guard band. The guard band violation data point includes a data point of the tracking data that is above the upper limit of the guard band and / or a data point of the tracking data that is below the lower limit of the guard band. Block 326 can be similar to block 306 of FIG. 3A.

[0120] At block 328, processing logic identifies the guard band violation shape characterization. The guard band violation shape characterization may be determined by block 308 of FIG3A. The guard band violation shape characterization may indicate a particular type (e.g., shape, area, duration, magnitude, location, intermittent, etc.) of guard band violations to be classified as anomalies (e.g., perform corrective action) or not anomalies (e.g., do not perform corrective action).

[0121] At block 330, processing logic determines a classification of the guard band violation data point based on the guard band violation shape characterization. In some embodiments, the classification indicates whether the guard band violation data point is an anomaly. In some embodiments, the classification indicates a type of anomaly associated with the guard band violation data point. In some embodiments, the classification indicates a type of corrective action to be performed in association with the guard band violation data point.

[0122] At block 332, processing logic causes execution of a corrective action associated with the substrate processing system based on the classification. Execution of the corrective action at block 332 may be similar to execution of the corrective action at block 228 of Figure 2B.

[0123] 3C , in some embodiments, at block 342 , processing logic identifies tracking data associated with substrates (e.g., good substrates) produced via the substrate processing system having a property value that satisfies a threshold. Block 302 may be similar to block 202 of FIG. 2A , block 242 of FIG. 2C , and / or block 302 of FIG. 3A .

[0124] At block 344, processing logic identifies historical performance data associated with the historical tracking data. In some examples, the historical performance data indicates whether the substrate is a good substrate or a bad substrate (e.g., all substrates have a property value that meets a threshold, and all substrates have a property value that does not meet the threshold). Block 344 can be similar to block 244 of FIG. 2C.

[0125] At block 346, processing logic identifies a guard band associated with the trace data. The guard band may be generated based on method 200A of FIG. 2A or method 200C of FIG. 2C. Block 346 may be similar to block 304 of FIG. 3B.

[0126] At block 348, processing logic determines historical guard band violation data points of the historical tracking data based on the guard band. The guard band violation data points include data points of the historical tracking data that are above the upper limit of the guard band and / or data points of the tracking data that are below the lower limit of the guard band. Block 348 can be similar to block 306 of FIG. 3A.

[0127] At block 350, processing logic trains a machine learning model using data inputs including historical guard band violation data points (e.g., and target outputs including historical performance data) to generate a trained machine learning model associated with a guard band violation shape representation to classify additional guard band violation data points. The trained machine learning model may be used by FIG. 3D. In some embodiments, the guard band violation data points and the violation shape representation (e.g., shape summary statistics) are provided to the trained machine learning model to classify the guard band violation data points. In some embodiments, the guard band violation shape representation is determined based on the guard band violation data points (e.g., see block 308 of FIG. 3B).

[0128] Referring to FIG3D , in some embodiments, at block 362, processing logic identifies tracking data associated with substrate production via a substrate processing system (e.g., the same substrate processing system as FIG3C ). The tracking data may be associated with a substrate for which it is not known whether the attribute data of the substrate meets a threshold (e.g., a good substrate) or does not meet the threshold (e.g., a bad substrate). Block 362 may be similar to block 342 of FIG3B .

[0129] At block 364, processing logic identifies a guard band associated with the trace data. Block 364 may be similar to block 304 of Figure 3A, block 324 of Figure 3B, and / or block 344 of Figure 3C.

[0130] At block 366, processing logic determines a guard band violation data point of the tracking data based on the guard band. The guard band violation data point includes a data point of the tracking data that is above the upper limit of the guard band and / or a data point of the tracking data that is below the lower limit of the guard band. Block 366 may be similar to block 306 of FIG. 3A, block 326 of FIG. 3B, and / or block 346 of FIG. 3C.

[0131] At block 368, processing logic provides the guard band violation data points as input to a trained machine learning model (e.g., trained via block 350 of FIG. 3C ) associated with the guard band violation shape representation. In some embodiments, the guard band violation data points and the guard band violation shape representation (e.g., shape summary statistics) are provided as input to the trained machine learning model to classify the guard band violation data points. In some embodiments, the guard band violation shape representation is determined based on the guard band violation data points (e.g., see block 308 of FIG. 3B ).

[0132] At block 370 , processing logic receives output from the trained machine learning model indicating prediction data.

[0133] At block 372, processing logic determines a classification for one or more data points of the tracking data based on the prediction data. The classification at block 372 may be similar to the classification at block 330 of FIG3B.

[0134] At block 384, processing logic causes execution of a corrective action associated with the substrate processing system based on the classification. Block 384 may be similar to block 332 of Figure 3B.

[0135] 4A-4D are flow diagrams of methods associated with dynamic acceptable regions outside guard band limits (e.g., tracking time-dependent changes in data) in accordance with certain embodiments. FIG4A is a flow diagram of method 400A associated with determining acceptable regions outside guard band limits, FIG4B is a flow diagram of method 400B associated with using and possibly adjusting acceptable regions outside guard band limits, FIG4C is a flow diagram of method 400C associated with training a machine learning model for determining acceptable regions outside guard band limits, and FIG4D is a flow diagram of method 400D associated with using a trained machine learning model to use and possibly adjust acceptable regions outside guard band limits.

[0136] Tracking data for a good wafer may vary over time due to acceptable drift, oscillation, noise, spikes, etc.

[0137] 4A, in some embodiments, at block 402, processing logic identifies tracking data associated with substrates (e.g., good substrates) produced via a substrate processing system having a property value that satisfies a threshold. Block 302 may be similar to block 202 of FIG. 2A and / or block 302 of FIG. 3A.

[0138] At block 404, processing logic determines a dynamically acceptable region outside of the guard band limits based on the trace data.

[0139] In some implementations, processing logic determines the upper and lower limits of the guard band based on the tracking data (e.g., via block 206 of FIG. 2A , via block 246 of FIG. 2C , via 3 sigma of the average of the tracking data, etc.) The area between the upper and lower limits of the guard band is a safe area (e.g., a green area) and data points in this area are considered healthy.

[0140] The processing logic determines an acceptable region outside the guard band limits. The acceptable region may be a warning region (e.g., a yellow region) where data points are still normal and are used to track the dynamics of the substrate processing system. The abnormal region (e.g., a red region) is outside the acceptable region, and data points within the abnormal region are considered positive (e.g., abnormal).

[0141] The acceptable region can be determined by user input (e.g., 1 sigma, 4 angstrom sensor data change, 4% sensor data change, etc. outside the guard band limit). The acceptable region can change over time (e.g., a dynamic acceptable region). In some embodiments, a predetermined amount of sensor value change from the guard band limit is acceptable for a predetermined amount of time (e.g., a predetermined number of runs). For example, a 4 angstrom or 4% change in sensor value relative to the guard band limit for 10 runs may be acceptable.

[0142] The dynamic acceptable region and / or guard band limits may change over time. For example, after a predetermined amount of runs (e.g., 10 runs), the acceptable region and / or guard band limits may be adjusted by a predetermined amount (e.g., increased by 4 angstroms, widened by 4%, etc.). As the dynamic acceptable region and / or guard band limits change over time, a new acceptable region may be updated.

[0143] At block 406, processing logic causes execution of a corrective action associated with the substrate processing equipment based on the dynamic acceptable region being outside the guard band limit. Execution of the corrective action may be based on the additional tracking data being outside the acceptable region (see FIG. 4B).

[0144] Referring to FIG4B , in some embodiments, at block 422, processing logic identifies tracking data associated with substrate production via a substrate processing system (e.g., the same substrate processing system as FIG4A ). The tracking data may be associated with a substrate for which it is unknown whether the attribute data of the substrate meets a threshold (e.g., a good substrate) or does not meet the threshold (e.g., a bad substrate). Block 422 may be similar to block 222 of FIG2B and / or block 322 of FIG3B .

[0145] At block 424, processing logic compares the trace data to a dynamically acceptable region outside of the guard band limits. The dynamically acceptable region may be determined by block 404 of FIG. 4A.

[0146] At block 426 , processing logic updates the dynamic acceptable region outside the guard band limit based on the trace data in response to one or more data points of the trace data being within the dynamic acceptable region.

[0147] At block 428, processing logic causes execution of a corrective action associated with the substrate processing equipment in response to one or more data points of the tracking data being outside of the dynamically acceptable region. Execution of the corrective action at block 428 may be similar to execution of the corrective action at block 228 of FIG. 2B and / or block 332 of FIG. 3B.

[0148] 4C , in some embodiments, at block 442, processing logic identifies tracking data associated with substrates (e.g., good substrates) produced via the substrate processing system having a property value that satisfies a threshold. Block 302 may be similar to block 202 of FIG. 2A , block 242 of FIG. 2C , block 302 of FIG. 3A , block 342 of FIG. 3C , and / or block 402 of FIG. 4A .

[0149] At block 444, processing logic identifies historical performance data associated with the historical tracking data. In some examples, the historical performance data indicates whether the substrate is a good substrate or a bad substrate (e.g., all substrates have a property value that meets a threshold, and all substrates have a property value that does not meet the threshold). Block 444 can be similar to block 244 of FIG. 2C and / or block 344 of FIG. 3C.

[0150] At block 446, processing logic trains the machine learning model with data inputs including historical tracking data (e.g., and target outputs including historical performance data) to generate a trained machine learning model indicating a dynamically acceptable region outside of the guard band limits. The trained machine learning model can be used by FIG. 4D.

[0151] Referring to FIG4D , in some embodiments, at block 462, processing logic identifies tracking data associated with production of substrates via a substrate processing system (e.g., the same substrate processing system as FIG4C ). The tracking data may be associated with substrates for which it is not known whether the attribute data of the substrate meets a threshold (e.g., a good substrate) or does not meet a threshold (e.g., a bad substrate). Block 362 may be similar to block 442 of FIG4B .

[0152] At block 464 , processing logic provides the trace data as input to a trained machine learning model associated with a dynamically acceptable region outside of the guard band limits (eg, the trained machine learning model of 446 of FIG. 4C ).

[0153] At block 466 , processing logic receives output from the trained machine learning model indicating predicted data.

[0154] At block 468, in response to determining that one or more data points of the trace data are within the acceptable region based on the prediction data, processing logic updates the dynamic acceptable region outside of the guard band limits. Block 468 may be similar to block 426 of FIG. 4B.

[0155] At block 470, in response to determining that one or more data points of the tracking data are outside of an acceptable region based on the predicted data, processing logic causes corrective action associated with the substrate processing system to be performed. Block 470 may be similar to block 428 of Figure 4B.

[0156] Figures 5A-5E illustrate graphs 500A-500E of allowed types of variation according to certain embodiments (see, e.g., Figures 2A-2D). Figures 5A-5B illustrate graphs 500A-500B of inter-track variability. Figure 5C shows graph 500C of an upper limit that is different from a lower limit. Figure 5D illustrates graph 500D of position-dependent variation. Figure 5E illustrates graph 500E of track segmentation.

[0157] Referring to FIG. 5A , a chart 500A illustrates tracking data 502A-502B. Tracking data 502A may be sensor data (e.g., from one or more sensors) for producing a first substrate (e.g., one or more first substrates), and tracking data 502B may be sensor data (e.g., from the same one or more sensors) for a second substrate (e.g., one or more second substrates). In substrate production operations, acquisition of tracking data 502B may occur earlier than acquisition of tracking data 502A, resulting in an offset in the x-direction (e.g., the x-axis is time and the y-axis is sensor value, tracking data 502A rises earlier than tracking data 502B). This is referred to as inter-track variability (e.g., variation).

[0158] 5B , graph 500B shows tracking data associated with the production of many substrates over time. Guard band 504A that does not account for inter-track variability results in many false positives (e.g., many guard band violation data points that do not correspond to bad substrates). In some examples, line 506 is calculated by taking the average of the tracking data, and guard band 504A is created by taking 3 sigma from line 506 so that guard band 504A is equidistant from line 506 over time. Inter-track variability results in many false positives for guard band 504A, and simply extending guard band 504A results in many false negatives (e.g., data points that deviate in the y direction will not be acquired if guard band 504A is extended).

[0159] By the method 200A of FIG. 2A or the method 200C of FIG. 2C , the guard band 504 is generated taking into account the inter-track variability. Tracking data from good substrates is used to determine the allowed types of variations. For example, the initial guard band 504A can be made from the tracking data of good substrates, and then, since the substrates all have attribute values ​​that meet the threshold, the guard band 504A is widened (e.g., expanded horizontally) in the x-direction in the region to form the guard band 504B, thereby accommodating the inter-track variability.

[0160] 5C , graph 500C illustrates an upper limit that is different from a lower limit.

[0161] 5B, guard band 504A may have upper and lower limits that are equidistant from line 506 that passes through the mean of the tracking data (e.g., 3 sigma). Tracking data from a good wafer may have different amounts of variation above and below line 506. By simply making the upper and lower limits of guard band 504A equidistant, there may be many false positives.

[0162] By the method 200A of Fig. 2A or the method 200C of Fig. 2C, the guard band 504B is generated by taking into account the upper and lower limits that are different from each other. For example, as shown in Fig. 5C, at the beginning of the transition, there may be an upper side change 508A that is larger than the lower side change 508B, and at the end of the transition, there may be a lower side change 508D that is larger than the upper side change 508C.

[0163] 5D , a graph 500D illustrates a graph 500D of position-dependent changes.

[0164] As discussed in FIG5B , guard band 504A may have upper and lower limits (e.g., 3 sigma) that are the same distance from line 506 through the mean of the trace data over time. Trace data from a good wafer may have different amounts of variation at different portions of time. By having the same size guard band 504A over time, many false positives may occur.

[0165] By the method 200A of FIG2A or the method 200C of FIG2C , the guard band 504B is generated by taking into account different amounts of variation 510 over time. For example, as shown in FIG5D , there may be a lower variation 510A in a flat region (e.g., the distance between the upper and lower limits of the guard band 504B of a good substrate is small), and there may be a higher variation 510B in a transient region (e.g., the distance between the upper and lower limits of the guard band 504B of a good substrate is large).

[0166] Referring to FIG5E , graph 500E illustrates a graph 500E of a track segment. As discussed in FIG5B , guard band 504A can have upper and lower limits (e.g., 3 sigma) that are the same distance from line 506 passing through the average of the tracking data over time. In some embodiments, the change in the tracking data is greater than a threshold amount (e.g., see region 512 in FIG5E ).

[0167] If the two boundaries are close to each other, an intermediate position can be used. If one of the boundaries is close to a boundary that changes by more than a threshold amount, the segment boundary can be maintained.

[0168] Two segments in the tracking data showing sharp changes in the original boundary may not be deleted from the tracking data.The parameters of the guard band may be adjusted based on the knowledge of the segments at the sharp change segment points in the region 512.

[0169] The abrupt change in region 512 may be one or more of the following:

[0170] 1) Track the change in slope of the data from negative to positive or from positive to negative (e.g., slope(right)*slope(left) < 0);

[0171] 2) the slope of the right segment and / or the left segment exceeds a threshold slope (e.g., abs(slope(right))>0.1 or abs(slope(left))>0.1 (in normal values)); and / or

[0172] 3) The mean change (e.g., standard deviation) of two connected segments is greater than the mean change (e.g., standard deviation) of the next two segments by a threshold amount (e.g., mean(std[right, left]) > 2*mean(std[right_2, left_2]).

[0173] If two adjacent segments of the tracking data have the above 1), 2), and 3), then the segment boundary (e.g., the data points in region 512) can be considered as a fixed boundary. In some embodiments, region 512 as a segment boundary causes the generation of guard band 504B for points in region 512 instead of removing data points in 512 from the generation of guard band 504B. In some embodiments, region 512 as a segment boundary causes the data points in region 512 to become the limit of guard band 504B with a smaller allowed variation.

[0174] 6A-6E illustrate guard band violation analysis in accordance with certain embodiments (eg, see FIGS. 3A-3D ).

[0175] In FIG. 6A , tracking data is displayed on each of charts 602A-602Z (e.g., block 302 of FIG. 3A , block 344 of FIG. 3C ). Each chart 602A-602Z may depict tracking data for the production of a different substrate, each of which is a good substrate (e.g., having a property value that satisfies a threshold). Each chart 602A-60Z also illustrates a guard band, and each chart 602A-602Z has some data point sets that fall outside the guard band limits. These data point sets are referred to as guard band violation data point sets 604A-604Z (e.g., data point sets associated with guard band violations). Since the tracking data is for good substrates, although the guard band violation data point sets 604A-604Z violate the guard band, they do not indicate bad substrates (e.g., they are false positives).

[0176] Parameters 606 are extracted for each set of guard band violation data points 604 of the trace data. The parameters 606 may include area, duration, amplitude, etc. In some examples, the area may be the area between the guard band limit and a portion of a line passing through the trace data outside the guard band limit. In some examples, the duration may be the amount of time that consecutive guard band violation data points are outside the guard band limit. The amplitude may be the amplitude (e.g., y value) of the difference between the guard band violation data point and the guard band limit (e.g., in the y direction). In some examples, each guard band violation data point may have a corresponding parameter input.

[0177] The parameters 606 and the joint probability density function of the parameters 606 can be used to form a graph 608. The graph 608 can be generated by fitting a multivariate Gaussian distribution (e.g., finding the joint probability density function of the three variables). The guard band violation data point set 604 can form a guard band violation shape representation 610 (e.g., a circle on the graph 608) around the guard band violation data point set 604 (e.g., the good guard band violation data point 614A).

[0178] Graph 608 including guard band violation shape representation 610 may be used to generate graph 612. Graph 612 shows guard band violation shape representation 610 separating good guard band violation data points 614A corresponding to good substrates and bad guard band violation data points 614B corresponding to bad substrates.

[0179] In response to the guard band violation data point set 604 being for a good substrate, the guard band violation shape representation 610 surrounds (e.g., encloses) the data points for the good substrate in the graph 608. The guard band violation shape representation 610 can be used to determine whether a future set of guard band violation data points 604 is a good guard band violation data point 614A corresponding to a good substrate (e.g., inside the guard band violation shape representation 610 on the graph 608, below the guard band violation shape representation 610 on the graph 612) or corresponds to a bad substrate (e.g., outside the guard band violation shape representation 610 on the graph 608, above the guard band violation shape representation 610 on the graph 612).

[0180] In some embodiments, the set 604 of guard band violation data points for the parameters 606, the graph 608, and the graph 612 corresponds to a bad substrate (e.g., a substrate having a property value that does not meet a threshold value). A guard band violation shape representation 610 then surrounds (e.g., encloses) the data points for the bad substrate in the graph 608. The guard band violation shape representation 610 can be used to determine whether a future substrate is bad (e.g., within the guard band violation shape representation 610 on the graph 608, below the guard band violation shape representation 610 on the graph 612) or good (e.g., outside the guard band violation shape representation 610 on the graph 608, above the guard band violation shape representation 610 on the graph 612).

[0181] In response to the guard band violation data point set 604 being for a specific type of substrate (e.g., a specific type of defective substrate, a substrate having a specific property value that does not meet a threshold), the guard band violation shape representation 610 surrounds (e.g., encloses) the data points of the specific type of substrate. The guard band violation shape representation 610 can be used to determine whether a future substrate is a substrate of the specific type (e.g., within the guard band violation shape representation 610 on the graph 608, below the guard band violation shape representation 610 on the graph 612) or is not a substrate of the specific type (e.g., outside the guard band violation shape representation 610 on the graph 608, above the guard band violation shape representation 610 on the graph 612).

[0182] 6B illustrates a graph 620 of a guard band violation analysis according to some embodiments. Graph 620 includes a band average 622 (e.g., average trace data), a guard band limit 624, an offset 626 between the guard band limit 624 and the band average 622, and guard band violation data points 628. Graph 620 has a gap 638 between a first guard band violation data point 628A and a second guard band violation data point 628B.

[0183] The violation duration 630 of the guard band violation data point 628 is the distance (eg, duration of the violation in the x-direction) between the first guard band violation data point 628A outside the guard band limit 624 and the last guard band violation data point 628B outside the guard band limit 624 .

[0184] The violation magnitude 632 (eg, maximum violation, peak violation) is the distance between the guard band limit 624 and a guard band violation data point 628 (eg, the guard band violation data point 628 that is farthest from the guard band limit 624 in the y-direction).

[0185] The violation area 634 is the area between a line passing through the guard band violation data points and the guard band limit 624 (eg, the average value of the violations).

[0186] The violation location 636 may be the location of the maximum violation magnitude 632 (eg, the x-value, time value at which the violation occurred).

[0187] Violation intermittency can include merging or splitting of two or more violations.

[0188] The guard band violation shape characterization 610 may be further based on one or more of a violation duration 630, a violation magnitude 632, a violation area 634, and / or a violation location 636. Guard band violation data points 628 corresponding to a violation duration 630, a violation magnitude 632, a violation area 634, and / or a violation location 636 that meet a threshold may be ignored. Guard band violation data points 628 corresponding to a violation duration 630, a violation magnitude 632, a violation area 634, and / or a violation location 636 that meet a threshold may correspond to a substrate type (e.g., a good substrate, a bad substrate, a type of bad substrate, etc.).

[0189] 6C illustrates a graph 640 of a guard band violation analysis according to certain embodiments. Graph 640 includes a band average 622 (e.g., averaged trace data), a guard band limit 624, instances of a guard band violation data point 628, a gap 638 between two instances of a guard band violation data point 628, and a violation magnitude 632 between the band average 622 and the last guard band violation data point 628 of the first instance of the guard band violation data point 628. Line 642 is between the data points of the band average 622 corresponding to the last guard band violation data point of the first instance of the guard band violation data point 628C and the first guard band violation data point of the second instance of the guard band violation data point 628D.

[0190] The guard band violation shape representation 610 can classify the substrate or tracking data into good, bad, or poor types based on the number of violations. In response to the length of the line 642 meeting a threshold (e.g., being large enough), the first instance of the guard band violation data point 628C and the second instance of the guard band violation data point 628D are considered as a single violation, otherwise the first instance of the guard band violation data point 628C and the second instance of the guard band violation data point 628D are considered as separate violations.

[0191] 6D illustrates a flow diagram of a method 660 for guard band violation analysis according to some embodiments. Classification of guard band violation data points may be via multivariate (eg, multivariate) analysis (eg, based on sensor data from different types of sensors).

[0192] In some embodiments, the trace data from sensors 662A-662N is individually compared to corresponding guard bands 664A-664N. Guard band violation data points from the comparison of the trace data from sensor 662 to guard band 664 are processed via a violation probability density function (pdf) 666 and compared to a threshold 668 to provide test trace data 670.

[0193] The test trace data provides a score 672 for each sensor 662. A score ranking 674 is performed based on the score 672. A sensor importance ranking 676 is performed based on the score ranking 674. A composite score 677 is generated based on the sensor importance ranking 676. The composite score 677 is compared to a composite threshold 678 to provide a fault detection result 679 (e.g., classifying a guard band violation data point as a good substrate or a bad substrate, an anomaly detection result).

[0194] 6E illustrates a flow chart of a method 680 for guard band violation analysis according to certain embodiments. Guard band violation shape characterization may be based on guard band violation data points and segment feature extraction.

[0195] At block 682, trace data is identified. This may be similar to block 302 of FIG. 3A and / or block 342 of FIG. 3C for generating guard bands. This may be similar to block 322 of FIG. 3B and / or block 362 of FIG. 3D for using guard bands.

[0196] At block 684, full trace analysis is performed on the trace data from block 682. For example, area, duration, amplitude, etc. may be determined (eg, see FIGS. 6A-6C ).

[0197] At block 686, a guard band model is generated based on the full trace analysis of the trace data. This may be similar to block 206 of FIG. 2A and / or block 246 of FIG. 2C.

[0198] At block 688, abnormal features (e.g., guard band violation data points) are determined by comparing the trace data to a guard band model (e.g., guard bands). This may be similar to block 306 of FIG. 3A, block 326 of FIG. 3B, block 346 of FIG. 3C for training a model, and / or block 366 of FIG. 3D. The abnormal features (e.g., guard band violation data points) may be persistent or intermittent.

[0199] At block 690, segment feature extraction is performed on the tracking data of block 682. This may be similar to FIG. 5E (e.g., segment boundaries are considered fixed boundaries, such as data points in region 512 of FIG. 5E). In some embodiments, the guard band model generates anomaly features of block 688 that can be combined with features identified at block 690 (e.g., persistent features of block 692 and intermittent features of block 692).

[0200] At block 692, persistent features of the tracking data are determined based on the segmented feature extraction of block 690. A persistent feature may be a feature of the tracking data that meets a threshold number of occurrences.

[0201] At block 694, feature parameter cross-correlation of the persistent features of the tracking data is performed (e.g., based on subject matter expertise 699, such as user input). Feature parameter cross-correlation can be based on parameters such as amplitude, location, area, duration, etc. (see FIG. 6B ). In some embodiments, the guard band violation shape characterization can classify guard band violation data points corresponding to the persistent features as being associated with good substrates.

[0202] At block 696, intermittent features of the tracking data are determined based on the segmented feature extraction of block 690. An intermittent feature may be a feature of the tracking data that does not meet a threshold number of occurrences.

[0203] At block 698, feature presence cross-correlation is performed based on the intermittent features from block 696, the anomalous features from block 688, and / or subject matter expertise 699. In some embodiments, the guard band violation shape characterization may classify guard band violation data points corresponding to intermittent features as being associated with bad substrates.

[0204] 7A-7F illustrate dynamically acceptable regions outside of guard band limits according to certain embodiments.

[0205] FIG. 7A illustrates a flow diagram of a method 700 associated with a dynamic region outside of a guard band limit.

[0206] At block 702, processing logic identifies baseline tracking data. This may be similar to block 302 of FIG. 3A and / or block 342 of FIG. 3C for generating guard bands. This may be similar to block 322 of FIG. 3B and / or block 362 of FIG. 3D for using guard bands.

[0207] At block 704, processing logic identifies a guard band based on the trace data. The guard band may be generated by block 206 of Figure 2A and / or block 246 of Figure 2C.

[0208] At block 706 , processing logic performs a multivariate (eg, multivariate) analysis (MVA) distribution based on the guard band and baseline tracking data.

[0209] At block 708, processing logic identifies a probability on the baseline tracking data. The probability can be a probability that the data point is in a safe region (e.g., region 736 of graph 734 or graph 742 of FIG. 7B ), an alarm region (e.g., region 738 of graph 734 or graph 742 of FIG. 7B ), or an abnormal region (e.g., region 740 of graph 734 or graph 742 of FIG. 7B ).

[0210] At block 710, processing logic generates an internal threshold. The internal threshold may be a line that separates an alarm region (eg, region 738 of graph 734 or graph 742 of FIG. 7B) from an abnormal region (eg, region 740 of graph 734 or graph 742 of FIG. 7B).

[0211] At block 712, processing logic generates enhanced tracking data. The enhanced tracking data may be simulated tracking data formed by adjusting the baseline tracking data of block 702 using one or more of blocks 714-720.

[0212] At block 714, the enhanced tracking data may include tracking data with a slight drift. The drift may include an increase in the sensor value of the baseline tracking data of block 702 in the y-direction.

[0213] At block 716, the enhanced tracking data may include tracking data with minor repetitions.One or more portions of the baseline tracking data of block 702 may repeat over time (eg, in the x-direction).

[0214] At block 718, the enhanced tracking data may include tracking data with less noise (eg, and / or oscillations).The baseline tracking data of block 702 may be adjusted (eg, increased and decreased) in the y-direction to simulate noise.

[0215] At block 720, the enhanced tracking data may include tracking data with micro-spikes. The baseline tracking data of block 702 may include peaks and valleys to simulate micro-spikes.

[0216] At block 722 , processing logic identifies a probability of the baseline tracking data plus the enhanced tracking data. Block 722 may be similar to block 708 .

[0217] At block 724 , processing logic generates an external threshold. Block 724 may be similar to block 710 .

[0218] 7B illustrates the dynamic region outside of the guard band limits.Graph 730 illustrates baseline tracking data (eg, of block 702), and graph 732 illustrates enhanced tracking data (eg, of block 712).

[0219] Graph 734 illustrates region 736 (e.g., allowed region, green region), region 738 (e.g., warning region, yellow region), and region 740 (e.g., abnormal region). Graph 734 may be formed based on tracking data from a good substrate. All data points on graph 734 may be located in region 736 or region 738.

[0220] Graph 742 shows new tracking data for determining whether a substrate is a good substrate or a bad substrate. Graph 742 has tracking data in region 736, region 738, and region 740. The data points in region 740 correspond to bad substrates. Region 738 (e.g., yellow region, alarm region) will be recalculated based on the data points in region 738.

[0221] FIG. 7C illustrates a flow diagram of a method 744 associated with a dynamic region outside of the guard band limits.

[0222] At block 746, processing logic identifies a training set of tracking data. The training set may be previous tracking data for producing substrates using the substrate processing equipment.

[0223] At block 748, processing logic identifies new tracking data. In some embodiments, the new tracking data includes new sensor data associated with producing a new substrate using the same substrate processing equipment as block 746 or using a different substrate processing equipment. In some embodiments, the new tracking data includes simulated tracking data that is made based on the tracking data of block 746 and includes one or more of drift, noise, spikes, oscillations, etc.

[0224] At block 750, processing logic sorts the new trace data from block 748. The new trace data may be sorted based on its representativeness (e.g., proximity) to the trace data of block 746. The new trace data may be sorted based on proximity to each other (e.g., to remove anomalies). Data points of the new trace data may be rated as good (e.g., green area), acceptable and used to adjust guard bands (e.g., yellow area), or abnormal (e.g., red area). For example, a data point of the new trace data may be in the green area (e.g., no input guard band adjustment), in the acceptable yellow area (e.g., no anomalies, but with input GB adjustment), or in the abnormal red area (e.g., anomalies, but no input guard band adjustment).

[0225] At block 752, processing logic selects at least a portion of the new trace data from block 748 (eg, based on the ranking from block 750). Processing logic may select the highest ranked trace data based on the ranking of block 750.

[0226] At block 754, processing logic updates (e.g., see FIG. 7E ) or retrains (e.g., see FIG. 7F ) the old guard band based on the training set and / or tracking data selected from block 752. Processing logic may trigger a guard band update when certain criteria are met (e.g., the tracking data is noisy, the tracking data is not drifting, etc.).

[0227] At block 756 , processing logic identifies the new guard band that has been updated or retrained from block 754 .

[0228] 7D illustrates the dynamic region outside the guard band limits. Graph 760 illustrates new trace data (eg, from block 748 of FIG. 7C ), and graph 762 illustrates the trace ordering of the trace data of graph 760 (eg, see block 750 of FIG. 7C ).

[0229] Graph 764 illustrates a selected trace (eg, block 752 from FIG. 7C ), and graph 766 illustrates an adaptive guard band updated from the selected trace (eg, the most informative trace) from graph 764 (eg, the new guard band of block 756 of FIG. 7C ).

[0230] FIG7E shows the dynamic region outside the guard band limit for noise. Block diagram 770A illustrates that processing data (eg, trace data) that generates the guard band is accumulating. In response to processing logic detecting noise (eg, periodic changes), processing logic updates the guard band with all historical data.

[0231] Graph 772A illustrates initial trace data (eg, first 60 traces), and graph 774A illustrates an initial guard band of the trace data of graph 772A.

[0232] Graph 776A illustrates trace data (eg, first 120 traces) and graph 778A illustrates subsequent guard bands of the trace data of graph 776 A. As shown in graph 778A, the guard bands widen over time to accommodate increases in acceptable noise.

[0233] FIG7F illustrates a dynamic region outside of the guard band limits for drift. Block diagram 770A illustrates that older historical data (e.g., previous trace data) is forgotten, and only some amount (e.g., zero or more) of newer historical data is combined with new processing data (e.g., current trace data) to generate the guard band. In response to processing logic detecting drift (e.g., a change in the y-direction of the sensor value over time), the processing logic triggers the forgetting mechanism and updates the guard band with only the most recent trace data.

[0234] Graph 772B illustrates initial trace data (eg, first 50 traces) and graph 774B illustrates an initial guard band for the trace data of graph 772B.

[0235] Graph 776B illustrates trace data (eg, the next 50 traces) and graph 778B illustrates subsequent guard bands of the trace data of graph 776 B. As shown in graph 778B, the guard bands move (eg, rise) in the y-direction over time to accommodate acceptable drift.

[0236] 8A-8B illustrate guard band adaptation according to certain embodiments. The guard band adaptation shown in any portion of FIG. 8A-8B may be used in any method of the present disclosure (e.g., FIG. 2A-2D, 3A-3D, and / or 4A-4D) to adapt the guard band to system dynamics (e.g., not to violations such as fault level transitions).

[0237] 8A illustrates horizontal scaling. Graph 810A illustrates tracking data 812A and tracking data 812B. Tracking data 812A and 812B may have different scales in the y direction. As shown in graph 810B, tracking data 812A and / or tracking data 812B may be horizontally scaled (e.g., recipe endpoint horizontal scaling).

[0238] In some embodiments, the tracking data 812A and 812B are for good substrates. By horizontally scaling, more accurate guard bands can be created based on the tracking data 812A and 812B. By horizontally scaling the tracking data that may correspond to good or bad substrates, the difference between the tracking data and the guard bands can be more accurately identified (e.g., reducing false positives and false negatives).

[0239] 8B illustrates vertical scaling and horizontal warping. Graph 850A illustrates tracking data 812A and tracking data 812B having different scales in the y-direction. Vertical scaling may be performed on tracking data 812A and 812B of graph 850A to generate graph 850B (e.g., amplitude normalization). Horizontal warping may be performed on tracking data 812A and 812B of graph 850B to generate graph 850C (e.g., dynamic time warping).

[0240] By tracking warping and scaling, out-of-phase and different amplitude factors can be ignored. Horizontal warping is applied to ignore out-of-phase factors and preserve vertical noise. Vertical and horizontal scaling are applied to address domain shift (e.g., applying guard bands to different domains, such as different formulations).

[0241] Fig. 9 shows a block diagram of a computer system 900 according to certain embodiments. In some embodiments, the computer system 900 can be connected to other computer systems (e.g., via a network such as a local area network (LAN), an intranet, an extranet, or an interconnected network). The computer system 900 can operate with the ability of a server or client computer in a client-server environment, or operate as a peer computer in a peer-to-peer or distributed network environment. The computer system 900 can make a personal computer (PC), a tablet computer, a set-top box (STB), a personal digital assistant (PDA), a cellular phone, a network device, a server, a network router, a switch or a bridge, or any device that can perform a set of instructions (sequentially or otherwise) specifying the action to be taken by the device. In addition, the term "computer" should include any set of computers that perform a set (or multiple sets) of instructions individually or collectively to perform any one or more methods described herein.

[0242] In another aspect, the computer system 900 may include a processing device 902 , a volatile memory 904 (e.g., a random access memory (RAM)), a nonvolatile memory 906 (e.g., a read-only memory (ROM) or an electronically erasable programmable ROM (EEPROM)), and a data storage device 918 , which may communicate with each other via a bus 908 .

[0243] The processing device 902 may be composed of one or more processors, such as a general-purpose processor (e.g., a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a microprocessor that implements other types of instruction sets, or a microprocessor that implements a combination of multiple types of instruction sets) or a special-purpose processor (e.g., an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), or a network processor).

[0244] The computer system 900 may further include a network interface device 922 (e.g., coupled to the network 974). The computer system 900 may also include a video display unit 910 (e.g., LCD), an alphanumeric input device 912 (e.g., keyboard), a cursor control device 914 (e.g., mouse), and a signal generating device 920.

[0245] In some embodiments, the data storage device 918 may include a non-transitory computer-readable storage medium 924 (e.g., a non-transitory machine-readable storage medium) on which instructions 926 encoding any one or more of the methods or functions described herein may be stored, including instructions encoding components of FIG. 1 (e.g., prediction component 114 for prediction or detection, model 190, etc.) and for implementing the methods described herein. The instructions stored on the memory, when executed, cause the processing device to perform one or more operations described herein.

[0246] The instructions 926 may also reside, in whole or in part, within the volatile memory 904 and / or within the processing device 902 during execution of the instructions by the computer system 900 , and thus the volatile memory 904 and the processing device 902 may also constitute machine-readable storage media.

[0247] Although the computer-readable storage medium 924 is shown as a single medium in the illustrative example, the term "computer-readable storage medium" shall be deemed to include a single medium or multiple media (e.g., a centralized or distributed database, and / or associated caches and servers) storing one or more sets of executable instructions. The term "computer-readable storage medium" shall also be deemed to include any tangible medium capable of storing or encoding a set of instructions to be executed by a computer and causing the computer to perform any one or more of the methods of the present disclosure. The term "computer-readable storage medium" shall include, but is not limited to, solid-state memories, and optical and magnetic media.

[0248] The methods, components, and features described herein may be implemented by discrete hardware components, or may be integrated into the functionality of other hardware components (such as ASICs, FPGAs, DSPs, or similar devices). In addition, the methods, components, and features may be implemented by firmware modules or functional circuit systems within a hardware device. Furthermore, the methods, components, and features may be implemented in any combination of hardware devices and computer program components or in a computer program.

[0249] Unless expressly stated otherwise, terms such as "identify," "generate," "cause," "provide," "receive," "determine," "update," "compare," "train," "distort," "scale," "acquire," etc. refer to actions and processes performed or implemented by a computer system that manipulate and transform data represented as physical (electronic) quantities within registers and memories of the computer system into other data similarly represented as physical quantities in memories or registers within the computer system or other such information storage, transmission, or display devices. In addition, the terms "first," "second," "third," "fourth," etc., used herein refer to labels used to distinguish different elements and may not have ordinal meanings according to their numerical names.

[0250] The examples described herein also relate to an apparatus for performing the methods described herein. This apparatus may be specially constructed for performing the methods described herein, or it may include a general-purpose computer system selectively programmed by a computer program stored in the computer system. Such a computer program may be stored in a computer-readable tangible storage medium.

[0251] The methods and illustrative examples described herein are not inherently related to any particular computer or other device. Various general purpose systems may be used in accordance with the teachings described herein, or it may prove convenient to construct more specialized devices to perform each of the methods described herein and / or their respective functions, routines, subroutines, or operations. Examples of the structures of various such systems are set forth in the description above.

[0252] The above description is intended to be illustrative rather than limiting. Although the present disclosure has been described with reference to specific illustrative examples and embodiments, it will be appreciated that the present disclosure is not limited to the described examples and embodiments. Therefore, the scope of the disclosure should be determined with reference to the appended claims and the full scope of equivalent technical solutions granted by these claims.

Claims

1. A method comprising the following steps: identifying tracking data comprising a plurality of data points associated with producing, via a substrate processing system, a substrate having a property value that satisfies a threshold; determining a guard band violating data point of the plurality of data points of the trace data based on the guard band; as well as A guard band violation shape characterization is determined based on the guard band violation data points, wherein classification of additional guard band violation data points of additional tracking data is to be based on the guard band violation shape characterization, wherein performing corrective action associated with the substrate processing system is based on the classification. 2 . The method of claim 1 , wherein the guard band violation shape characterization is a weighted combination of one or more of guard band violation duration, guard band violation magnitude, guard band violation area, guard band violation location, or guard band violation intermittency. 3 . The method of claim 1 , wherein the classification of the additional guard band violation data points comprises concatenating consecutive or future violations into a single violation based on the guard band violation shape characterization. 4 . The method of claim 3 , wherein the step of cascading the consecutive or future violations into the single violation is further based on one or more non-violation shapes between two violations of the consecutive or future violations. The method of claim 1 , wherein the step of determining the guard band violation data points is via multivariate analysis.

6. The method of claim 1, wherein the step of determining the guard band violation data point comprises the step of: characterizing the guard band violation data point via multivariate analysis.

7. The method of claim 1, wherein the step of determining the guard band violation data point is further based on: segmenting a portion of the trace data associated with a change in value exceeding a threshold change; and extracting features from the trace data.

8. The method of claim 1, wherein the step of determining the guard band violation shape representation comprises the step of training a machine learning model using data input comprising the trace data to form a trained machine learning model associated with the guard band violation shape representation.

9. The method according to claim 1, further comprising the following steps: Identify additional tracking data; determining the additional guard band violation data points of the additional tracking data based on the guard band; providing the additional guard band violation data points as data input to a trained machine learning model; receiving output data including prediction data from the trained machine learning model; as well as The classification of the additional guard band violation data point is determined based on the predicted data.

10. A method comprising the following steps: identifying tracking data comprising a plurality of data points associated with production of a substrate via a substrate processing system; determining a guard band violating data point of the plurality of data points of the trace data based on the guard band; as well as A classification of the guard band violation data point is determined based on a guard band violation shape characterization, wherein performing a corrective action associated with the substrate processing system is based on the classification of the guard band violation data point. 11 . The method of claim 10 , wherein the guard band violation shape characterization is a weighted combination of one or more of guard band violation duration, guard band violation magnitude, guard band violation area, guard band violation location, or guard band violation intermittency.

12. The method of claim 10, wherein the step of determining the classification of the guard band violation data point comprises concatenating consecutive or future violations into a single violation based on the guard band violation shape characterization.

13. The method of claim 10, wherein the step of determining the guard band violation data points is via multivariate analysis.

14. The method of claim 10, wherein the step of determining the guard band violation data point is further based on: segmenting a portion of the trace data associated with a change in value that exceeds a threshold change; and extracting features from the trace data.

15. The method according to claim 10, further comprising the steps of: receiving historical tracking data associated with historical production of historical substrates having historical attribute values ​​satisfying a threshold via the substrate processing system; and A machine learning model is trained using data inputs including the historical tracking data to form a trained machine learning model associated with the guard band violation shape characterization.

16. The method according to claim 10, further comprising the steps of: providing the guard band violation data points as data input to a trained machine learning model; receiving output data including prediction data from the trained machine learning model; and The classification of the guard band violation data point is determined based on the prediction data.

17. A non-transitory computer-readable storage medium having instructions stored thereon, which when executed by a processing device causes the processing device to perform operations comprising: identifying tracking data comprising a plurality of data points associated with producing, via a substrate processing system, a substrate having a property value that satisfies a threshold; determining a guard band violating data point of the plurality of data points of the trace data based on the guard band; as well as A guard band violation shape characterization is determined based on the guard band violation data points, wherein classification of additional guard band violation data points of additional tracking data is to be based on the guard band violation shape characterization, wherein performing corrective action associated with the substrate processing system is based on the classification.

18. The non-transitory computer-readable storage medium of claim 17, wherein the guard band violation shape characterization is a weighted combination of one or more of guard band violation duration, guard band violation magnitude, guard band violation area, guard band violation location, or guard band violation intermittency.

19. The non-transitory computer-readable storage medium of claim 17, wherein the classification of the additional guard band violation data points comprises concatenating consecutive violations into a single violation based on the guard band violation shape characterization.

20. The non-transitory computer readable storage medium of claim 17, wherein the step of determining the guard band violation data points is via multivariate analysis.

Images