Guard Band in Substrate Processing System

Abstract

The method includes identifying trace data including a plurality of data points, the trace data being associated with the fabrication of substrates having characteristic values that meet a threshold through a substrate processing system. The method further includes determining guard band violation data points among the plurality of data points of the trace data based on a guard band. The method further includes determining a guard band violation shape characterization based on the guard band violation data points. Classification of additional guard band violation data points of additional trace data is based on the guard band violation shape characterization. Execution of a corrective action associated with the substrate processing system is based on the classification.

Description

Technical Field

[0001] The present disclosure relates to guard bands, and more particularly to guard bands in a substrate processing system.

Background Art

[0002] Products can be fabricated by performing one or more manufacturing processes using manufacturing equipment. For example, a substrate can be fabricated through substrate processing operations using substrate processing equipment. A product having specific characteristics should be fabricated. Sensor data is monitored in relation to the substrate manufacturing process.

Summary of the Invention

[0003] The following is a simplified summary of the present disclosure to provide a basic understanding of some aspects of the present disclosure. This summary is not an extensive overview of the present disclosure. It is not intended to identify key or critical elements of the present disclosure nor to delineate any scope of particular embodiments of the present disclosure or any scope of the claims. The sole purpose of this summary is to present some concepts of the present disclosure in a simplified form as a prelude to the more detailed description that is presented later.

[0004] In one aspect of the present disclosure, a method includes identifying trace data including a plurality of data points, the trace data being associated with the fabrication of a substrate having a characteristic value that meets a threshold through a substrate processing system. The method further includes determining, based on a guard band, guard band violation data points among the plurality of data points of the trace data. The method further includes determining, based on the guard band violation data points, a guard band violation shape characterization. Classification of additional guard band violation data points of additional trace data is based on the guard band violation shape characterization. Execution of a corrective action associated with the substrate processing system is based on the classification.

[0005] In one aspect of the present disclosure, the method includes identifying trace data including a plurality of data points, the trace data being associated with the fabrication of a substrate via a substrate processing system. The method further includes determining guard band violation data points among the plurality of data points of the trace data based on a guard band. The method further includes determining a classification of the guard band violation data points based on guard band violation shape characterization. Execution of a corrective action associated with the substrate processing system is based on the classification of the guard band violation data points.

[0006] In one aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing instructions that, when executed, cause a processing device to perform operations including identifying trace data including a plurality of data points, the trace data being associated with the fabrication of a substrate having a characteristic value that meets a threshold via a substrate processing system. The operations further include determining guard band violation data points among the plurality of data points of the trace data based on a guard band. The operations further include determining a guard band violation shape characterization based on the guard band violation data points. Classification of additional guard band violation data points of additional trace data is based on the guard band violation shape characterization. Execution of a corrective action associated with the substrate processing system is based on the classification.

[0007] The present disclosure is shown by way of example and not limitation in the figures of the accompanying drawings.

Brief Description of the Drawings

[0008]

Figure 1

Figures 2A - 2D

Figures 3A - 3D

Figures 4A - 4D

Figures 5A - 5E

Figures 6A - 6E

Figures 7A - 7F

Figures 8A - 8B

Figure 9

[0009] Techniques related to guard bands in a substrate processing system (e.g., guard band enhancement, guard band violation profiling, and dynamic regions outside the guard band) are described herein. The guard band can be upper and lower thresholds (e.g., an acceptable error range) before and after a target value. Sensor data can be compared with the guard band to determine whether the sensor data is within the threshold range (e.g., a good substrate) or outside the threshold range (e.g., a bad substrate).

[0010] Products are made by performing one or more manufacturing processes using manufacturing equipment. A product having specific characteristics should be made. For example, a substrate is made through substrate processing operations using substrate processing equipment. Substrates that meet specific characteristic values (e.g., dimensions confirmed by measurement data) should be used, and substrates that do not meet the specific characteristic values should be discarded. Sensor data associated with the substrate processing operations is collected over time. The sensor data is monitored for multiple purposes, including attempting to make a substrate that meets specific characteristic values, attempting to confirm that the equipment is functioning properly, attempting to determine whether the equipment requires current or future repair or replacement, and attempting to determine adjustments to equipment parameters to make the process more effective (e.g., with respect to conditions or criteria such as yield or quality, throughput or quantity, cost or lifespan). For the purpose of simplifying the explanation, this discussion focuses on the purpose of attempting to make a substrate that meets specific characteristic values.

[0011] Conventionally, sensor data is summarized over specific policies or policy actions associated with the fabrication of substrates by a device. This summary is defined using a set of statistical information such as mean and variance. These summary statistical information are then compared to set limits such as the mean between a lower limit value and an upper limit value. If the sensor data is within the range of the limits, it is presumed that the substrate meets a specific characteristic value, and if the sensor data is outside the range of the limits, it is presumed that the substrate does not meet the specific characteristic value. Limits that are too narrow cause false detections (e.g., inaccurately predicting that a substrate does not meet a characteristic value). Limits that are too wide cause detection misses (e.g., inaccurately predicting that a substrate meets a characteristic value). Variations in sensor data and variations in substrate processing equipment can cause many false detections. Allowing for variations in sensor data and substrate processing equipment by widening the limits can cause many detection misses. Conventional systems inaccurately label substrates as meeting or not meeting characteristic values, causing waste of materials, reduction in yield, defective products, increase in user time, increase in equipment downtime, etc. Attempting to correct incorrect labeling uses extra processing costs, bandwidth, energy consumption, measurement operations, user time, etc.

[0012] The methods, devices, and systems of the present disclosure provide an improvement to guardbands in a substrate processing system that solves the above and other deficiencies of conventional solutions.

[0013] In some embodiments, a processing device identifies trace data associated with the fabrication of a substrate having a characteristic value that meets a threshold (e.g., a good substrate). The trace data can include a set of sensor data associated with the fabrication of different substrates from different types of sensors. In some embodiments, the data is analyzed at the trace level by using guard bands (e.g., not only providing summary statistics of the sensors across a policy or policy action). The guard bands can provide upper and lower thresholds across the length of the trace data. This provides many advantages compared to summary statistics (e.g., the present disclosure can identify and profile specific violations in the trace data). The processing device generates an initial guard band based on the trace data (e.g., taking 3σ before and after the average of the trace data in each time period within the data, where 3σ is determined by analyzing multiple runs of the data in that specific time period). The processing device determines an acceptable dispersion type of the guard band based on the trace data. Since the trace data is for a good wafer, the acceptable dispersion type of the guard band can be time-shifted from one or more of the set of sensor data to meet the average of the sensor data (e.g., move in the x direction). The acceptable dispersion type can include a guard band upper limit that is a different distance from the guard band lower limit from the average of the trace data. The acceptable dispersion type can include wider guard band limits and narrower guard band limits in a specific portion of the guard band. The processing device generates a guard band based on the trace data and the acceptable dispersion type. The processing device compares additional trace data (e.g., of a substrate whose quality is unknown whether it is good or bad) with the guard band, and in response to one or more data points not being within the range of the guard band, the processing device performs a corrective action (e.g., discarding the substrate, interrupting the substrate processing operation, inspecting the substrate, etc.).

[0014] In some embodiments, both trace data for good substrates and trace data for bad substrates are used. The guard band is determined or updated based on an improved understanding of the differences between good and bad trace data, and the bad trace data may be a single category or multiple categories representing different states or degrees of bad substrates.

[0015] In some embodiments, the processing device identifies trace data associated with the fabrication of substrates (e.g., good substrates) having characteristic values that meet a threshold. The processing device determines guard band violation data points of the trace data based on the guard band. The processing device determines a guard band violation shape characterization based on the guard band violation data points. The processing device compares additional trace data (e.g., of a substrate whose quality, whether good or bad, is unknown) with the guard band to determine guard band violation data points. The processing device determines a classification of the guard band violation data points (e.g., whether the guard band violation data points correspond to good wafers or bad wafers) based on the guard band violation shape characterization.

[0016] In some embodiments, the processing device identifies trace data associated with the fabrication of a substrate (e.g., a good substrate) having characteristic values that meet a threshold. Based on the trace data, the processing device determines a dynamic acceptable region outside the guard band limits. The dynamic acceptable region is the region outside the guard band corresponding to the data points of a good substrate that results from acceptable noise, acceptable drift, etc. The processing device compares additional trace data (e.g., of a substrate whose quality is unknown) with the acceptable region outside the guard band limits. In response to one or more of the data points being outside the range of the acceptable region, the processing device takes corrective action (e.g., discarding the substrate). In response to one or more of the data points being within the range of the acceptable region, the processing device updates the acceptable region. For example, the updated acceptable region can tolerate additional drift or noise based on the average of the additional trace data processed.

[0017] In some embodiments, the processing device (e.g., executing a guard band method) presents trace data runs having violations, including the shape characteristics of these violations, as a result of trace data runs concluded to be within the range of acceptable guard band limits, and / or regions where the processing device (e.g., executing a guard band method) cannot clearly determine the occurrence of a violation. The processing device (e.g., executing a guard band method) can enable a user (e.g., a subject matter expert) to confirm or invalidate the conclusions and suggestions of the guard band method. The processing device (e.g., executing a guard band method) can use feedback from the user (e.g., a subject matter expert) to update the guard band limits, other violation evaluations, and / or guard band characterization.

[0018] Aspects of the present disclosure provide technical advantages over conventional solutions. The present disclosure has fewer false detections and missed detections compared to conventional solutions. This enables less material waste, increased yield, fewer defective products, less user time, less equipment downtime, and the like. The present disclosure has fewer corrections for mislabeling of substrates compared to conventional solutions. This enables less process processor cost, used bandwidth, energy consumption, measurement operations, user time, and the like.

[0019] Some embodiments of the present disclosure are described in relation to substrate processing. In some embodiments, the present disclosure is also applicable to other types of manufacturing processes.

[0020] Some embodiments of the present disclosure are described in relation to monitoring sensor data for the purpose of fabricating a substrate that meets certain characteristic values. In some embodiments, the present disclosure can also monitor sensor data for other purposes, such as confirming that a device is functioning properly, determining to perform repair or replacement (e.g., preventive maintenance) of a device or device component, determining adjustments to device parameters to make a process more effective (e.g., with respect to conditions or criteria such as yield or quality, throughput or quantity, cost or lifespan).

[0021] Some embodiments of the present disclosure are described in relation to performing corrective actions. In some embodiments, performing corrective actions can include identifying characteristics of trace data as degraded, different, or the like.

[0022] FIG. 1 is a block diagram showing an exemplary system 100 (exemplary system architecture) according to a particular embodiment. System 100 includes a client device 120, manufacturing equipment 124 (e.g., substrate processing equipment), sensors 126, measurement equipment 128, a prediction server 112, and a data store 140. The prediction server 112 can be part of a prediction system 110. The prediction system 110 can further include server machines 170 and 180. Using the prediction system 110, it is possible to predict whether an abnormality has occurred, detect that an abnormality has occurred, etc. (e.g., using guardband technology).

[0023] In some embodiments, the manufacturing equipment 124 (e.g., a cluster tool) is part of a substrate processing system (e.g., an integrated processing system). The manufacturing equipment 124 includes one or more of a controller, a sealed system (e.g., a substrate carrier, a front opening unified pod (FOUP), an auto-teach FOUP, a process kit sealed system, a substrate sealed system, a cassette, etc.), a side storage pod (SSP), an aligner device (e.g., an aligner chamber), a factory interface (e.g., an equipment front end module (EFEM)), a load lock, a transfer chamber, one or more processing chambers, a robotic arm (e.g., disposed within the transfer chamber, within the front interface, etc.). The sealed system, SSP, and load lock attached to the factory interface, and the robotic arm disposed within the factory interface are for transferring contents (e.g., substrates, process kit rings, carriers, certified wafers, etc.) between the sealed system, SSP, load lock, and factory interface. The aligner device is disposed within the factory interface to align the contents. The load lock and processing chamber attached to the transfer chamber, and the robotic arm disposed within the transfer chamber are for transferring contents (e.g., substrates, process kit rings, carriers, certified wafers, etc.) between the load lock, processing chamber, and transfer chamber. In some embodiments, the manufacturing equipment 124 includes components of the substrate processing system. In some embodiments, the manufacturing equipment 124 is used to fabricate one or more products (e.g., substrates, semiconductors, wafers, etc.). In some embodiments, the manufacturing equipment 124 is used to fabricate one or more components used within the substrate processing system.

[0024] Sensor 126 can be coupled to manufacturing equipment 124. Sensor 126 can provide sensor data associated with manufacturing equipment 124 (e.g., associated with the production of a corresponding product such as a substrate by manufacturing equipment 124). The sensor data can be stored as time-dependent measurement values (e.g., trace data 142). Trace data 142 can include past trace data 144 and current trace data 146. Trace data 142 can be used for the health state of the equipment and / or the health state of the product (e.g., product quality). Manufacturing equipment 124 can produce products by running a run according to a policy or over a period of time. In some embodiments, trace data 142 can include one or more values such as temperature (e.g., heater temperature), interval (SP), pressure, high frequency radio frequency (HFRF), voltage of an electrostatic chuck (ESC), current, flow rate, power, voltage, etc. Trace data 142 can be associated with or indicative of manufacturing parameters such as hardware parameters (e.g., settings or components, such as size, type, etc.) of manufacturing equipment 124 or process parameters of manufacturing equipment 124. Alternatively, or in addition, data associated with some hardware parameters can be stored as manufacturing parameters. Manufacturing parameters can indicate input settings for manufacturing devices (e.g., heater power, gas flow rate, etc.). Trace data 142 and / or manufacturing parameters can be provided when manufacturing equipment 124 is executing a manufacturing process (e.g., device read values when processing a product). Trace data 142 can be different for each product (e.g., for each substrate).

[0025] The measuring instrument 128 can be used to measure the characteristics of products such as substrates (e.g., processed substrates, partially processed substrates, etc.). The measuring instrument can incorporate analysis to estimate measurement values or make better judgments. The measurement data can be included in the performance data 150 together with other performance metrics such as equipment maintenance, yield, etc. The performance data 150 can include past performance data 152 and current performance data 154. The measurement data and / or the performance data 150 can include virtual measurement data, non-virtual measurement data, a mixture of virtual and non-virtual measurement data, etc.

[0026] In some embodiments, the trace data 142, the performance data 150, and / or the manufacturing parameters can be processed (e.g., by the client device 120 and / or the prediction server 112). The processing of the trace data 142 can include generating features. In some embodiments, these features are patterns (e.g., gradients, widths, heights, peaks, etc.) within the trace data 142 or the performance data 150, or combinations of values from the trace data 142 or the performance data 150 (e.g., power derived from voltage and current, etc.). The trace data 142 can include features that can be used by the prediction component 114 and / or the client device 120 to perform signal processing and / or obtain prediction data 168 for executing corrective measures. The prediction component 114 may be used to predict whether an anomaly has occurred, detect that an anomaly has occurred (e.g., using guardband techniques), etc.

[0027] Each instance of the trace data 142 (e.g., a set) can correspond to a product (e.g., a substrate), a set of manufacturing equipment 124, the type of substrate fabricated by the manufacturing equipment 124, and so on. Similarly, each instance of the performance data 150 or manufacturing parameters can correspond to a product, a set of manufacturing equipment, the type of substrate fabricated by the manufacturing equipment, and so on. The data store 140 can further store information indicating a set of different data types, such as a set of trace data, sensor data, measurement data, and / or a set of manufacturing parameters associated with the same product, manufacturing equipment, substrate type, and so on.

[0028] In some embodiments, the prediction system 110 can generate the prediction data 168 using supervised machine learning (e.g., a supervised data set, the performance data 150 includes measurement data, and the trace data 142 used to train the model 190 is associated with good substrates and bad substrates, etc.). In some embodiments, the prediction system 110 can generate the prediction data 168 using semi-supervised learning (e.g., a semi-supervised data set, the performance data 150 is a prediction rate, and the trace data 142 used to train the model 190 is associated with only good substrates, etc.). In some embodiments, the prediction system 110 can generate the prediction data 168 using unsupervised machine learning (e.g., an unsupervised data set, clustering, clustering based on the trace data 142, etc.). In some embodiments, the prediction system 110 can generate the prediction data 168 using one or more models such as a machine learning model, a statistical model, and the like.

[0029] The client device 120, the manufacturing equipment 124, the sensor 126, the measurement device 128, the prediction server 112, the data store 140, the server machine 170, and the server machine 180 can be coupled to each other via the network 130 to generate the prediction data 168 and execute corrective measures.

[0030] In some embodiments, network 130 is a public network that provides client device 120 access to prediction server 112, data store 140, and other publicly available computing devices. In some embodiments, network 130 is a private network that provides client device 120 access to manufacturing equipment 124, sensors 126, measurement equipment 128, data store 140, and other privately available computing devices. Network 130 can include one or more wide area networks (WANs), local area networks (LANs), wired networks (e.g., Ethernet networks), wireless networks (e.g., 802.11 networks or Wi-Fi networks), cellular networks (e.g., Long Term Evolution (LTE) networks), routers, hubs, switches, server computers, cloud computing networks, and / or combinations thereof.

[0031] Client device 120 can include computing devices such as personal computers (PCs), laptops, mobile phones, smartphones, tablet computers, netbook computers, network-connected television sets (“smart TVs”), network-connected media players (e.g., Blu-ray players), set-top boxes, over-the-top (OTT) streaming devices, operator boxes, etc. Client device 120 can include corrective measure component 122. Corrective measure component 122 can receive user input (e.g., via a graphical user interface (GUI) displayed via client device 120).

[0032] In some embodiments, the corrective action component 122 obtains trace data 142 (e.g., current trace data 146) associated with the manufacturing equipment 124 (e.g., from a data store 140, etc.) and provides the trace data 142 (e.g., current trace data 146) associated with the manufacturing equipment 124 to the prediction system 110. In some embodiments, the corrective action component 122 stores the trace data 142 in the data store 140, and the prediction server 112 retrieves the trace data 142 from the data store 140. In some embodiments, the prediction server 112 can store the output of the trained machine learning model 190 (e.g., prediction data 168) in the data store 140, and the client device 120 can retrieve the output from the data store 140. In some embodiments, the corrective action component 122 receives an instruction for a corrective action from the prediction system 110 and implements the corrective action. The client device 120 can include an operating system that enables a user to perform one or more of generating, viewing, or editing data (e.g., instructions associated with the manufacturing equipment 124, corrective actions associated with the manufacturing equipment 124, etc.).

[0033] In some embodiments, the past performance data 152 corresponds to past characteristic data of products (e.g., manufactured using manufacturing parameters associated with the past trace data 144 and stored manufacturing parameters), and the prediction data 168 is associated with predicted characteristic data (e.g., predicted characteristic data of a product to be or made in the state recorded by the current trace data 146 and / or manufacturing parameters). In some embodiments, the prediction data 168 is predicted measurement data (e.g., virtual measurement data) of a product to be or made according to the state recorded as the current trace data 146 and / or manufacturing parameters. In some embodiments, the prediction data 168 is an indication of an anomaly (e.g., an abnormal product, an abnormal component, an abnormal manufacturing device 124, abnormal energy usage, etc.) and one or more causes of these anomalies. In some embodiments, the prediction data 168 is an indication of a change or drift over time in some component such as the manufacturing device 124, the sensor 126, the measurement device 128, etc. In some embodiments, the prediction data 168 is an indication of the end of life of components such as the manufacturing device 124, the sensor 126, the measurement device 128, etc.

[0034] Executing a manufacturing process that results in a defective product can be costly in terms of time, energy, products, components, manufacturing device 124, cost of defect identification and disposal of defective products, etc. By generating the prediction data 168 based on the trace data 142 and executing a corrective action based on the prediction data 168, the system 100 can have the technical advantage of avoiding the costs associated with the production, identification, and disposal of defective products.

[0035] Executing a manufacturing process that results in a failure of a component of manufacturing machine 124 can be costly in terms of downtime, product damage, equipment damage, orders for expedited delivery of replacement components, etc. By generating prediction data 168 based on trace data 142 (e.g., manufacturing parameters that are being used or should be used to manufacture a product) and executing corrective actions (e.g., predicted maintenance operations such as component replacement, processing, cleaning, etc.) based on the prediction data 168, system 100 can have the technical advantage of avoiding one or more costs such as unexpected component failures, unplanned downtime, loss of production rate, unexpected equipment failures, product waste, etc. By monitoring the performance of components (e.g., manufacturing machine 124, sensor 126, measurement device 128, etc.) over time, an indication of a deteriorating component can be provided.

[0036] Manufacturing parameters may be sub - optimal for producing a product, which can result in costly consequences such as increased consumption of resources (e.g., energy, coolant, gas, etc.), increased amount of time to produce a product, increased component failures, increased amount of defective products, etc. By generating prediction data 168 based on the characteristics of trace data 142 and executing corrective actions to update the manufacturing parameters (e.g., setting optimal manufacturing parameters) based on the prediction data 168, system 100 can have the technical advantage of using optimal manufacturing parameters (e.g., hardware parameters, process parameters, optimal design) to avoid the costly consequences of sub - optimal manufacturing parameters.

[0037] The corrective actions can be associated with one or more of computational process control (CPC), statistical process control (SPC) (e.g., SPC on electronic components for determining a process in control, SPC for predicting the useful life of a component, SPC for comparison with a 3σ graph, etc.), advanced process control (APC), model-based process control, preventive maintenance, design optimization, updating of manufacturing parameters or manufacturing policies for current or future manufacturing processes, feedback control, modification of machine learning, etc.

[0038] In some embodiments, the corrective actions include providing an alert (e.g., an alert to stop or not execute a manufacturing process if the predicted data 168 indicates a predicted anomaly such as an anomaly in a product, component, or manufacturing equipment 124). In some embodiments, the corrective actions include scheduling preventive maintenance. In some embodiments, the corrective actions include scheduling corrective maintenance. In some embodiments, the corrective actions include updating a process policy to fabricate subsequent substrates. In some embodiments, the corrective actions can be determined in consideration of an ongoing substrate processing process and can include updating the current process. In some embodiments, the corrective actions include correcting chamber drift associated with manufacturing equipment 124 (e.g., substrate processing equipment). In some embodiments, the corrective actions include correcting sensor drift of a sensor associated with manufacturing equipment 124 (e.g., substrate processing equipment). In some embodiments, the corrective actions include providing feedback control (e.g., modifying manufacturing parameters in response to the predicted data 168 indicating a predicted anomaly). In some embodiments, the corrective actions include providing machine learning (e.g., modifying one or more manufacturing parameters based on the predicted data 168). In some embodiments, the execution of the corrective actions includes updating one or more manufacturing parameters. In some embodiments, one or more of the corrective actions are executed in relation to components of the substrate processing equipment.

[0039] Manufacturing parameters can include hardware parameters (e.g., component replacement, use of specific components, replacement of processing chips, firmware updates, etc.) and / or process parameters (e.g., temperature, pressure, flow rate, current, voltage, gas flow, ramp rate, etc.). In some embodiments, the corrective action includes preventive maintenance operations (e.g., replacement, processing, cleaning of components of manufacturing equipment 124, etc.). In some embodiments, the corrective action includes performing design optimization (e.g., manufacturing parameters, manufacturing process, update of manufacturing equipment 124 for an optimized product, etc.). In some embodiments, the corrective action includes updating policies (e.g., setting manufacturing equipment 124 to idle mode, sleep mode, warm-up mode, etc.).

[0040] The prediction server 112, the server machine 170, and the server machine 180 can each include one or more computing devices such as a rack-mounted server, a router computer, a server computer, a personal computer, a mainframe computer, a laptop computer, a tablet computer, a desktop computer, a graphics processing unit (GPU), an application-specific integrated circuit (ASIC) accelerator (e.g., a tensor processing unit (TPU)).

[0041] The prediction server 112 can include a prediction component 114. In some embodiments, the prediction component 114 receives current trace data 146 (e.g., receives from the client device 120, retrieves from the data store 140), and based on the current data, can generate an output (e.g., prediction data 168) for executing a corrective action associated with the manufacturing equipment 124. In some embodiments, the prediction component 114 can use one or more trained models 190 to determine an output for executing a corrective action based on the current data. In some embodiments, the prediction component 114 predicts that an anomaly has occurred or will occur. In some embodiments, the prediction component 114 predicts that another event has occurred or may occur.

[0042] The model 190 can be a single model or multiple models. The models can be applied sequentially, multiple models can be used simultaneously, an appropriate model can be selected based on some metric, combinations of these techniques can be used, and so on. The model 190 (or the models included in the model 190) can be a machine learning model including a supervised, unsupervised, or semi-supervised machine learning model. The model 190 may not be a machine learning model, and can be, for example, a statistical model, a correlation model, etc.

[0043] In some embodiments, the first model 190 is used to generate a guard band (e.g., see FIGS. 2A - 2D), the second model 190 is used for guard band violation profiling (e.g., see FIGS. 3A - 3D), and the third model 190 is used to generate a dynamic region outside the range of the guard band (e.g., see FIGS. 4A - 4D).

[0044] In some embodiments, the data input to model 190 can include trace data 142 from a single sensor 126. In other embodiments, the data input to model 190 can include trace data 142 from multiple sensors 126 that indicate values of different characteristics. The data input can include manufacturing parameters. The features extracted from the trace data 142, the method of feature extraction, the corrective measures and / or predictive data 168 associated with the features, and the method of associating the corrective measures and / or predictive data 168 can all be adjusted for the data provided as input.

[0045] In some embodiments, the prediction component 114 receives the current trace data 146, provides the current trace data 146 as input to the model 190, and obtains an output indicating the predictive data 168 from the model 190. In some embodiments, the predictive data 168 indicates performance data 150 (e.g., measurement data, yield, etc.). In some embodiments, the predictive data 168 indicates a corrective action.

[0046] In some embodiments, the model 190 obtains the trace data 142 (e.g., data indicating a policy associated with the trace data 142, components of manufacturing equipment, etc.) as input and generates the predictive data 168 as output. The model 190 can be a single model or can include multiple models. The model 190 can determine which process should be executed based on the input data, or the user can indicate which analysis is appropriate for the input data, or a combination thereof.

[0047] The data store 140 and / or can be a memory (e.g., random access memory), a drive (e.g., hard drive, flash drive), a database system, or another type of component or device capable of storing data. The data store 140 and / or can include multiple storage components (e.g., multiple drives or multiple databases) that can span multiple computing devices (e.g., multiple server computers). The data store 140 can store trace data 142, performance data 150, and prediction data 168. The trace data 142 can include past trace data 144 and current trace data 146. The trace data can include sensor data time traces during the duration of a manufacturing process, associations of data with physical sensors, pre-processed data such as average and composite data, and data indicating sensor performance over time (e.g., in many manufacturing processes). The manufacturing parameters and performance data 150 can include similar characteristics. The past trace data 144, manufacturing parameters, and past performance data 152 can be past data (e.g., at least a portion of the training model 190). The current trace data 146 can be current data (e.g., at least a portion that should be input into the model 190 next after the past data) for which prediction data 168 (e.g., for performing corrective actions) should be generated.

[0048] In some embodiments, the prediction system 110 further includes a server machine 170 and a server machine 180. The server machine 170 includes a dataset generator 172 capable of generating a dataset (e.g., a set of data inputs and a set of target outputs) for training, validating, and / or testing one or more models such as machine learning models. The model 190 can include one or more machine learning models or can be other types of models such as statistical models. Models incorporating machine learning can be trained using input data and optionally target output data. Models not incorporating machine learning can also be trained. In some embodiments, the dataset generator 172 can split past data (e.g., past trace data 144 stored in the data store 140, manufacturing parameters, or past performance data 152) into a training set (e.g., 60 percent of the past data), a validation set (e.g., 20 percent of the past data), and a test set (e.g., 20 percent of the past data). In some embodiments, the prediction system 110 generates (e.g., via the prediction component 114) multiple sets of elements. For example, a first set of elements can correspond to a first set of types of sensor data (e.g., a first set of sensors, a first combination of values from the first set of sensors, a first pattern of values from the first set of sensors) corresponding to each of the datasets (e.g., the training set, the validation set, and the test set), and a second set of elements can correspond to a second set of types of sensor data (e.g., a second set of sensors different from the first set of sensors, a second combination of values different from the first combination, a second pattern different from the first pattern) corresponding to each of the datasets.

[0049] Server machine 180 includes a training engine 182, an authentication engine 184, a selection engine 185, and / or a test engine 186. The engines (e.g., training engine 182, authentication engine 184, selection engine 185, and test engine 186) can refer to hardware (e.g., circuits, dedicated logic, programmable logic, microcode, processing devices, etc.), software (e.g., instructions executed on a processing device, a general-purpose computer system, or a dedicated machine), firmware, microcode, or a combination thereof. The training engine 182 can be capable of training a machine learning model 190 or various machine learning models included within the model 190 using one or more sets of elements associated with a training set from the dataset generator 172. The training engine 182 can generate multiple trained machine learning models 190, and each trained machine learning model 190 corresponds to a distinct set of elements of the training set (e.g., sensor data from a distinct set of sensors). For example, a first trained machine learning model may be trained using all elements (e.g., X1~X5), a second trained machine learning model may be trained using a first subset of elements (e.g., X1, X2, X4), and a third trained machine learning model may be trained using a second subset of elements (e.g., X1, X3, X4, and X5), and the second subset of elements may partially overlap the first subset of elements. The dataset generator 172 receives the output of a trained machine learning model (e.g., a model trained to perform a first operation of trace data processing), collects that data into training, authentication, and test datasets, and can use these datasets to train a second machine learning model (e.g., a model to be trained to perform a second operation of trace data processing).

[0050] The authentication engine 184 may be able to authenticate the trained machine learning model 190 using the corresponding set of elements of the authentication set from the dataset generator 172. For example, a first trained machine learning model 190 trained using a first set of elements of the training set can be authenticated using a first set of elements of the authentication set. The authentication engine 184 can determine the accuracy of each of the trained machine learning models 190 based on the corresponding set of elements of the authentication set. The authentication engine 184 can discard the trained machine learning models 190 that have an accuracy that does not meet the threshold accuracy. In some embodiments, the selection engine 185 may be able to select one or more trained machine learning models 190 that have an accuracy that meets the threshold accuracy. In some embodiments, the selection engine 185 may be able to select the trained machine learning model 190 that has the highest accuracy among the trained machine learning models 190. In some embodiments, the authentication engine 184 and the selection engine 185 can repeat this process for each machine learning model included in the model 190.

[0051] In some embodiments, the authentication engine 184 performs verification and / or authentication (e.g., performs verification and validation (V&V)). Verification and authentication can be independent procedures that are both used to determine whether a product, service, or system (e.g., a machine learning model) meets requirements and specifications as well as intended purposes. Authentication can include the assurance that the machine learning model meets the needs of a customer or other identified stakeholders (e.g., including acceptance and compliance by external customers). Verification can include an assessment of whether the machine learning model complies with regulations, requirements, specifications, or imposed conditions (e.g., internal processes).

[0052] Test engine 186 can potentially test the trained machine learning models included within model 190 using corresponding sets of elements of the test set from data set generator 172. For example, a first trained machine learning model 190 trained using a first set of elements of the training set can be tested using a first set of elements of the test set. Test engine 186 can determine, based on the test set, the trained machine learning model included within model 190 that has the highest accuracy out of all of the trained machine learning models. Test engine 186 can repeat this process for all of the machine learning models included within model 190.

[0053] Model 190 can refer to model artifacts created by training engine 182 using a training set that includes data inputs and corresponding target outputs (correct responses for each training input). Patterns in the data set that map the data inputs to the target outputs (correct responses) can be discovered, and the machine learning model is provided with mappings that capture these patterns. The machine learning model can use one or more of support vector machine (SVM), radial basis function (RBF), clustering, supervised machine learning, semi-supervised machine learning, unsupervised machine learning, k-nearest neighbor algorithm (k-NN), linear regression, random forest, neural network (e.g., artificial neural network), etc.

[0054] The prediction component 114 can provide the current trace data 146 to the model 190, execute the machine learning model 190 trained on the input, and obtain one or more outputs. The prediction component 114 can determine (e.g., extract) the prediction data 168 from the output of the model 190, and determine (e.g., extract) the confidence data from the output indicating the level of confidence that the prediction data 168 is an accurate predictor of the process associated with the input data for the product to be made or to be made using the manufacturing equipment 124 with the current trace data 146 and / or manufacturing parameters. The prediction component 114 can also potentially determine a confidence range associated with a prediction event such as a remaining useful life (RUL) window including upper and lower limits. The prediction component 114 or the corrective action component 122 can use the confidence data to determine whether to trigger and / or when to trigger a corrective action associated with the manufacturing equipment 124 based on the prediction data 168.

[0055] The confidence data can include or indicate a level of confidence that the prediction data 168 is an accurate prediction for the product associated with at least a portion of the input data. In one example, the confidence level is inclusively a real number from 0 to 1, where 0 indicates no confidence that the prediction data 168 is an accurate prediction for the product processed according to the input data, and 1 indicates absolute confidence that the prediction data 168 accurately predicts the characteristics of the product processed according to the input data. In response to the confidence data indicating a confidence level below a threshold level for a given number of cases (e.g., percentage of cases, frequency of cases, frequency of occurrence, total number of cases, etc.), the prediction component 114 can retrain the model 190 (e.g., based on the current trace data 146, manufacturing parameters, current performance data 154, etc.).

[0056] For purposes of explanation and not limitation, aspects of the present disclosure describe using past data (e.g., past trace data 144, past performance data 152) to train one or more machine learning models 190 and inputting current data (e.g., current trace data 146) into one or more trained machine learning models 190 to determine prediction data 168. In other embodiments, a discovery model or rule-based model is used to determine prediction data 168 (e.g., without using a trained machine learning model). The prediction component 114 can monitor past trace data 144 and past performance data 152.

[0057] In some embodiments, the functions of the client device 120, the prediction server 112, the server machine 170, and the server machine 180 can be provided by a smaller number of machines. For example, in some embodiments, the server machines 170 and 180 can be integrated into a single machine, and in some other embodiments, the server machine 170, the server machine 180, and the prediction server 112 can be integrated into a single machine. In some embodiments, the client device 120 and the prediction server 112 can be integrated into a single machine.

[0058] Generally, the functions described in one embodiment as being performed by the client device 120, the prediction server 112, the server machine 170, and the server machine 180 can, where appropriate, also be performed by the prediction server 112 in other embodiments. Additionally, the functions attributed to specific components can also be performed by different or multiple components operating together. For example, in some embodiments, the prediction server 112 can determine corrective actions based on the prediction data 168. In another example, the client device 120 can determine the prediction data 168 based on the output from a trained machine learning model.

[0059] In addition, the functions of specific components can also be performed by different or multiple components that operate together. Through an appropriate application programming interface (API), access can be made to one or more of the prediction server 112, the server machine 170, or the server machine 180 as a service provided to other systems or devices.

[0060] In an embodiment, a "user" can be represented as a single individual. However, other embodiments of the present disclosure also include a "user" that is an entity managed by multiple users and / or automated sources. For example, a group of individual users integrated as a group of administrators can be regarded as a "user".

[0061] Embodiments of the present disclosure can be applied to data quality evaluation, feature enhancement, model evaluation, virtual metrology (VM), predictive maintenance (PdM), limit optimization, detection of anomalies or defects, classification of anomalies or defects, and the like.

[0062] Figures 2A - 2D, 3A - 3D, and 4A - 4D are flowcharts of methods 200A - D, 300A - D, and 400A - D associated with a guard band according to certain embodiments. In some embodiments, methods 200A - D, 300A - D, and 400A - D are executed by processing logic including hardware (e.g., circuits, dedicated logic, programmable logic, microcode, processing devices, etc.), software (e.g., instructions executed on a processing device, a general - purpose computer system, or a dedicated machine), firmware, microcode, or combinations thereof. In some embodiments, methods 200A - D, 300A - D, and 400A - D are executed at least in part by prediction system 110. In some embodiments, methods 200A - D, 300A - D, and 400A - D are executed at least in part by one or more of prediction system 110 (e.g., prediction server 112, prediction component 114), client device 120 (e.g., corrective action component), manufacturing equipment 124, and / or measurement device 128. In some embodiments, a non - transitory storage medium stores instructions that, when executed by a processing device (e.g., of prediction system 110, server machine 180, prediction server 112, etc.), cause the processing device to execute one or more of methods 200A - D, 300A - D, and 400A - D.

[0063] For simplicity of explanation, methods 200A - D, 300A - D, and 400A - D are illustrated and described as a series of acts. However, acts according to the present disclosure can be performed in various orders and / or concurrently with other acts not presented and described herein. Further, in some embodiments, not all of the acts shown are performed to implement methods 200A - D, 300A - D, and 400A - D according to the disclosed subject matter. Additionally, it will be understood and recognized by those skilled in the art that methods 200A - D, 300A - D, and 400A - D can alternatively be represented as a state diagram or events as a series of correlated states.

[0064] In some embodiments, FIGS. 2A-2D and FIGS. 3A-3D are directed to sequential data guardband analysis (e.g., sequential guardband analysis for defect detection and classification (FDC)) for improved defect diagnosis, defect classification, and prediction. In some embodiments, the present disclosure uses full trace analysis (FTA) of guardband defect diagnosis to provide improved analysis of sequential data streams (e.g., trace data) to provide improved detection, classification, and prediction, including reduced false warnings (e.g., false detections) and missed warnings (e.g., missed detections). Conventional sensor trace analysis can have false detections and missed detections (e.g., cannot adequately capture normal variations, cannot adapt to the model over time, cannot handle policy endpoints, cannot handle phase-shifted traces, cannot transfer the model to different regions, etc.), resulting in the conventional low adoption rate of guardbands. Conventional summary statistics can miss parts of the trace data (e.g., transient regions). The present disclosure can be applied to address false detections and missed detections and provide robustness over time and flexibility to different regions (see, e.g., FIGS. 8A-8B). The present disclosure can use guardbands to analyze both transient and steady-state regions and extract complex features (e.g., more complex than univariate analysis (UVA)).

[0065] Using the guardbands of the present disclosure, anomalies or defects occurring within the trace data can be detected, but there is no possibility of following typical anomaly shapes such as spikes or variations. Conventional guardbands can have output quality issues based on false detections (e.g., within transient events), stretching at the boundaries before and after transitions, variability between traces treated as anomalies, and all violations treated as equal without profiling or quantification. The guardbands of the present disclosure can address these drawbacks. The guardbands of the present disclosure can be provided alone or in combination with other analytical capabilities such as semi-automated feature extraction (SFE).

[0066] A guard band can be a channel defined for a data stream that is meant to identify regions of commonality (e.g., acceptable values) depending on the position within the data stream. The data stream is often the sensor values during some event such as a fabrication run. The data stream is often trace data which is a series of data values presented and arranged over time (e.g., where the x-axis is time). The ordering may not be based on time (e.g., it may be an indication of the number of things such as the number of products fabricated or the number of errors logged). The term “acceptable” depends on the application environment and purpose of the guard band analysis. For example, “acceptable” may mean “not abnormal” or “defect-free”. The guard band channel can have upper and lower limits. In some embodiments, these limits can be calculated along the duration of the data stream using statistical methods. For example, the channel can represent ±3σ of the values of multiple traces of a particular sensor, where the variance is calculated at each time value of the data stream. By using smoothing techniques (e.g., time window averaging), it is possible to make the channel more robust to noise and smoother.

[0067] For an abnormal trace, minor variations in the transient segment can be used with FTA guard banding. Using FTA guard banding, multiple traces or trace segments can be analyzed from a particular sensor over multiple runs. FTA guard banding can establish upper and lower limits (e.g., 3σ) that indicate the normal range or channel for sensor data over time. This solution identifies and profiles deviations from the guard band. The FTA capabilities can be used complementarily with SFE to provide a comprehensive analysis as input to a fingerprint library. The guard band can be associated with a single sensor (UVA) or across multiple sensors (multivariate analysis (MVA)). In the case of MVA, the sensor values (e.g., y-axis) can be a metric representing some combination of the sensors involved in the MVA.

[0068] In some embodiments, the guard band can include multiple guard bands (e.g., a warning guard band that is completely contained within an error guard band). In some embodiments, the first region is a normal region regardless of other parameters, the second region is a region where the classification of good or bad is based on the analysis of other parameters, and the third region is a defective region regardless of other parameters.

[0069] In some embodiments, the x-axis parameter of the guard band, or the parameter that defines the ordering manner of the data, is time (e.g., granularly associated with the read rate of the sensor). In some embodiments, the substrate number within a process tool can correspond to the substrate process.

[0070] In some embodiments, the guardband violation definition can be an anomaly, defect, alert, event trigger, and / or prediction. The determination and interpretation of a guardband violation may relate to the purpose of the guardband (e.g., detection of a defect or anomaly). A single data point outside the range of a guardband channel can indicate a defect. A probability distribution function can be used to define the area, duration, and magnitude of a portion of a trace that includes a violation region. These violation regions can be evaluated (e.g., violation region attribute values) to interpret whether there is a defect.

[0071] Figures 2A - 2D are flowcharts of methods 200A - D associated with generating a guardband according to certain embodiments. Figure 2A is a flowchart of method 200A associated with generating a guardband, Figure 2B is a flowchart of method 200B associated with using a guardband, Figure 2C is a flowchart of method 200C associated with generating a guardband via machine learning, and Figure 2D is a flowchart of method 200D associated with using a guardband via machine learning.

[0072] Referring to Figure 2A, in some embodiments, at block 202, the processing logic identifies trace data associated with the fabrication of substrates (e.g., good substrates) via a substrate processing system having characteristic values that meet a threshold. The trace data includes a separate set of sensor data over time for each substrate. In some embodiments, the trace data includes sensor data from a plurality of different sensors (e.g., different types of sensor data) for each substrate.

[0073] At block 204, the processing logic determines an acceptable type of variance (e.g., of a guardband) based on the trace data.

[0074] In some embodiments, the processing logic generates guard bands based on the trace data. The guard bands can include upper and lower limits for defect detection (e.g., anomaly detection). In some embodiments, to form the guard bands, all the averages of the trace data are generated, and then an offset from the average (e.g., 3σ) is used as the upper and lower limits. Conventionally, the upper and lower limits are equally spaced from the average of the trace data.

[0075] Since the trace data is for a good substrate, any variations from the guard bands formed by the trace data are of an acceptable dispersion type.

[0076] In some embodiments, block 204 includes, at block 210, processing logic that determines the variation between traces of the trace data (see FIGS. 5A - 5B). In some embodiments, the acceptable dispersion type includes the variation between traces of the time shift. The trace data can include sensor values (e.g., y - axis) with respect to time (e.g., x - axis). Since the time at which the recording of the sensor data starts is different, some of the sets of sensor data over time may be shifted (e.g., time shift on the x - axis). In some embodiments, the processing logic tracks the difference between traces and, when a normal time - shift dispersion is found, autonomously adjusts the guard - band dispersion horizontally. This prevents false detection when a policy step or trace shifts slightly. This increases robustness and reduces false detection.

[0077] In some embodiments, block 204 includes, at block 212, processing logic that determines an upper limit different from the lower limit based on the trace data (see FIG. 5C). In some embodiments, the acceptable dispersion types include a first acceptable dispersion type for forming the upper limit of the guard band and a second acceptable dispersion type for forming the lower limit of the guard band. The first acceptable dispersion type (e.g., quantity) may be different from the second acceptable dispersion type (e.g., quantity). The variations on each side of the guard band are calculated separately (e.g., during a signal transition) to reduce false detections.

[0078] In some embodiments, block 204 includes, at block 214, processing logic that determines a position-dependent dispersion of the trace data (see FIG. 5D). Parameters and weights (e.g., duration, level, region, concatenation of consecutive violations, etc.) and other guard band settings can be adjusted according to the position within the guard band (e.g., along the x-axis). Examples include: 1) adjusting along the x-axis corresponding to different operations along the process so that the guard band is more / less sensitive or adaptive, 2) adjusting the guard band according to the characteristics of the signal (e.g., more conservatively in regions where the guard band changes rapidly in the y direction and more aggressively in regions where the guard band value is relatively constant), 3) adjusting the guard band according to the warnings detected along the guard band (e.g., adjusting the distribution parameters so that future spike features are perceived as larger or smaller on that trace after the first spike is detected). This improves guard band performance and further enables the incorporation of subject matter expertise into guard band analysis.

[0079] In some embodiments, block 204 includes, at block 216, processing logic that determines trace segmenting of trace data (see FIG. 5E). In some embodiments, the acceptable dispersion type is further based on segmenting of portions of the trace data (e.g., values exceeding a threshold change, associated with a change in values within a threshold). For example, a feature (e.g., within a threshold) can trigger a wider guard band for the next segment of the guard band. In some embodiments, the acceptable dispersion type is further based on extraction of features from the trace data.

[0080] In some embodiments, block 204 includes, at block 218, processing logic that performs feature extraction of trace data (e.g., semi - automatic feature extraction (SFE)). For example, guard band parameters can be varied for different segments or different features. This improves guard band performance and overall analysis by enabling combination of different capabilities (e.g., guard band, feature extraction, and / or trace segmenting).

[0081] In some embodiments, the trace data includes sensor data from different types of sensors, and the acceptable dispersion type is via MVA. The MVA metric combines values of multiple sensors into a single metric (e.g., the first principal component in principal component analysis (PCA)). MVA techniques can be used to apply guard band parameters across multiple guard bands. Examples include 1) determining violations and violation distributions according to violation characteristics across two or more traces that occur (or do not occur) simultaneously, and / or 2) a motion state based on that signal for one or more other signals associated with a guard band for the signal. This improves guard band performance, further enables incorporation of subject matter expertise into guard band analysis, and enables handling of correlations across multiple signals.

[0082] In some embodiments, the features include one or more of persistent features (e.g., slopes and flat surfaces), user-defined features (e.g., connection of slopes and gradients), intermittent features such as slopes, flat surfaces, and FTA features, x offset, y offset, shape, length, strain correction, etc.

[0083] In block 205, the processing logic generates a guard band based on an acceptable dispersion type. For example, the guard band can enable time shift (e.g., within the x-axis), different upper and lower limits, multivariate (e.g., multivariable) analysis, segmentation of portions of trace data, extraction of features of trace data, etc. In some embodiments, the guard band is generated by training a machine learning model as shown in FIG. 2C.

[0084] In block 208, the processing logic executes a corrective action associated with the substrate processing system based on the guard band. Block 208 can include comparing additional trace data with the guard band to determine whether to execute a corrective action (see, e.g., FIG. 2B).

[0085] Referring to FIG. 2B, in some embodiments, in block 222, the processing logic identifies trace data associated with substrate fabrication through a substrate processing system (e.g., the same substrate processing system as in FIG. 2A). The trace data can be associated with substrates for which it is not known whether the characteristic data of the substrate meets a threshold (e.g., a good substrate) or does not meet the threshold (e.g., a bad substrate). The trace data includes a separate set of sensor data over time for each substrate. In some embodiments, the trace data includes sensor data from a plurality of different sensors (e.g., different types of sensor data) for each substrate.

[0086] At block 224, the processing logic compares the trace data to a guard band generated based on an acceptable distribution type (see, e.g., block 206 of FIG. 2A). The guard band can have upper and lower limits for data points over time.

[0087] At block 226, the processing logic determines that one or more data points of the trace data are not within the range of the guard band. The one or more data points can include at least one data point that exceeds the upper limit and / or at least one data point that is below the lower limit (e.g., these data points do not match the data points of a substrate having a characteristic value that meets a threshold).

[0088] At block 228, the processing logic executes a corrective action associated with the substrate processing system. In some embodiments, the corrective action includes providing an alert, interrupting the substrate processing equipment, inspecting the substrate, discarding the substrate, updating manufacturing parameters, etc. In some embodiments, the performance of the corrective action is specific to the type or number of data points outside the range of the guard band.

[0089] Referring to FIG. 2C, in some embodiments, at block 242, the processing logic identifies past trace data associated with the fabrication of a substrate via the substrate processing system. In some embodiments, the substrate (e.g., a good substrate) has a characteristic value that meets a threshold. Block 242 can be similar to block 202 of FIG. 2A. In some embodiments, the substrate (e.g., a bad substrate) has a characteristic value that does not meet the threshold.

[0090] In some embodiments, at block 244, the processing logic identifies past performance data associated with the past trace data. In some examples, the past performance data indicates whether the substrate is a good substrate or a bad substrate (e.g., all substrates have a characteristic value that meets a threshold, all substrates have a characteristic value that does not meet a threshold).

[0091] In block 246, the processing logic trains a machine learning model with a data input that includes past trace data (e.g., target output including past performance data), and generates a trained machine learning model that indicates a guard band associated with an acceptable dispersion type. By training the machine learning model, the guard band can be based on the acceptable dispersion type described in block 204 of FIG. 2A.

[0092] The trained machine learning model can be used to determine whether additional trace data satisfies the guard band (see, e.g., FIG. 2D).

[0093] Referring to FIG. 2D, in some embodiments, in block 262, the processing logic identifies trace data associated with substrate fabrication via a substrate processing system (e.g., the same substrate processing system as in FIG. 2C). The trace data can be associated with a substrate for which it is unknown whether the characteristic data of the substrate satisfies a threshold (e.g., a good substrate) or does not satisfy the threshold (e.g., a bad substrate). Block 262 can be similar to block 242 of FIG. 2B.

[0094] In block 264, the processing logic provides the trace data as an input to a trained machine learning model (e.g., the trained machine learning model of FIG. 246 in FIG. 2C) associated with a guard band generated based on an acceptable dispersion type.

[0095] In block 266, the processing logic receives an output indicating prediction data from the trained machine learning model.

[0096] In block 268, the processing logic determines, based on the prediction data, that one or more data points of the trace data are not within the range of the guard band of the trained machine learning model. The one or more data points may exceed the upper limit of the guard band or may be below the lower limit of the guard band.

[0097] At block 270, the processing logic executes corrective actions associated with the substrate processing system based on the prediction data. Block 270 can be similar to block 228 in FIG. 2B.

[0098] FIGS. 3A - 3D are flowcharts of methods associated with guard band violation profiling according to certain embodiments. FIG. 3A is a flowchart of method 300A associated with determining guard band violation shape characterization and classifying guard band violation data points, FIG. 3B is a flowchart of method 300B associated with classifying guard band violation data points based on guard band violation shape characterization, FIG. 3C is a flowchart of method 300C associated with training a machine learning model to classify guard band violation data points, and FIG. 3D is a flowchart of method 300D associated with using the trained machine learning model to classify guard band violation data points.

[0099] Referring to FIG. 3A, in some embodiments, at block 302, the processing logic identifies trace data associated with the fabrication of a substrate (e.g., a good substrate) via a substrate processing system having characteristic values that meet a threshold. Block 302 can be similar to block 202 in FIG. 2A.

[0100] At block 304, the processing logic identifies a guard band associated with the trace data. The guard band can be generated based on method 200A of FIG. 2A or method 200C of FIG. 2C.

[0101] At block 306, the processing logic determines guard band violation data points of the trace data based on the guard band. The guard band violation data points include data points of the trace data that exceed the upper limit of the guard band and / or data points of the trace data that fall below the lower limit of the guard band.

[0102] In some embodiments, the trace data is from different types of sensors, and the determination of guard band violation data points is performed via multivariate (e.g., multi-variable) analysis.

[0103] In some embodiments, the determination of guard band violation data points includes segmenting portions of the trace data associated with changes in values that exceed a threshold change. In some embodiments, the determination of guard band violation data points includes extracting features from the trace data.

[0104] At block 308, the processing logic determines a guard band violation shape characterization based on the guard band violation data points (see FIGS. 6A-6E).

[0105] In some embodiments, the guard band violation shape characterization is one or more weighted combinations of a guard band violation duration (e.g., the number of sequential guard band violation data points that are outside the range of the guard band limits), a guard band violation magnitude (e.g., how much the guard band violation data points exceed the upper limit or fall below the lower limit), a guard band violation region (e.g., the region between the line passing through the guard band violation data points and the guard band limits), a guard band violation position (e.g., the location of the guard band violation data points relative to the guard band limits), and / or a guard band violation intermittency (e.g., how frequently the guard band violation data points exceed the guard band upper limit and / or the guard band lower limit).

[0106] In some embodiments, determining guard band violation shape characterization includes linking consecutive violations into a single violation. By analyzing the behavior between consecutive violations, it is possible to better understand whether the violations are related. Typical analysis can include the time between violations, the level of return to normal state between violations, and the similarity of factors contributing to consecutive violations (e.g., x-direction shift). Linking consecutive violations can improve guard band performance by identifying system problems rather than guard band violations. This reduces the variability in guard band violation reports and enables the incorporation of subject matter expertise into guard band analysis.

[0107] Guard band violation shape characterization can be used to characterize any guard band feature in terms of parameters such as duration, level, area, etc. These parameters can be weighted to better capture a specific violation type.

[0108] In block 310, the processing logic executes a corrective action associated with the substrate processing system based on the guard band violation shape characterization. Block 310 can include classifying additional guard band violation data points of additional trace data based on the guard band violation shape characterization and determining whether to execute a corrective action (see, e.g., FIG. 3B).

[0109] Referring to FIG. 3B, in some embodiments, in block 322, the processing logic identifies trace data associated with substrate fabrication through a substrate processing system (e.g., the same substrate processing system as in FIG. 3A). The trace data can be associated with a substrate for which it is not known whether the substrate's characteristic data meets a threshold (e.g., a good substrate) or does not meet the threshold (e.g., a bad substrate). Block 322 can be similar to block 222 of FIG. 2B.

[0110] In block 324, the processing logic identifies a guard band associated with the trace data. Block 324 can be similar to block 304 in FIG. 3A.

[0111] In block 326, the processing logic determines guard band violation data points of the trace data based on the guard band. The guard band violation data points include data points of the trace data that exceed the upper limit of the guard band and / or data points of the trace data that fall below the lower limit of the guard band. Block 326 can be similar to block 306 in FIG. 3A.

[0112] In block 328, the processing logic identifies a guard band violation shape characterization. The guard band violation shape characterization can be determined by block 308 in FIG. 3A. The guard band violation shape characterization can indicate whether a particular type of guard band violation (e.g., shape, area, duration, size, position, intermittency, etc.) should be classified as an anomaly (e.g., execute a corrective action), or should not be classified as an anomaly (e.g., do not execute a corrective action).

[0113] In block 330, the processing logic determines the classification of the guard band violation data points based on the guard band violation shape characterization. In some embodiments, the classification indicates whether the guard band violation data points are anomalies. In some embodiments, the classification indicates the type of anomaly associated with the guard band violation data points. In some embodiments, the classification indicates the type of corrective action to be executed in relation to the guard band violation data points.

[0114] In block 332, the processing logic executes a corrective action associated with the substrate processing system based on the classification. The execution of the corrective action in block 332 can be similar to the execution of the corrective action in block 228 of FIG. 2B.

[0115] Referring to FIG. 3C, in some embodiments, at block 342, the processing logic identifies trace data associated with the fabrication of a substrate (e.g., a good substrate) via a substrate processing system having a characteristic value that meets a threshold. Block 302 can be similar to block 202 of FIG. 2A, block 242 of FIG. 2C, and / or block 302 of FIG. 3A.

[0116] At block 344, the processing logic identifies past performance data associated with the past trace data. In some examples, the past performance data indicates whether the substrate is a good substrate or a bad substrate (e.g., all substrates have characteristic values that meet the threshold, all substrates have characteristic values that do not meet the threshold). Block 344 can be similar to block 244 of FIG. 2C.

[0117] At block 346, the processing logic identifies a guard band associated with the trace data. The guard band can be generated based on method 200A of FIG. 2A or method 200C of FIG. 2C. Block 346 can be similar to block 304 of FIG. 3B.

[0118] At block 348, the processing logic determines past guard band violation data points of the past trace data based on the guard band. The past guard band violation data points include data points of the past trace data that exceed the upper limit of the guard band and / or data points of the trace data that fall below the lower limit of the guard band. Block 348 can be similar to block 306 of FIG. 3A.

[0119] In block 350, the processing logic trains a machine learning model with a data input that includes past guard band violation data points (e.g., a target output that includes past performance data), generates a trained machine learning model associated with guard band violation shape characterization, and classifies additional guard band violation data points. The trained machine learning model can be used by Figure 3D. In some embodiments, guard band violation data points and violation shape characterizations (e.g., shape summary statistical information) are provided to the trained machine learning model to classify the guard band violation data points. In some embodiments, the guard band violation shape characterization is determined based on the guard band violation data points (see, e.g., block 308 in Figure 3B).

[0120] Referring to Figure 3D, in some embodiments, in block 362, the processing logic identifies trace data associated with substrate fabrication via a substrate processing system (e.g., the same substrate processing system as in Figure 3C). The trace data can be associated with a substrate for which it is not known whether the substrate's characteristic data meets a threshold (e.g., a good substrate) or does not meet the threshold (e.g., a bad substrate). Block 362 can be similar to block 342 in Figure 3B.

[0121] In block 364, the processing logic identifies a guard band associated with the trace data. Block 364 can be similar to block 304 in Figure 3A, block 324 in Figure 3B, and / or block 344 in Figure 3C.

[0122] In block 366, the processing logic determines guard band violation data points of the trace data based on the guard band. The guard band violation data points include data points of the trace data that exceed the upper limit of the guard band and / or data points of the trace data that are below the lower limit of the guard band. Block 366 can be similar to block 306 in Figure 3A, block 326 in Figure 3B, and / or block 346 in Figure 3C.

[0123] At block 368, the processing logic provides guard band violation data points as an input to a trained machine learning model associated with guard band violation shape characterization (e.g., trained via block 350 of FIG. 3C). In some embodiments, the guard band violation data points and guard band violation shape characterization (e.g., shape summary statistics) are provided as inputs to the trained machine learning model to classify the guard band violation data points. In some embodiments, the guard band violation shape characterization is determined based on the guard band violation data points (see, e.g., block 308 of FIG. 3B).

[0124] At block 370, the processing logic receives an output indicative of prediction data from the trained machine learning model.

[0125] At block 372, the processing logic determines a classification of one or more data points of the trace data based on the prediction data. The classification at block 372 can be similar to the classification at block 330 of FIG. 3B.

[0126] At block 384, the processing logic executes a corrective action associated with the substrate processing system based on the classification. Block 384 can be similar to block 332 of FIG. 3B.

[0127] Figures 4A - 4D are flowcharts of methods associated with a dynamically acceptable region outside the guard band limit (e.g., time - dependent variations in trace data) according to certain embodiments. Figure 4A is a flowchart of method 400A associated with determining an acceptable region outside the guard band limit, Figure 4B is a flowchart of method 400B associated with using and optionally adjusting an acceptable region outside the guard band limit, Figure 4C is a flowchart of method 400C associated with training a machine - learning model to determine an acceptable region outside the guard band limit, and Figure 4D is a flowchart of method 400D associated with using a trained machine - learning model to use and optionally adjust an acceptable region outside the guard band limit.

[0128] The trace data of a good wafer may change over time due to acceptable drifts, variations, noise, spikes, etc.

[0129] Referring to Figure 4A, in some embodiments, at block 402, the processing logic identifies trace data associated with the fabrication of a substrate (e.g., a good substrate) via a substrate processing system having characteristic values that meet a threshold. Block 302 can be similar to block 202 of Figure 2A and / or block 302 of Figure 3A.

[0130] At block 404, the processing logic determines a dynamically acceptable region outside the guard band limit based on the trace data.

[0131] In some embodiments, the processing logic determines the upper and lower bounds of the guard band based on the trace data (e.g., via block 206 of Figure 2A, block 246 of Figure 2C, 3σ from the average of the trace data, etc.). The region between the upper and lower bounds of the guard band is a safe region (e.g., a green region), and data points within this region are considered healthy.

[0132] The processing logic determines an acceptable region outside the guard band limits. The acceptable region can be an alarm region (e.g., a yellow region), where the data points are still normal and are used to track the motion state of the substrate processing system. An abnormal region (e.g., a red region) is outside the range of the acceptable region, and data points within the abnormal region are considered to be positive (e.g., abnormal).

[0133] The acceptable region can be determined via user input (e.g., 1σ outside the guard band limits, 4 angstroms of sensor data variation, 4% of sensor data variation, etc.). The acceptable region can vary over time (e.g., a dynamic acceptable region). In some embodiments, for a predetermined amount of time (e.g., a predetermined run amount), a predetermined amount of variation of the sensor value from the guard band limits can be made acceptable. For example, for 10 runs, a 4 angstrom or 4% change in the sensor value from the guard band limits can be made acceptable.

[0134] The dynamic acceptable region and / or the guard band limits can change over time. For example, for a predetermined run amount (e.g., 10 runs), the acceptable region and / or the guard band limits can be adjusted by a predetermined amount (e.g., increased by 4 angstroms, widened by 4%, etc.). Since the dynamic acceptable region and / or the guard band limits change over time, the new acceptable region can be updated.

[0135] In block 406, the processing logic executes a corrective action associated with the substrate processing equipment based on the dynamic acceptable region outside the guard band limits. The execution of the corrective action can be based on additional trace data being outside the range of the acceptable region (see FIG. 4B).

[0136] Referring to FIG. 4B, in some embodiments, at block 422, the processing logic identifies trace data associated with substrate fabrication via a substrate processing system (e.g., the same substrate processing system as in FIG. 4A). The trace data can be associated with substrates for which it is unknown whether the substrate characteristic data meets a threshold (e.g., a good substrate) or does not meet the threshold (e.g., a bad substrate). Block 422 can be similar to block 222 of FIG. 2B and / or block 322 of FIG. 3B.

[0137] At block 424, the processing logic compares the trace data to a dynamically acceptable region outside the guard band limits. The dynamically acceptable region can be determined by block 404 of FIG. 4A.

[0138] At block 426, in response to one or more data points of the trace data being within the range of the dynamically acceptable data, the processing logic updates the dynamically acceptable region outside the guard band limits based on the trace data.

[0139] At block 428, in response to one or more data points of the trace data being outside the range of the dynamically acceptable data, the processing logic executes a corrective action associated with the substrate processing equipment. The execution of the corrective action at block 428 can be similar to the execution of the corrective action at block 228 of FIG. 2B and / or block 332 of FIG. 3B.

[0140] Referring to FIG. 4C, in some embodiments, at block 442, the processing logic identifies trace data associated with the fabrication of a substrate (e.g., a good substrate) via a substrate processing system having characteristic values that meet a threshold. Block 302 can be similar to block 202 of FIG. 2A, block 242 of FIG. 2C, block 302 of FIG. 3A, block 342 of FIG. 3C, and / or block 402 of FIG. 4A.

[0141] In block 444, the processing logic identifies past performance data associated with past trace data. In some examples, the past performance data indicates whether the substrate is a good substrate or a bad substrate (e.g., all substrates have characteristic values that meet a threshold, all substrates have characteristic values that do not meet a threshold). Block 444 can be similar to block 244 of FIG. 2C and / or block 344 of FIG. 3C.

[0142] In block 446, the processing logic trains a machine learning model with a data input that includes past trace data (e.g., a target output that includes past performance data), and generates a trained machine learning model that indicates a dynamic acceptable region outside the guard band limits. The trained machine learning model can be used by FIG. 4D.

[0143] Referring to FIG. 4D, in some embodiments, in block 462, the processing logic identifies trace data associated with substrate fabrication via a substrate processing system (e.g., the same substrate processing system as FIG. 4C). The trace data can be associated with a substrate for which it is not known whether the characteristic data of the substrate meets a threshold (e.g., a good substrate) or does not meet a threshold (e.g., a bad substrate). Block 362 can be similar to block 442 of FIG. 4B.

[0144] In block 464, the processing logic provides the trace data as an input to a trained machine learning model associated with a dynamic acceptable region outside the guard band limits (e.g., the trained machine learning model of FIG. 446 of FIG. 4C).

[0145] In block 466, the processing logic receives an output indicating prediction data from the trained machine learning model.

[0146] At block 468, in response to determining, based on the prediction data, that one or more data points of the trace data are within the range of the acceptable region, the processing logic updates a dynamic acceptable region outside the range of the guard band limit. Block 468 can be similar to block 426 in FIG. 4B.

[0147] At block 470, in response to determining, based on the prediction data, that one or more data points of the trace data are outside the range of the acceptable region, the processing logic executes a corrective action associated with the substrate processing system. Block 470 can be similar to block 428 in FIG. 4B.

[0148] FIGS. 5A - 5E show graphs 500A - E of an acceptable dispersion type (see, e.g., FIGS. 2A - 2D) according to a particular embodiment. FIGS. 5A - 5B show graphs 500A - B of the variation between traces. FIG. 5C shows a graph 500C with an upper limit different from the lower limit. FIG. 5D shows a graph 500D of position - dependent dispersion. FIG. 5E shows a graph 500E of trace segmentation.

[0149] Referring to FIG. 5A, graph 500A shows trace data 502A - B. Trace data 502A can be sensor data (e.g., from one or more sensors) for the fabrication of a first substrate (e.g., one or more first substrates), and trace data 502B can be sensor data (e.g., from the same one or more sensors) for a second substrate (e.g., one or more second substrates). The complement of trace data 502B can be performed at an earlier stage in the substrate fabrication operation than the complement of trace data 502A, thereby causing a shift in the x - direction (e.g., the x - axis is time and the y - axis is the sensor value, and trace data 502A rises steeply earlier than trace data 502B). This is called the variation (e.g., dispersion) between traces.

[0150] Referring to FIG. 5B, graph 500B shows trace data over time associated with the fabrication of many substrates. Guard band 504A, which does not account for the variation between traces, causes many false detections (e.g., many guard band violation data points that do not correspond to defective substrates). In some examples, by obtaining the average of the trace data, line 506 is calculated, and by obtaining 3σ from line 506, guard band 504A is created, such that guard band 504A is equally distant from line 506 over time. The variation between traces causes many false detections in guard band 504A, and simply expanding guard band 504A should create many detection misses (e.g., data points deviating in the y direction should not be captured if guard band 504A is expanded).

[0151] By method 200A of FIG. 2A or method 200C of FIG. 2C, guard band 504B that accounts for the variation between traces is generated. Using trace data from good substrates, the acceptable types of variation are determined. For example, an initial guard band 504A can be made from trace data from good substrates, and then since all substrates have characteristic values that meet the threshold, guard band 504A is expanded in the x direction (e.g., horizontally expanded) within the area for forming guard band 504B to correspond to the variation between traces.

[0152] Referring to FIG. 5C, graph 500C shows a graph 500C with an upper limit different from the lower limit.

[0153] As discussed in FIG. 5B, guard band 504A may have an upper and lower limit (e.g., 3σ) equally spaced from a line 506 passing through the average of the trace data. Trace data from good wafers can have different amounts of variance above and below line 506. By only equally separating the upper and lower limits of guard band 504A, many false detections can occur.

[0154] By method 200A of FIG. 2A or method 200C of FIG. 2C, a guard band 504B that takes into account different upper and lower limits is generated. For example, as shown in FIG. 5C, at the beginning of the transition, an upper dispersion 508A larger than the lower dispersion 508B may occur, and at the end of the transition, a lower dispersion 508D larger than the upper dispersion 508C may occur.

[0155] Referring to FIG. 5D, graph 500D shows a graph of position-dependent dispersion 500D.

[0156] As discussed in FIG. 5B, guard band 504A may have upper and lower limits (e.g., 3σ) at the same distance from line 506 passing through the average of the trace data over time. Trace data from a good wafer can have different amounts of dispersion in different time portions. By having a guard band 504A of the same size over time, many false detections may occur.

[0157] By method 200A of FIG. 2A or method 200C of FIG. 2C, a guard band 504B that takes into account different amounts of dispersion 510 over time is generated. For example, as shown in FIG. 5D, within the flat region, a lower dispersion 510A may occur (e.g., the distance between the upper and lower limits of guard band 504B for a good substrate is smaller), and within the transition region, a higher dispersion 510B may occur (e.g., the distance between the upper and lower limits of guard band 504B for a good substrate is larger).

[0158] Referring to FIG. 5E, graph 500E shows a graph of trace segmentation 500E. As discussed in FIG. 5B, guard band 504A may have upper and lower limits (e.g., 3σ) at the same distance from line 506 passing through the average of the trace data over time. In some embodiments, the change in the trace data is greater than a threshold amount (see, e.g., region 512 in FIG. 5E).

[0159] When two boundaries are close to each other, the central position can be used. If one of the boundaries is close to a boundary with a change greater than the threshold amount, the segment boundary can be maintained.

[0160] Two segments of the trace data indicating a sharp change in the original boundary cannot be removed from the trace data. Based on the knowledge of segmentation regarding the sharp change of the segmentation point in area 512, the parameters of the guard band can be adjusted.

[0161] The sharp change in area 512 may be one or more of the following.

[0162] 1) Gradient change of the segment of the trace data from negative to positive or from positive to negative (e.g., slope(right)*slope(left)<0),

[0163] 2) The gradient of the right segment and / or the left segment exceeds the threshold gradient (e.g., abs(slope(right))>0.1 or abs(slope(left))>0.1 (normal value)), and / or

[0164] 3) The average variance (e.g., standard deviation) of two connected segments is a threshold amount greater than the average variance (e.g., standard deviation) of the next two segments (e.g., mean(std[right,left])>2*mean(std[right_2,left_2]).

[0165] If two adjacent segments of trace data have 1), 2), and 3) above, the segment boundary (e.g., the data point within region 512) can be regarded as a fixed boundary. In some embodiments, when region 512 is a segment boundary, instead of removing the data points within 512 from the generation of guard band 504B, guard band 504B is generated for those points within region 512. In some embodiments, when region 512 is a segment boundary, the data points within region 512 are the limits of guard band 504B having a smaller acceptable variance.

[0166] Figures 6A - 6E illustrate guard band violation profiling according to a particular embodiment (see, e.g., Figures 3A - 3D).

[0167] Referring to Figure 6A, trace data is shown in each of graphs 602A - Z (e.g., block 302 of Figure 3A, block 344 of Figure 3C). Each graph 602A - Z can show trace data for the fabrication of a different substrate, and each substrate is a good substrate (e.g., having characteristic values that meet a threshold). Each graph 602A - Z also shows a guard band, and each graph 602A - Z has some sets of data points that are outside the range of the guard band limits. These sets of data points are referred to as sets of guard band violation data points 604A - Z (e.g., sets of data points associated with guard band violations). Since the trace data is for good substrates, the sets of guard band violation data points 604A - Z violate the guard band but do not indicate a bad substrate (e.g., these are false detections).

[0168] For each set of guard band violation data points 604 of the trace data, a parameter 606 is extracted. The parameter 606 can include an area, a duration, a magnitude, etc. In some examples, the area can be the area between the guard band limit and a portion of a line passing through trace data outside the guard band limit. In some examples, the duration can be the amount of time during which successive guard band violation data points are outside the guard band limit. The magnitude can be the magnitude (e.g., the value of y) of the difference (e.g., in the y direction) between the guard band violation data point and the guard band limit. In some examples, each guard band violation data point can have a corresponding parameter input.

[0169] Using the parameter 606 and the joint probability density function of the parameter 606, a graph 608 can be formed. The graph 608 can be generated by fitting a multivariate Gaussian distribution (e.g., finding the joint probability density function of three variables). The set of guard band violation data points 604 can form a guard band violation shape characterization 610 (e.g., a circle on the graph 608) that surrounds a set of guard band violation data points 604 (e.g., good guard band violation data points 614A).

[0170] Using the graph 608 that includes the guard band violation shape characterization 610, a graph 612 can be generated. The graph 12 displays the guard band violation shape characterization 610 that separates the good guard band violation data points 614A corresponding to good substrates from the bad guard band violation data points 614B corresponding to bad substrates.

[0171] In response to the set of guard band violation data points 604 being for a good substrate, guard band violation shape characterization 610 surrounds (e.g., encloses) the data points of the good substrate within graph 608. Using guard band violation shape characterization 610, it can be determined whether a future set of guard band violation data points is a good guard band violation data point 614A corresponding to a good substrate (e.g., within the range of guard band violation shape characterization 610 on graph 608, below guard band violation shape characterization 610 on graph 612), or a good guard band violation data point 614A corresponding to a bad substrate (e.g., outside the range of guard band violation shape characterization 610 on graph 608, above guard band violation shape characterization 610 on graph 612).

[0172] In some embodiments, the set of guard band violation data points 604 used for parameter 606, graph 608, and graph 612 corresponds to a bad substrate (e.g., a substrate having characteristic values that do not meet a threshold). At this time, guard band violation shape characterization 610 surrounds (e.g., encloses) the data points of the bad substrate within graph 608. Using guard band violation shape characterization 610, it can be determined whether a future substrate is bad (e.g., within the range of guard band violation shape characterization 610 on graph 608, below guard band violation shape characterization 610 on graph 612), or not bad (e.g., outside the range of guard band violation shape characterization 610 on graph 608, above guard band violation shape characterization 610 on graph 612).

[0173] In response to the set of guard band violation data points 604 being for a particular type of substrate (e.g., a particular type of defective substrate having a particular characteristic value that does not meet a threshold), the guard band violation shape characterization 610 surrounds (e.g., encloses) the data points of the particular type of substrate. The guard band violation shape characterization 610 is used to determine whether a future substrate is of the particular type of substrate (e.g., below the guard band violation shape characterization 610 on graph 612 that is within the range of the guard band violation shape characterization 610 on graph 608), or not of the particular type of substrate (e.g., above the guard band violation shape characterization 610 on graph 612 that is outside the range of the guard band violation shape characterization 610 on graph 608).

[0174] FIG. 6B shows a graph 620 for guard band violation profiling according to a particular embodiment. The graph 620 includes a band average 622 (e.g., average trace data), guard band limits 624, an offset 626 between the guard band limits 624 and the band average 622, and guard band violation data points 628. The graph 620 has a gap 638 between a first case guard band violation data point 628A and a second case guard band violation data point 628B.

[0175] The violation duration 630 of the guard band violation data points 628 is the distance between a first guard band violation data point 628A outside the range of the guard band limits 624 and the last guard band violation data point 628B outside the range of the guard band limits 624 (e.g., the duration of the violation in the x direction).

[0176] The magnitude of the violation 632 (e.g., the maximum violation, the peak of the violation) is the distance between the guard band limits 624 and the guard band violation data point 628 (e.g., the guard band violation data point 628 that is farthest from the guard band limits 624 in the y direction).

[0177] The violation region 634 is the region between the line passing through the guard band violation data points and the guard band limit 624 (e.g., the average value of the violations).

[0178] The violation position 636 can be the position of the maximum violation magnitude 632 (e.g., the x value, time value at which there is a violation).

[0179] The intermittency of the violations can include the combination or splitting of two or more violations.

[0180] The guard band violation shape characterization 610 can be further based on one or more of the violation duration 630, the violation magnitude 632, the violation region 634, and / or the violation position 636. Guard band violation data points 628 corresponding to a violation duration 630, violation magnitude 632, violation region 634, and / or violation position 636 that meet a threshold can be ignored. Guard band violation data points 628 corresponding to a violation duration 630, violation magnitude 632, violation region 634, and / or violation position 636 that meet a threshold can correspond to the type of substrate (e.g., a good substrate, a bad substrate, the type of bad substrate, etc.).

[0181] FIG. 6C shows a graph 640 for guard band violation profiling according to a particular embodiment. The graph 640 includes the band average 622 (e.g., average trace data), the guard band limit 624, an example of a guard band violation data point 628, a gap 638 between two examples of guard band violation data points 628, and the violation magnitude 632 between the band average 622 and the last guard band violation data point 628 of the first example of guard band violation data points 628. A line 642 is positioned between the last guard band violation data point of the first example of guard band violation data points 628C and the data point of the band average 622 corresponding to the first guard band violation data point of the second example of guard band violation data points 628D.

[0182] The guard band violation shape characterization 610 can classify substrate or trace data as good, bad, or a type of defect based on the number of violations. In response to the length of the line 642 that meets the threshold (e.g., being large enough), the guard band violation data points 628C in the first case and the guard band violation data points 628D in the second case are considered a single violation; otherwise, the guard band violation data points 628C in the first case and the guard band violation data points 628D in the second case are considered separate violations.

[0183] FIG. 6D shows a flowchart of a method 660 for guard band violation profiling according to a particular embodiment. The classification of guard band violation data points can be performed via multivariate (e.g., multi-variable) analysis (e.g., based on sensor data from different types of sensors).

[0184] In some embodiments, the trace data from sensors 662A - N is compared separately with the corresponding guard bands 664A - N. The guard band violation data points from the comparison of the trace data from sensor 662 compared with guard band 664 are compared with a threshold 668 via a violation probability density function (pdf) 666 to process and provide test trace data 670.

[0185] The test trace data provides a score 672 for each sensor 662. Based on the score 672, a score ranking 674 is performed. Based on the score ranking 674, a sensor importance ranking 676 is performed. Based on the sensor importance ranking 676, a composite score 677 is generated. The composite score 677 is compared with a composite threshold 678 to provide a defect detection result 679 (e.g., the classification of guard band violation data points as for a good substrate or a bad substrate, an anomaly detection result).

[0186] FIG. 6E shows a flowchart of method 680 for guardband violation profiling according to a particular embodiment. Guardband violation shape characterization can be performed based on guardband violation data points and segmented feature extraction.

[0187] In block 682, trace data is identified. This can be similar to block 302 of FIG. 3A and / or block 342 of FIG. 3C for generating a guardband. This can be similar to block 322 of FIG. 3B and / or block 362 of FIG. 3D for using a guardband.

[0188] In block 684, full trace analysis is performed on the trace data from block 682. For example, the area, duration, size, etc. can be determined (see, e.g., FIGS. 6A - 6C).

[0189] In block 686, a guardband model is generated based on the full trace analysis of the trace data. This can be similar to block 206 of FIG. 2A and / or block 246 of FIG. 2C.

[0190] In block 688, by comparing the trace data with a guardband model (e.g., a guardband), abnormal features (e.g., guardband violation data points) are determined. This can be similar to block 306 of FIG. 3A, block 326 of FIG. 3B, block 346 of FIG. 3C for training a model, and / or block 366 of FIG. 3D. The abnormal features (e.g., guardband violation data points) can be persistent or intermittent.

[0191] In block 690, segmentation feature extraction is performed on the trace data of block 682. This can be made similar to that in FIG. 5E (for example, the segment boundaries such as the data points within region 512 in FIG. 5E are regarded as fixed boundaries). In some embodiments, a guard band model can create the abnormal features of block 688 and combine them with the features identified in block 690 (for example, the persistent features of block 692 and the intermittent features of block 692).

[0192] In block 692, based on the segmentation feature extraction of block 690, the persistent features of the trace data are determined. The persistent features can be the features of the trace data that satisfy a threshold amount of occurrence.

[0193] In block 694, the feature parameter correlation of the persistent features of the trace data is performed (for example, based on the subject matter expertise 699 such as user input). The feature parameter correlation can be performed based on parameters such as size, position, area, duration, etc. (see FIG. 6B). In some embodiments, the guard band violation shape characterization can classify the guard band violation data points corresponding to the persistent features as those associated with a good substrate.

[0194] In block 696, based on the segmentation feature extraction of block 690, the intermittent features of the trace data are determined. The intermittent features can be the features of the trace data that do not satisfy a threshold amount of occurrence.

[0195] In block 698, based on the intermittent features from block 696, the abnormal features from block 688, and / or the subject matter expertise 699, a feature presence correlation is performed. In some embodiments, the guard band violation shape characterization can classify the guard band violation data points corresponding to the intermittent features as those associated with a defective substrate.

[0196] Figures 7A - 7F show a dynamically tolerable region outside the guard band limit according to a particular embodiment.

[0197] Figure 7A shows a flowchart of a method 700 associated with a dynamic region outside the guard band limit.

[0198] In block 702, the processing logic identifies baseline trace data. This can be similar to block 302 of FIG. 3A and / or block 342 of FIG. 3C for generating a guard band. This can be similar to block 322 of FIG. 3B and / or block 362 of FIG. 3D for using the guard band.

[0199] In block 704, the processing logic identifies a guard band based on the trace data. The guard band can be generated by block 206 of FIG. 2A and / or block 246 of FIG. 2C.

[0200] In block 706, the processing logic performs a multivariable (e.g., multivariate) analysis (MVA) distribution based on the guard band and the baseline trace data.

[0201] In block 708, the processing logic identifies a probability on the baseline trace data. This probability can be the probability that a data point is in a safe region (e.g., region 736 of graph 734 or graph 742 in FIG. 7B), a warning region (e.g., region 738 of graph 734 or graph 742 in FIG. 7B), or an abnormal region (e.g., region 740 of graph 734 or graph 742 in FIG. 7B).

[0202] In block 710, the processing logic generates an internal threshold. The internal threshold can be a line that separates the warning region (e.g., region 738 of graph 734 or graph 742 in FIG. 7B) from the abnormal region (e.g., region 740 of graph 734 or graph 742 in FIG. 7B).

[0203] In block 712, the processing logic generates extended trace data. The extended trace data can be simulated trace data formed by adjusting the baseline trace data of block 702 by one or more of blocks 714 - 720.

[0204] In block 714, the extended trace data can include trace data with minor drift. The drift can include increasing the sensor values of the baseline trace data of block 702 in the y - direction.

[0205] In block 716, the extended trace data can include trace data with minor repetition. One or more portions of the baseline trace data of block 702 can be repeated over time (e.g., in the x - direction).

[0206] In block 718, the extended trace data can include trace data with minor noise (e.g., and / or fluctuations). The baseline trace data of block 702 can be adjusted (e.g., increased and decreased) in the y - direction to mimic the noise.

[0207] In block 720, the extended trace data can include trace data with minor spikes. The baseline trace data of block 702 can include peaks and valleys to mimic the minor spikes.

[0208] In block 722, the processing logic identifies probabilities on the baseline trace data and the extended trace data. Block 722 can be similar to block 708.

[0209] In block 724, the processing logic generates an external threshold. Block 724 can be similar to block 710.

[0210] FIG. 7B shows the dynamic region outside the guard band limit. Graph 730 shows the baseline trace data (e.g., of block 702), and graph 732 shows the extended trace data (e.g., of block 712).

[0211] Graph 734 shows region 736 (e.g., an acceptable region, a green region), region 738 (e.g., a warning region, a yellow region), and region 740 (e.g., an abnormal region). Graph 734 can be formed based on trace data from a good substrate. All of the data points on graph 734 can be located within region 736 or region 738.

[0212] Graph 742 shows new trace data for a substrate for which it is to be determined whether the substrate is good or bad. Graph 742 has trace data within region 736, region 738, and region 740. The data points within region 740 correspond to a bad substrate. Region 738 (e.g., the yellow region, the warning region) should be recalculated based on the data points within region 738.

[0213] FIG. 7C shows a flowchart of method 744 associated with the dynamic region outside the guard band limit.

[0214] At block 746, the processing logic identifies a training set of trace data. The training set can be the previous trace data used to fabricate a substrate by a substrate processing device.

[0215] At block 748, the processing logic identifies new trace data. In some embodiments, the new trace data includes new sensor data associated with fabricating a new substrate by the same substrate processing device as block 746 or a different substrate processing device. In some embodiments, the new trace data includes simulated trace data made based on the trace date of block 746 and includes one or more of drift, noise, spikes, variations, etc.

[0216] In block 750, the processing logic ranks the new trace data from block 748. The new trace data can be ranked based on representing (e.g., being proximate to) the trace data of block 746. The new trace data can be ranked based on proximity to each other (e.g., to remove anomalies). The data points of the new trace data can be ranked as being good (e.g., green region), acceptable, used to adjust a guard band (e.g., yellow region), or anomalous (e.g., red region). For example, the data points of the new trace data can be within the green region (e.g., without input guard band adjustment), within the acceptable yellow region (e.g., no anomalies but with input for GB adjustment), or within the anomalous red region (e.g., anomalous but without input for guard band adjustment).

[0217] In block 752, the processing logic selects at least a portion of the new trace data from block 748 (e.g., based on the ranking from block 750). The processing logic can select the most highly ranked trace data based on the ranking of block 750.

[0218] In block 754, the processing logic updates (e.g., see FIG. 7E) or retrains (e.g., see FIG. 7F) the old guard band based on the training set and / or the trace data selected from block 752. The processing logic can trigger a guard band update when certain criteria are met (e.g., the trace data has noise, there is no drift in the trace data, etc.).

[0219] In block 756, the processing logic identifies the new guard band updated or retrained from block 754.

[0220] FIG. 7D shows a dynamic region outside the guard band limit. Graph 760 shows new trace data (e.g., from block 748 of FIG. 7C), and graph 762 shows a trace ranking of the trace data of graph 760 (see, e.g., block 750 of FIG. 7C).

[0221] Graph 764 shows a selected trace (e.g., from block 752 of FIG. 7C), and graph 766 shows an adapted guard band (e.g., the new guard band of block 756 of FIG. 7C) updated from the selected trace (e.g., the most beneficial trace) from graph 764.

[0222] FIG. 7E shows a dynamic region outside the guard band limit for noise. Block diagram 770A shows that process data (e.g., trace data) for generating a guard band is being accumulated. In response to the processing logic detecting noise (e.g., a periodic change), the processing logic updates the guard band with all past data.

[0223] Graph 772A shows initial trace data (e.g., the first 60 traces), and graph 774A shows an initial guard band for the trace data of graph 772A.

[0224] Graph 776A shows trace data (e.g., the first 120 traces), and graph 778A shows the next guard band for the trace data of graph 776A. As shown in graph 778A, the guard band widens over time to accommodate an acceptable increase in noise.

[0225] Figure 7F shows the dynamic region outside the range of the guard band limit against drift. Block diagram 770A shows that older past data (e.g., previous trace data) is forgotten and only a certain amount (e.g., zero or more) of newer past data is combined with new process data (e.g., current trace data) to generate a guard band. In response to the processing logic detecting drift (e.g., the temporal change in the y - direction of sensor values), the processing logic triggers a forgetting mechanism and updates the guard band based only on the most recent trace data.

[0226] Graph 772B shows the initial trace data (e.g., the first 50 traces), and graph 774B shows the initial guard band for the trace data of graph 772B.

[0227] Graph 776B shows the trace data (e.g., the next 50 traces), and graph 778B shows the next guard band for the trace data of graph 776B. As shown in graph 778B, the guard band moves (e.g., rises) in the y - direction over time to accommodate acceptable drift.

[0228] Figures 8A - 8B show guard band adaptation according to a particular embodiment. The guard band adaptation shown in any part of Figures 8A - 8B can be used in any of the methods of the present disclosure (e.g., Figures 2A - 2D, Figures 3A - 3D, and / or Figures 4A - 4D) to adapt the guard band to the motion state of the system (e.g., not adapt to violations such as defect level shifts).

[0229] Figure 8A shows a horizontal magnification change. Graph 810A shows trace data 812A and trace data 812B. Trace data 812A and 812B can have different magnifications in the y - direction. As shown in graph 810B, trace data 812A and / or trace data 812B can undergo a horizontal magnification change (e.g., a horizontal magnification change at the policy end point).

[0230] In some embodiments, the trace data 812A and 812B are for a good substrate. By changing the magnification in the horizontal direction, a more accurate guard band can be created based on the trace data 812A and 812B. By horizontally magnifying the trace data that can correspond to a good or bad substrate, the difference between the trace data and the guard band can be more accurately identified (e.g., reducing false detection and detection omission).

[0231] FIG. 8B shows vertical magnification change and horizontal distortion correction. Graph 850A shows trace data 812A and trace data 812B having different magnifications in the y direction. The vertical magnification change of the trace data 812A and 812B in graph 850A can be performed to generate graph 850B (e.g., the amplitude is normalized). The horizontal distortion correction of the trace data 812A and 812B in graph 850B can be performed to generate graph 850C (e.g., dynamic time distortion correction).

[0232] By trace distortion correction and magnification change, factors of different amplitudes with a phase shift can be ignored. The horizontal distortion correction is applied to ignore the phase shift factor and maintain the vertical noise. The vertical and horizontal magnification changes are applied to handle region transfer (e.g., applying the guard band to different regions such as different strategies).

[0233] FIG. 9 is a block diagram showing a computer system 900 according to a particular embodiment. In some embodiments, computer system 900 can be connected to other computer systems (e.g., via a network such as a local area network (LAN), intranet, extranet, or the Internet). Computer system 900 can operate in the capacity of a server or client computer in a client - server environment, or can operate as a peer computer in a peer - to - peer or distributed network environment. Computer system 900 can be provided by a personal computer (PC), tablet PC, set - top box (STB), personal digital assistant (PDA), cellular phone, web device, server, network router, switch or bridge, or any device capable of (sequentially or otherwise) executing a set of instructions that specify actions to be taken by that device. Further, the term "computer" shall include any one or more groups of computers that execute, individually or jointly, one set (or more) of instructions for performing any one or more of the methods described herein.

[0234] In a further aspect, computer system 900 can include a processing device 902, volatile memory 904 (e.g., random access memory (RAM)), non - volatile memory 906 (e.g., read - only memory (ROM) or electrically erasable programmable ROM (EEPROM)), and a data storage device 918, which can communicate with each other via a bus 908.

[0235] The processing device 902 can be provided by one or more processors, such as a general-purpose processor (e.g., a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a microprocessor implementing other types of instruction sets, or a microprocessor implementing a combination of types of instruction sets), or a special processor (e.g., an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), or a network processor, etc.).

[0236] The computer system 900 can further include a network interface device 922 (e.g., coupled to the network 974). The computer system 900 can also include a video display unit 910 (e.g., an LCD), an alphanumeric input device 912 (e.g., a keyboard), a cursor control device 914 (e.g., a mouse), and a signal generation device 920.

[0237] In some embodiments, the data storage device 918 can include a non-transitory computer-readable storage medium 924 (e.g., a non-transitory machine-readable storage medium) that can store instructions 926 encoding any one or more of the methods or functions described herein, the instructions 926 encoding components of FIG. 1 (e.g., the prediction component 114, the model 190, etc. used for prediction or detection), and including instructions for implementing the methods described herein. These instructions, when executed, can cause the processing device to execute the methods described herein.

[0238] The instructions 926 can also reside, fully or partially, within the volatile memory 904 and / or the processing device 902 during its execution by the computer system 900, and thus the volatile memory 904 and the processing device 902 can also constitute a machine-readable storage medium.

[0239] In an illustrative example, computer-readable storage medium 924 is shown as a single medium, but the term "computer-readable storage medium" shall include a single medium or multiple media (e.g., a centralized or distributed database, and / or associated cache and server) that store one or more sets of executable instructions. The term "computer-readable storage medium" shall also include any tangible medium that can store or encode a set of instructions for causing a computer to execute any one or more of the methods described herein. The term "computer-readable storage medium" shall include, but is not limited to, solid state memory, optical media, and magnetic media.

[0240] The methods, components, and features described herein can be implemented by individual hardware components or integrated into the functionality of other hardware components such as ASICs, FPGAs, DSPs, or similar devices. Additionally, the methods, components, and features can be implemented by firmware modules or functional circuits within a hardware device. Further, the methods, components, and features can be implemented in any combination of hardware devices and computer program components or within a computer program.

[0241] Unless otherwise specifically stated, terms such as "identify", "generate", "cause", "provide", "receive", "determine", "update", "compare", "train", "distortion correction", "magnification change", "acquire", etc. refer to measures and processes executed or implemented by a computer system that manipulates and transforms data represented as physical (electronic) quantities in computer system registers and memories into other data similarly represented as physical quantities in computer system memories or registers or other such information storage, transmission, or display devices. Also, in this specification, terms such as "first", "second", "third", "fourth", etc. are meant to be labels for distinguishing different elements and may not have the meaning of an order following a numerical indication.

[0242] The examples described in this specification also refer to an apparatus for executing the methods described in this specification. This apparatus can be specifically constructed to execute the methods described in this specification or can include a general-purpose computer system selectively programmed by a computer program stored within the computer system. Such a computer program can be stored within a computer-readable tangible storage medium.

[0243] The methods and illustrations described in this specification are not inherently related to any particular computer or other device. In accordance with the teachings described in this specification, various general-purpose systems can be used, or it may prove convenient to construct more specialized devices for executing each of the methods and / or individual functions, routines, subroutines, or operations described in this specification. Examples of structures for various such systems are described in the above explanation.

[0244] The foregoing description is intended to be illustrative rather than restrictive. Although the present disclosure has been described with reference to specific illustrative examples and embodiments, it will be understood that the present disclosure is not limited to the examples and embodiments described. The scope of the present disclosure should be determined with reference to the following claims, along with the full scope of equivalents given in the claims.

Claims

1. Identifying trace data including a plurality of data points, wherein the trace data is associated with the fabrication of a substrate having a characteristic value that meets a threshold through a substrate processing system; Determining guard band violation data points among the plurality of data points of the trace data based on a guard band; Determining guard band violation shape characterization based on the guard band violation data points, wherein the classification of additional guard band violation data points of additional trace data is based on the guard band violation shape characterization, and the execution of a corrective action associated with the substrate processing system is based on the classification; A method comprising:

2. The method according to claim 1, wherein the guard band violation shape characterization is one or a weighted combination of one or more of guard band violation duration, magnitude of guard band violation, guard band violation area, guard band violation position, or intermittency of guard band violation.

3. The method according to claim 1, wherein the classification of the additional guard band violation data points includes linking consecutive or future violations into a single violation based on the guard band violation shape characterization.

4. The method according to claim 3, wherein the linking of the consecutive or future violations into the single violation is further based on one or more non-violation shapes between two of the consecutive or future violations.

5. The method according to claim 1, wherein the determination of the guard band violation data points is performed via multivariate analysis.

6. The method according to claim 1, wherein the determination of the guard band violation data points includes characterizing the guard band violation data points via multivariate analysis.

7. The method according to claim 1, wherein the determination of the guard band violation data points is further based on segmentation of a portion of the trace data associated with a change in value exceeding a threshold change and extraction of features from the trace data.

8. The method according to claim 1, wherein the determination of the guard band violation shape characterization includes training a machine learning model using a data input including the trace data and forming a trained machine learning model associated with the guard band violation shape characterization.

9. identifying additional trace data; determining additional guard band violation data points of the additional trace data based on the guard band; providing the additional guard band violation data points as data input to a trained machine learning model; receiving output data including prediction data from the trained machine learning model; determining the classification of the additional guard band violation data points based on the prediction data; and further comprising the method according to claim 1.

10. identifying trace data including a plurality of data points, wherein the trace data is associated with the fabrication of a substrate via a substrate processing system; determining guard band violation data points among the plurality of data points of the trace data based on a guard band; determining the classification of the guard band violation data points based on guard band violation shape characterization, wherein the execution of a corrective action associated with the substrate processing system is based on the classification of the guard band violation data points; and a method comprising.

11. The method according to claim 10, wherein the guard band violation shape characterization is one or a weighted combination of more of guard band violation duration, guard band violation magnitude, guard band violation area, guard band violation position, or guard band violation intermittency.

12. The method according to claim 10, wherein the determination of the classification of the guard band violation data points includes linking consecutive or future violations into a single violation based on the guard band violation shape characterization.

13. The method according to claim 10, wherein the determination of the guard band violation data points is performed via multivariate analysis.

14. The method according to claim 10, wherein the determination of the guard band violation data points is further based on segmentation of a portion of the trace data associated with a change in value exceeding a threshold change and extraction of features from the trace data.

15. Receiving past trace data associated with past fabrication of past substrates having past characteristic values that meet a threshold via the substrate processing system, Training a machine learning model using the data input including the past trace data to form a trained machine learning model associated with the guard band violation shape characterization The method according to claim 10, further comprising.

16. Providing the guard band violation data points as data input to a trained machine learning model, Receiving output data including prediction data from the trained machine learning model, Determining the classification of the guard band violation data points based on the prediction data The method according to claim 10, further comprising.

17. A non-transitory computer-readable storage medium storing instructions that, when executed, cause a processing device to perform operations, wherein the operations are Identifying trace data including a plurality of data points, wherein the trace data is associated with the fabrication of a substrate having a characteristic value that meets a threshold via a substrate processing system; Determining guard band violation data points among the plurality of data points of the trace data based on a guard band; Determining guard band violation shape characterization based on the guard band violation data points, wherein the classification of additional guard band violation data points of additional trace data is based on the guard band violation shape characterization, and the execution of a corrective action associated with the substrate processing system is based on the classification. A non-transitory computer-readable storage medium including the determination. Claim 18 The non-transitory computer-readable storage medium according to claim 17, wherein the guard band violation shape characterization is one or a weighted combination of a guard band violation duration, a magnitude of the guard band violation, a guard band violation area, a guard band violation position, or an intermittency of the guard band violation. Claim 19 The non-transitory computer-readable storage medium according to claim 17, wherein the classification of the additional guard band violation data points includes connecting consecutive violations into a single violation based on the guard band violation shape characterization. Claim 20 The non-transitory computer-readable storage medium according to claim 17, wherein the determination of the guard band violation data points is performed via multivariate analysis.

Images