Guard bands in substrate processing systems

Abstract

The method includes identifying trace data including a plurality of data points, the trace data being associated with the fabrication of substrates having characteristic values that meet a threshold through a substrate processing system. The method further includes determining guard band violation data points among the plurality of data points of the trace data based on a guard band. The method further includes determining a guard band violation shape characterization based on the guard band violation data points. Classification of additional guard band violation data points of additional trace data is based on the guard band violation shape characterization. Execution of a corrective action associated with the substrate processing system is based on the classification.

Description

[Technical Field]

[0001] TECHNICAL FIELD The present disclosure relates to guard bands, and more particularly to guard bands in substrate processing systems. [Background technology]

[0002] A product may be produced by performing one or more manufacturing processes using manufacturing equipment. For example, substrate processing equipment may be used to produce substrates through substrate processing operations. A product having specific characteristics is to be produced. Sensor data is monitored in connection with the substrate manufacturing process. Summary of the Invention

[0003] The following is a simplified summary of the present disclosure to provide a basic understanding of some aspects of the disclosure. This summary is not an exhaustive overview of the disclosure. It is not intended to identify key or critical elements of the disclosure, nor is it intended to delineate the scope of any particular embodiments of the disclosure or the scope of the claims. Its sole purpose is to present some concepts of the disclosure in a simplified form as a prelude to the more detailed description that is presented later.

[0004] In one aspect of the disclosure, a method includes identifying trace data including a plurality of data points, the trace data associated with fabrication of substrates having characteristic values ​​that satisfy a threshold through a substrate processing system. The method further includes determining guard band violation data points among the plurality of data points of the trace data based on the guard bands. The method further includes determining guard band violation shape characterizations based on the guard band violation data points. Classification of additional guard band violation data points of the additional trace data is based on the guard band violation shape characterizations. Implementation of corrective actions associated with the substrate processing system is based on the classifications.

[0005] In one aspect of the disclosure, a method includes identifying trace data including a plurality of data points, the trace data associated with fabrication of a substrate through a substrate processing system. The method further includes determining guardband violation data points among the plurality of data points of the trace data based on guardbands. The method further includes determining classifications of the guardband violation data points based on guardband violation shape characterizations. Implementing corrective actions associated with the substrate processing system is based on the classifications of the guardband violation data points.

[0006] In one aspect of the present disclosure, a non-transitory computer-readable storage medium is provided that stores instructions that, when executed, cause a processing device to perform operations, including identifying trace data including a plurality of data points, the trace data associated with fabrication of substrates having characteristic values ​​that satisfy a threshold through a substrate processing system. The operations further include determining guard band violation data points among the plurality of data points of the trace data based on guard bands. The operations further include determining guard band violation shape characterizations based on the guard band violation data points. Classifying additional guard band violation data points of the additional trace data based on the guard band violation shape characterizations. Performing corrective actions associated with the substrate processing system is based on the classifications.

[0007] The present disclosure is illustrated by way of example, and not limitation, in the figures of the accompanying drawings. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram illustrating an exemplary system architecture, according to certain embodiments. [Figures 2A-2D] 1 is a flowchart of a method associated with generating guard bands, according to certain embodiments. [Figures 3A-3D] 1 is a flow diagram of a method associated with guard band violation profiling, according to certain embodiments. [Figures 4A-4D]1 is a flow diagram of a method associated with a dynamic acceptable area outside of guard band limits, according to certain embodiments. [Figures 5A-5E] FIG. 1 illustrates an allowable distribution-type graph, in accordance with certain embodiments. [Figures 6A-6E] FIG. 1 illustrates guard band violation profiling, in accordance with certain embodiments. [Figures 7A-7F] FIG. 10 illustrates a dynamic acceptable area outside of guard band limits in accordance with certain embodiments. [Figure 8A-8B] FIG. 1 illustrates guard band adaptation, according to certain embodiments. [Figure 9] FIG. 1 is a block diagram illustrating a computer system, in accordance with certain embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0009] Techniques related to guard bands in substrate processing systems (e.g., guard band enforcement, guard band violation profiling, and dynamic areas outside of guard bands) are described herein. Guard bands can be upper and lower thresholds (e.g., acceptable error ranges) around a target value. Sensor data can be compared to the guard bands to determine whether the sensor data is within the threshold range (e.g., a good substrate) or outside the threshold range (e.g., a bad substrate).

[0010] Products are produced by performing one or more manufacturing processes using manufacturing equipment. Products having specific characteristics should be produced. For example, substrate processing equipment is used to produce substrates through substrate processing operations. Substrates that meet specific characteristic values ​​(e.g., dimensions as determined by metrology data) should be used, and substrates that do not meet the specific characteristic values ​​should be discarded. Sensor data associated with the substrate processing operations is collected over time. The sensor data is monitored for multiple purposes, including attempting to produce substrates that meet specific characteristic values, verifying that equipment is functioning properly, determining whether equipment requires current or future repair or replacement, and determining adjustments to equipment parameters to make the process more effective (e.g., with respect to conditions or criteria such as yield or quality, throughput or quantity, cost or lifetime, etc.). For purposes of simplicity, this discussion focuses on the objective of producing substrates that meet specific characteristic values.

[0011] Traditionally, sensor data is summarized over a particular recipe or recipe action associated with the production of a substrate by equipment. This summary is defined using a set of statistics, such as a mean and variance. These summary statistics are then compared to set limits, such as an average between a lower and upper value. If the sensor data is within the limits, it is presumed that the substrate meets a particular characteristic value; if the sensor data is outside the limits, it is presumed that the substrate does not meet the particular characteristic value. Limits that are too narrow will result in false positives (e.g., inaccurately predicting that the substrate will not meet the characteristic value). Limits that are too wide will result in false negatives (e.g., inaccurately predicting that the substrate will meet the characteristic value). Variations in the sensor data and in the substrate processing equipment can cause many false positives. Widening the limits to allow for variations in the sensor data and in the substrate processing equipment can cause many false negatives. Traditional systems inaccurately label substrates as meeting or not meeting characteristic values, resulting in wasted material, reduced yields, defective products, increased user time, increased equipment downtime, etc. Attempting to correct mislabeling uses extra processing costs, bandwidth, energy consumption, measurement operations, user time, etc.

[0012] The disclosed methods, devices, and systems provide guardband improvements in substrate processing systems that address these and other deficiencies of conventional solutions.

[0013] In some embodiments, the processing device identifies trace data associated with the production of substrates (e.g., good substrates) having characteristic values ​​that meet a threshold. The trace data can include sets of sensor data associated with the production of different substrates from different types of sensors. In some embodiments, the data is analyzed at the trace level by using guard bands (e.g., rather than just providing summary statistics of the sensors across a recipe or recipe operation). The guard bands can provide upper and lower thresholds across the length of the trace data. This provides many advantages over summary statistics (e.g., the present disclosure can identify and profile specific violations in the trace data). The processing device generates initial guard bands based on the trace data (e.g., 3σ around the average of the trace data at each time slot in the data, where the 3σ is determined by analyzing multiple runs of data at that particular time slot). The processing device determines an acceptable variance type for the guard bands based on the trace data. Because the trace data is for good wafers, the acceptable variance type for the guard bands can be time-shifted (e.g., moved in the x-direction) from one or more of the sets of sensor data to meet the average of the sensor data. The allowable variance type can include an upper guard band limit that is a different distance from the average of the trace data than the lower guard band limit. The allowable variance type can include wider and narrower guard band limits in certain portions of the guard band. The processing device generates the guard bands based on the trace data and the allowable variance type. The processing device compares additional trace data (e.g., of a substrate that is not known to be good or bad) to the guard bands, and in response to one or more data points not falling within the guard bands, the processing device performs corrective action (e.g., scrapping the substrate, suspending substrate processing operations, inspecting the substrate, etc.).

[0014] In some embodiments, trace data for both good and bad substrates is used, and guard bands are determined or updated based on an improved understanding of the differences between good and bad trace data, which may be a single category or multiple categories representing different states or degrees of bad substrates.

[0015] In some embodiments, the processing device identifies trace data associated with the production of substrates (e.g., good substrates) having characteristic values ​​that satisfy thresholds. The processing device determines guard band violation data points in the trace data based on the guard bands. The processing device determines guard band violation shape characterizations based on the guard band violation data points. The processing device compares additional trace data (e.g., of substrates that are not known to be good or bad) to the guard bands to determine the guard band violation data points. The processing device determines a classification of the guard band violation data points (e.g., whether the guard band violation data points correspond to good wafers or bad wafers) based on the guard band violation shape characterizations.

[0016] In some embodiments, the processing device identifies trace data associated with the production of substrates having characteristic values ​​that meet thresholds (e.g., good substrates). The processing device determines a dynamic acceptable area outside the guard band limits based on the trace data. The dynamic acceptable area is an area outside the guard bands corresponding to data points of good substrates that are the result of acceptable noise, acceptable drift, etc. The processing device compares additional trace data (e.g., of substrates that are not known to be good or bad) to the acceptable area outside the guard band limits. In response to one or more of the data points being outside the acceptable area, the processing device performs corrective action (e.g., scrapping the substrate). In response to one or more of the data points being within the acceptable area, the processing device updates the acceptable area. For example, the updated acceptable area can allow for additional drift or noise based on an average of the processed additional trace data.

[0017] In some embodiments, a processing device (e.g., performing a guardband method) presents results of trace data runs concluded to be within acceptable guardband limits, trace data runs with violations including geometric characteristics of those violations, and / or regions where the processing device (e.g., performing a guardband method) cannot definitively determine the occurrence of a violation. The processing device (e.g., performing a guardband method) can allow a user (e.g., a subject matter expert) to confirm or invalidate the conclusions and suggestions of the guardband method. The processing device (e.g., performing a guardband method) can use feedback from a user (e.g., a subject matter expert) to update the guardband limits, other violation assessments, and / or guardband characterizations.

[0018] Aspects of the present disclosure provide technical advantages over conventional solutions. The present disclosure has fewer false positives and false negatives compared to conventional solutions, resulting in less material waste, increased yield, fewer defective products, less user time, less equipment downtime, etc. The present disclosure has fewer corrections for mislabeled substrates compared to conventional solutions, resulting in less process processor expense, bandwidth usage, energy consumption, measurement operations, user time, etc.

[0019] Some embodiments of the present disclosure are described in the context of substrate processing, although in some embodiments the present disclosure also applies to other types of manufacturing processes.

[0020] Some embodiments of the present disclosure are described in connection with monitoring sensor data for the purpose of producing substrates that meet certain property values. In some embodiments, the present disclosure may also monitor sensor data for other purposes, such as ensuring that equipment is functioning properly, determining to perform repairs or replacements (e.g., preventative maintenance) of equipment or equipment components, or determining adjustments to equipment parameters to make a process more effective (e.g., with respect to conditions or criteria such as yield or quality, throughput or quantity, cost or lifespan, etc.).

[0021] Some embodiments of the present disclosure are described in connection with taking corrective action, which in some embodiments may include identifying characteristics of the trace data as degraded, different, etc.

[0022] 1 is a block diagram illustrating an example system 100 (an example system architecture) according to certain embodiments. System 100 includes client devices 120, manufacturing equipment 124 (e.g., substrate processing equipment), sensors 126, metrology equipment 128, a prediction server 112, and a data store 140. Prediction server 112 may be part of a prediction system 110. Prediction system 110 may further include server machines 170 and 180. Prediction system 110 may be used to predict whether an anomaly has occurred, detect that an anomaly has occurred (e.g., using guard banding techniques), and the like.

[0023] In some embodiments, the manufacturing equipment 124 (e.g., a cluster tool) is part of a substrate processing system (e.g., an integrated processing system). The manufacturing equipment 124 includes one or more of a controller, a containment system (e.g., a substrate carrier, a front-opening unified pod (FOUP), an autoteach FOUP, a process kit containment system, a substrate containment system, a cassette, etc.), a side storage pod (SSP), an aligner device (e.g., an aligner chamber), a factory interface (e.g., a front end of equipment module (EFEM)), a load lock, a transfer chamber, one or more processing chambers, a robot arm (e.g., disposed in a transfer chamber, a front interface, etc.), etc. The containment system, SSP, and load lock attached to the factory interface, and the robot arm disposed in the factory interface, are for transferring contents (e.g., substrates, process kit rings, carriers, certification wafers, etc.) between the containment system, SSP, load lock, and the factory interface. The aligner device is disposed in the factory interface to align the contents. The load locks and processing chambers attached to the transfer chamber, and the robot arm located within the transfer chamber, are for transferring contents (e.g., substrates, process kit rings, carriers, certification wafers, etc.) between the load locks, processing chambers, and transfer chamber. In some embodiments, the manufacturing equipment 124 includes components of a substrate processing system. In some embodiments, the manufacturing equipment 124 is used to fabricate one or more products (e.g., substrates, semiconductors, wafers, etc.). In some embodiments, the manufacturing equipment 124 is used to fabricate one or more components used within the substrate processing system.

[0024] The sensors 126 may be coupled to the manufacturing equipment 124. The sensors 126 may provide sensor data associated with the manufacturing equipment 124 (e.g., associated with the fabrication of a corresponding product, such as a substrate, by the manufacturing equipment 124). The sensor data may be stored as measurements over time (e.g., trace data 142). The trace data 142 may include historical trace data 144 and current trace data 146. The trace data 142 may be used for equipment health and / or product health (e.g., product quality). The manufacturing equipment 124 may produce products according to a recipe or run over a period of time. In some embodiments, the trace data 142 may include one or more values ​​of temperature (e.g., heater temperature), spacing (SP), pressure, high frequency radio frequency (HFRF), electrostatic chuck (ESC) voltage, current, flow rate, power, voltage, etc. The trace data 142 may be associated with or indicative of manufacturing parameters, such as hardware parameters (e.g., settings or components, e.g., size, type, etc.) of the manufacturing equipment 124 or process parameters of the manufacturing equipment 124. Alternatively, or in addition, data associated with some hardware parameters may be stored as manufacturing parameters. The manufacturing parameters may indicate input settings for a manufacturing device (e.g., heater power, gas flow rate, etc.). The trace data 142 and / or manufacturing parameters may be provided as the manufacturing equipment 124 performs a manufacturing process (e.g., equipment readings as the product is being processed). The trace data 142 may vary from product to product (e.g., from substrate to substrate).

[0025] Metrology equipment 128 may be used to measure characteristics of products such as substrates (e.g., processed substrates, partially processed substrates, etc.). Metrology equipment may incorporate analytics to estimate or better determine metrology values. Metrology data may be included in performance data 150 along with other performance metrics such as equipment maintenance, yield, etc. Performance data 150 may include historical performance data 152 and current performance data 154. Metrology data and / or performance data 150 may include virtual metrology data, non-virtual metrology data, a mix of virtual and non-virtual metrology data, etc.

[0026] In some embodiments, the trace data 142, the performance data 150, and / or the manufacturing parameters may be processed (e.g., by the client device 120 and / or the prediction server 112). Processing the trace data 142 may include generating features. In some embodiments, these features are patterns (e.g., slope, width, height, peaks, etc.) within the trace data 142 or the performance data 150, or combinations of values ​​(e.g., power derived from voltage and current, etc.) from the trace data 142 or the performance data 150. The trace data 142 may include features that can be used by the prediction component 114 and / or the client device 120 to perform signal processing and / or obtain prediction data 168 for taking corrective actions. The prediction component 114 may be used to predict whether an anomaly has occurred, detect that an anomaly has occurred (e.g., using guard banding techniques), etc.

[0027] Each instance (e.g., set) of trace data 142 may correspond to a product (e.g., a substrate), a set of manufacturing equipment 124, a type of substrate produced by the manufacturing equipment 124, etc. Each instance of performance data 150 or manufacturing parameters may similarly correspond to a product, a set of manufacturing equipment, a type of substrate produced by the manufacturing equipment, etc. Data store 140 may further store information relating sets of different data types, e.g., information indicating sets of trace data, sets of sensor data, sets of metrology data, and / or sets of manufacturing parameters associated with the same product, manufacturing equipment, substrate type, etc.

[0028] In some embodiments, the prediction system 110 can generate the predicted data 168 using supervised machine learning (e.g., a supervised data set, where the performance data 150 includes metrology data and the trace data 142 used to train the model 190 is associated with good and bad substrates, etc.). In some embodiments, the prediction system 110 can generate the predicted data 168 using semi-supervised learning (e.g., a semi-supervised data set, where the performance data 150 is a prediction percentage and the trace data 142 used to train the model 190 is associated with only good substrates, etc.). In some embodiments, the prediction system 110 can generate the predicted data 168 using unsupervised machine learning (e.g., an unsupervised data set, clustering, clustering based on the trace data 142, etc.). In some embodiments, the prediction system 110 can generate the predicted data 168 using a model that is one or more of a machine learning model, a statistical model, etc.

[0029] The client devices 120, manufacturing equipment 124, sensors 126, measurement equipment 128, prediction server 112, data store 140, server machine 170, and server machine 180 can be coupled to each other via network 130 to generate predictive data 168 and perform corrective actions.

[0030] In some embodiments, network 130 is a public network that provides client device 120 with access to prediction server 112, data store 140, and other publicly available computing devices. In some embodiments, network 130 is a private network that provides client device 120 with access to manufacturing equipment 124, sensors 126, measurement equipment 128, data store 140, and other privately available computing devices. Network 130 may include one or more wide area networks (WANs), local area networks (LANs), wired networks (e.g., Ethernet networks), wireless networks (e.g., 802.11 networks or Wi-Fi networks), cellular networks (e.g., Long Term Evolution (LTE) networks), routers, hubs, switches, server computers, cloud computing networks, and / or combinations thereof.

[0031] Client device 120 may include computing devices such as personal computers (PCs), laptops, mobile phones, smartphones, tablet computers, netbook computers, network-connected televisions ("smart TVs"), network-connected media players (e.g., Blu-ray players), set-top boxes, over-the-top (OTT) streaming devices, operator boxes, etc. Client device 120 may include a corrective action component 122. Corrective action component 122 may receive user input (e.g., via a graphical user interface (GUI) displayed via client device 120).

[0032] In some embodiments, the corrective action component 122 obtains trace data 142 (e.g., current trace data 146) associated with the manufacturing equipment 124 (e.g., from the data store 140, etc.) and provides the trace data 142 (e.g., current trace data 146) associated with the manufacturing equipment 124 to the prediction system 110. In some embodiments, the corrective action component 122 stores the trace data 142 in the data store 140, and the prediction server 112 retrieves the trace data 142 from the data store 140. In some embodiments, the prediction server 112 can store the output of the trained machine learning model 190 (e.g., predicted data 168) in the data store 140, and the client device 120 can retrieve the output from the data store 140. In some embodiments, the corrective action component 122 receives the corrective action instructions from the prediction system 110 and implements the corrective action. The client device 120 may include an operating system that enables a user to one or more of create, view, or edit data (e.g., instructions associated with the manufacturing equipment 124, corrective actions associated with the manufacturing equipment 124, etc.).

[0033] In some embodiments, historical performance data 152 corresponds to historical characteristic data of a product (e.g., produced using manufacturing parameters associated with historical trace data 144 and stored manufacturing parameters), and predictive data 168 is associated with predicted characteristic data (e.g., predicted characteristic data of a product to be produced or produced under conditions recorded by current trace data 146 and / or manufacturing parameters). In some embodiments, predictive data 168 is predicted metrology data (e.g., virtual metrology data) of a product to be produced or produced according to conditions recorded as current trace data 146 and / or manufacturing parameters. In some embodiments, predictive data 168 is an indication of anomalies (e.g., an abnormal product, an abnormal component, an abnormal manufacturing equipment 124, an abnormal energy usage, etc.) and one or more causes of these anomalies. In some embodiments, predictive data 168 is an indication of change or drift over time in some component, such as manufacturing equipment 124, a sensor 126, or a measurement device 128. In some embodiments, the predictive data 168 is an end of life indication for a component such as manufacturing equipment 124, a sensor 126, or a measurement device 128.

[0034] Running a manufacturing process that results in a defective product can be costly in terms of time, energy, product, components, manufacturing equipment 124, costs of identifying the defects and disposing of the defective product, etc. By generating predictive data 168 based on trace data 142 and performing corrective actions based on predictive data 168, system 100 can have the technical advantage of avoiding costs associated with creating, identifying, and disposing of defective products.

[0035] Running a manufacturing process that results in a failure of a component of manufacturing equipment 124 can be costly in terms of downtime, product damage, equipment damage, expedited ordering of replacement components, etc. By generating predictive data 168 based on trace data 142 (e.g., manufacturing parameters being used or to be used to manufacture a product) and performing corrective actions (e.g., predicted operational maintenance such as replacing, treating, cleaning, etc.) based on predictive data 168, system 100 can have the technical advantage of avoiding the costs of one or more of unexpected component failures, unscheduled downtime, lost production rate, unexpected equipment failures, product waste, etc. Monitoring the performance of components (e.g., manufacturing equipment 124, sensors 126, measurement devices 128, etc.) over time can provide an indication of deteriorating components.

[0036] The manufacturing parameters may be suboptimal for producing the product, which may result in costly consequences such as increased resource (e.g., energy, coolant, gas, etc.) consumption, increased amount of time to produce the product, increased component failures, increased quantity of defective products, etc. By generating predictive data 168 based on features of trace data 142 and performing corrective actions to update manufacturing parameters (e.g., set optimal manufacturing parameters) based on the predictive data 168, system 100 may have the technical advantage of using optimal manufacturing parameters (e.g., hardware parameters, process parameters, optimal design) to avoid the costly consequences of suboptimal manufacturing parameters.

[0037] The corrective action may be associated with one or more of computational process control (CPC), statistical process control (SPC) (e.g., SPC on electronic components to determine the process under control, SPC to predict the useful life of components, SPC to compare with 3σ graphs, etc.), advanced process control (APC), model-based process control, preventative operational maintenance, design optimization, updating manufacturing parameters or updating manufacturing recipes for current or future manufacturing processes, feedback control, machine learning corrections, etc.

[0038] In some embodiments, the corrective action includes providing an alert (e.g., a warning to stop or not run a manufacturing process if the predictive data 168 indicates a predicted anomaly, such as an anomaly in a product, component, or manufacturing equipment 124). In some embodiments, the corrective action includes scheduling preventive maintenance. In some embodiments, the corrective action includes scheduling corrective maintenance. In some embodiments, the corrective action includes updating a process recipe to fabricate subsequent substrates. In some embodiments, the corrective action may be determined in consideration of an ongoing substrate treatment process and may include updating a current process. In some embodiments, the corrective action includes correcting chamber drift associated with the manufacturing equipment 124 (e.g., substrate processing equipment). In some embodiments, the corrective action includes correcting sensor drift of a sensor associated with the manufacturing equipment 124 (e.g., substrate processing equipment). In some embodiments, the corrective action includes providing feedback control (e.g., modifying a manufacturing parameter in response to the predictive data 168 indicating a predicted anomaly). In some embodiments, the corrective action includes providing machine learning (e.g., modifying one or more manufacturing parameters based on the predictive data 168). In some embodiments, performing the corrective action includes updating one or more manufacturing parameters. In some embodiments, the one or more corrective actions are performed in connection with components of the substrate processing equipment.

[0039] The manufacturing parameters may include hardware parameters (e.g., replacing a component, using a specific component, replacing a processing chip, updating firmware, etc.) and / or process parameters (e.g., temperature, pressure, flow rate, current, voltage, gas flow, ramp rate, etc.). In some embodiments, the corrective action includes preventive operational maintenance (e.g., replacing, treating, cleaning, etc., components of the manufacturing equipment 124). In some embodiments, the corrective action includes performing design optimization (e.g., updating manufacturing parameters, manufacturing process, manufacturing equipment 124 for an optimized product, etc.). In some embodiments, the corrective action includes updating a strategy (e.g., placing the manufacturing equipment 124 in idle mode, sleep mode, warm-up mode, etc.).

[0040] Prediction server 112, server machine 170, and server machine 180 may each include one or more computing devices such as a rack-mounted server, a router computer, a server computer, a personal computer, a mainframe computer, a laptop computer, a tablet computer, a desktop computer, a graphics processing unit (GPU), an accelerator application-specific integrated circuit (ASIC) (e.g., a tensor processing unit (TPU)), etc.

[0041] The prediction server 112 can include a prediction component 114. In some embodiments, the prediction component 114 can receive current trace data 146 (e.g., received from the client device 120, retrieved from the data store 140) and generate output (e.g., predicted data 168) for performing corrective actions associated with the manufacturing equipment 124 based on the current data. In some embodiments, the prediction component 114 can use one or more trained models 190 to determine the output for performing corrective actions based on the current data. In some embodiments, the prediction component 114 predicts that an anomaly has occurred or will occur. In some embodiments, the prediction component 114 predicts that another event has occurred or may occur.

[0042] Model 190 can be a single model or many models. Models can be applied sequentially, multiple models can be used simultaneously, an appropriate model can be selected based on some metric, a combination of these approaches can be used, and so on. Model 190 (or the models included in model 190) can be machine learning models, including supervised, unsupervised, or semi-supervised machine learning models. Model 190 does not have to be a machine learning model, and can be, for example, a statistical model, a correlation model, etc.

[0043] In some embodiments, a first model 190 is used to generate guard bands (see, e.g., Figures 2A-2D), a second model 190 is used for guard band violation profiling (see, e.g., Figures 3A-3D), and a third model 190 is used to generate dynamic areas outside the guard bands (see, e.g., Figures 4A-4D).

[0044] In some embodiments, the data input to the model 190 may include trace data 142 from a single sensor 126. In other embodiments, the data input to the model 190 may include trace data 142 from many sensors 126 exhibiting values ​​of different characteristics. The data input may include manufacturing parameters. The features extracted from the trace data 142, the method of extracting the features, the corrective action and / or predictive data 168 associated with the features, and the method of associating the corrective action and / or predictive data 168 may all be tailored to the data provided as input.

[0045] In some embodiments, the prediction component 114 receives current trace data 146, provides the current trace data 146 as input to a model 190, and obtains output from the model 190 indicative of predicted data 168. In some embodiments, the predicted data 168 is indicative of performance data 150 (e.g., metrology data, yield, etc.). In some embodiments, the predicted data 168 is indicative of corrective actions.

[0046] In some embodiments, model 190 takes trace data 142 (e.g., data indicative of a recipe, a component of manufacturing equipment, etc., associated with trace data 142) as input and produces predicted data 168 as output. Model 190 can be a single model or can include many models. Model 190 can determine which process to execute based on the input data, or a user can indicate which analysis is appropriate for the input data, or a combination thereof.

[0047] The data store 140 and / or may be memory (e.g., random access memory), a drive (e.g., a hard drive, a flash drive), a database system, or another type of component or device capable of storing data. The data store 140 and / or may include multiple storage components (e.g., multiple drives or multiple databases) that may span multiple computing devices (e.g., multiple server computers). The data store 140 may store trace data 142, performance data 150, and prediction data 168. The trace data 142 may include historical trace data 144 and current trace data 146. The trace data may include sensor data time traces over the duration of a manufacturing process, associations of data with physical sensors, preprocessed data such as average and composite data, and data indicative of sensor performance over time (e.g., over many manufacturing processes). The manufacturing parameters and performance data 150 may include similar characteristics. The historical trace data 144, manufacturing parameters, and historical performance data 152 may be historical data (e.g., at least a portion of the training model 190). The current trace data 146 may be the current data (e.g., at least a portion to be input into the model 190 next to the historical data) from which the predictive data 168 (e.g., to perform corrective action) should be generated.

[0048] In some embodiments, prediction system 110 further includes server machine 170 and server machine 180. Server machine 170 includes dataset generator 172 capable of generating datasets (e.g., a set of data inputs and a set of target outputs) for training, validating, and / or testing one or more models, such as machine learning models. Model 190 may include one or more machine learning models or may be other types of models, such as statistical models. Models incorporating machine learning may be trained using input data and, optionally, target output data. Models that do not incorporate machine learning may also be trained. In some embodiments, dataset generator 172 may divide historical data (e.g., historical trace data 144, manufacturing parameters, or historical performance data 152 stored in data store 140) into a training set (e.g., 60 percent of the historical data), a validation set (e.g., 20 percent of the historical data), and a test set (e.g., 20 percent of the historical data). In some embodiments, prediction system 110 (e.g., via prediction component 114) generates multiple sets of elements. For example, a first set of elements may correspond to a first set of types of sensor data (e.g., from a first set of sensors, a first combination of values ​​from the first set of sensors, a first pattern of values ​​from the first set of sensors) corresponding to each of the datasets (e.g., a training set, a validation set, and a test set), and a second set of elements may correspond to a second set of types of sensor data (e.g., from a second set of sensors different from the first set of sensors, a second combination of values ​​different from the first combination, a second pattern different from the first pattern) corresponding to each of the datasets.

[0049] Server machine 180 includes a training engine 182, an authentication engine 184, a selection engine 185, and / or a test engine 186. The engines (e.g., training engine 182, authentication engine 184, selection engine 185, and test engine 186) may refer to hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, processing device, etc.), software (e.g., instructions executed on a processing device, general-purpose computer system, or dedicated machine), firmware, microcode, or a combination thereof. Training engine 182 may be capable of training machine learning model 190 or various machine learning models included in model 190 using one or more sets of elements associated with a training set from dataset generator 172. Training engine 182 may generate multiple trained machine learning models 190, each corresponding to a distinct set of elements of the training set (e.g., sensor data from a distinct set of sensors). For example, a first trained machine learning model may be trained using all elements (e.g., X1-X5), a second trained machine learning model may be trained using a first subset of elements (e.g., X1, X2, X4), and a third trained machine learning model may be trained using a second subset of elements (e.g., X1, X3, X4, and X5), where the second subset of elements may partially overlap with the first subset of elements. The dataset generator 172 may receive the output of the trained machine learning model (e.g., a model trained to perform a first operation of trace data processing), collect that data into training, validation, and test datasets, and use these datasets to train a second machine learning model (e.g., a model to be trained to perform a second operation of trace data processing).

[0050] The authentication engine 184 may be capable of authenticating the trained machine learning models 190 using a corresponding set of elements of the authentication set from the dataset generator 172. For example, a first trained machine learning model 190 trained using a first set of elements of the training set may be authenticated using the first set of elements of the authentication set. The authentication engine 184 may determine the accuracy of each of the trained machine learning models 190 based on the corresponding set of elements of the authentication set. The authentication engine 184 may discard trained machine learning models 190 with an accuracy that does not meet a threshold accuracy. In some embodiments, the selection engine 185 may be capable of selecting one or more trained machine learning models 190 with an accuracy that meets the threshold accuracy. In some embodiments, the selection engine 185 may be capable of selecting the trained machine learning model 190 with the highest accuracy among the trained machine learning models 190. In some embodiments, the authentication engine 184 and the selection engine 185 may repeat this process for each machine learning model included in the model 190.

[0051] In some embodiments, the validation engine 184 performs validation and / or validation (e.g., validation and validation (V&V)). Validation and validation can be independent procedures used together to determine whether a product, service, or system (e.g., a machine learning model) meets requirements and specifications and intended purposes. Validation can include assurance that a machine learning model meets the needs of a customer or other identified stakeholder (e.g., involving acceptance and conformance by external customers). Validation can include evaluation of whether a machine learning model complies with regulations, requirements, specifications, or imposed conditions (e.g., internal processes).

[0052] Test engine 186 may be able to test the trained machine learning models included in model 190 using corresponding sets of elements of the test set from dataset generator 172. For example, a first trained machine learning model 190 trained using a first set of elements of the training set may be tested using the first set of elements of the test set. Based on the test set, test engine 186 may determine the trained machine learning model included in model 190 that has the highest accuracy of all of the trained machine learning models. Test engine 186 may repeat this process for all machine learning models included in model 190.

[0053] The model 190 may reference a model artifact created by the training engine 182 using a training set that includes data inputs and corresponding target outputs (correct responses for each training input). Patterns in the dataset that map the data inputs to the target outputs (correct responses) may be discovered, and a machine learning model is provided with a mapping that captures these patterns. The machine learning model may use one or more of a support vector machine (SVM), a radial basis function (RBF), clustering, supervised machine learning, semi-supervised machine learning, unsupervised machine learning, a k-nearest neighbor algorithm (k-NN), linear regression, random forests, a neural network (e.g., an artificial neural network), etc.

[0054] The prediction component 114 can provide the current trace data 146 to the model 190, run the trained machine learning model 190 on the input, and obtain one or more outputs. The prediction component 114 can determine (e.g., extract) prediction data 168 from the output of the model 190 and can determine (e.g., extract) confidence data from the output indicating a confidence level that the prediction data 168 is an accurate predictor of the process associated with the current trace data 146 and / or input data for products produced or to be produced using the manufacturing equipment 124 with the manufacturing parameters. The prediction component 114 may also be capable of determining a confidence range associated with the predicted event, such as a residual image useful life (RUL) window including upper and lower bounds. The prediction component 114 or the corrective action component 122 can use the confidence data to determine whether and / or when to trigger a corrective action associated with the manufacturing equipment 124 based on the prediction data 168.

[0055] The confidence data may include or indicate a confidence level that the prediction data 168 is an accurate prediction for a product associated with at least a portion of the input data. In one example, the confidence level is a real number between 0 and 1, inclusive, where 0 indicates no confidence that the prediction data 168 is an accurate prediction for a product processed according to the input data, and 1 indicates absolute confidence that the prediction data 168 accurately predicts a characteristic of a product processed according to the input data. In response to confidence data indicating a confidence level below a threshold level for a predetermined number of cases (e.g., percentage of cases, frequency of cases, frequency of occurrence, total number of cases, etc.), the prediction component 114 may retrain the model 190 (e.g., based on current trace data 146, manufacturing parameters, current performance data 154, etc.).

[0056] For purposes of explanation and not limitation, aspects of the disclosure describe using historical data (e.g., historical trace data 144, historical performance data 152) to train one or more machine learning models 190 and inputting current data (e.g., current trace data 146) into the one or more trained machine learning models 190 to determine predicted data 168. In other embodiments, a heuristic or rule-based model is used to determine predicted data 168 (e.g., without using a trained machine learning model). The prediction component 114 can monitor historical trace data 144 and historical performance data 152.

[0057] In some embodiments, the functionality of client device 120, prediction server 112, server machine 170, and server machine 180 may be provided by fewer machines. For example, in some embodiments, server machines 170 and 180 may be combined into a single machine, and in some other embodiments, server machine 170, server machine 180, and prediction server 112 may be combined into a single machine. In some embodiments, client device 120 and prediction server 112 may be combined into a single machine.

[0058] In general, functions described in one embodiment as being performed by client device 120, prediction server 112, server machine 170, and server machine 180 may also be performed by prediction server 112 in other embodiments, where appropriate. In addition, functions attributed to particular components may also be performed by different or multiple components operating together. For example, in some embodiments, prediction server 112 may determine corrective actions based on prediction data 168. In another example, client device 120 may determine prediction data 168 based on output from a trained machine learning model.

[0059] Additionally, the functionality of a particular component may be performed by different or multiple components working together. One or more of prediction server 112, server machine 170, or server machine 180 may be accessed as a service offered to other systems or devices via an appropriate application programming interface (API).

[0060] In embodiments, a "user" may refer to a single individual. However, other embodiments of the present disclosure encompass "users" that are entities managed by multiple users and / or automated sources. For example, a set of individual users combined as a group of administrators may be considered a "user."

[0061] Embodiments of the present disclosure may be applied to data quality assessment, feature enhancement, model evaluation, virtual metrology (VM), predictive maintenance (PdM), marginal optimization, anomaly or defect detection, anomaly or defect classification, and the like.

[0062] 2A-2D, 3A-3D, and 4A-4D are flowcharts of methods 200A-D, 300A-D, and 400A-D associated with guard bands, according to certain embodiments. In some embodiments, methods 200A-D, 300A-D, and 400A-D are performed by processing logic that includes hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, processing device, etc.), software (e.g., instructions executed on a processing device, general-purpose computer system, or dedicated machine), firmware, microcode, or a combination thereof. In some embodiments, methods 200A-D, 300A-D, and 400A-D are performed, at least in part, by prediction system 110. In some embodiments, methods 200A-D, 300A-D, and 400A-D are performed, at least in part, by one or more of prediction system 110 (e.g., prediction server 112, prediction component 114), client device 120 (e.g., corrective action component), manufacturing equipment 124, and / or metrology equipment 128. In some embodiments, a non-transitory storage medium stores instructions that, when executed by a processing device (e.g., of prediction system 110, server machine 180, prediction server 112, etc.), cause the processing device to perform one or more of methods 200A-D, 300A-D, and 400A-D.

[0063] For ease of explanation, methods 200A-D, 300A-D, and 400A-D are shown and described as a series of operations. However, operations in accordance with the present disclosure may occur in various orders and / or concurrently, with other operations not shown and described herein. Furthermore, in some embodiments, not all illustrated operations are performed to implement methods 200A-D, 300A-D, and 400A-D in accordance with the disclosed subject matter. In addition, those skilled in the art will understand and appreciate that methods 200A-D, 300A-D, and 400A-D may alternatively be represented via a state diagram or events as a series of interrelated states.

[0064] In some embodiments, FIGS. 2A-2D and 3A-3D are directed to sequential data guard band analysis for improved defect diagnosis, defect classification, and prediction (e.g., sequential guard band analysis for defect detection and classification (FDC)). In some embodiments, the present disclosure provides improved analysis of sequential data streams (e.g., trace data) using full trace analysis (FTA) of guard band defect diagnosis to provide improved detection, classification, and prediction, including reduced false alarms (e.g., false positives) and missed alarms (e.g., missed detections). Traditional sensor trace analysis can have false positives and missed detections (e.g., inability to adequately capture normal variation, inability to fit a model over time, inability to address strategy endpoints, inability to address out-of-phase traces, inability to propagate a model to different regions, etc.), resulting in traditional low adoption rates of guard bands. Traditional summary statistics can leak portions of the trace data (e.g., transient regions). The present disclosure can be applied to address false positives and false negatives, provide robustness over time, and flexibility to different regions (see, e.g., FIGS. 8A-8B). The present disclosure can use guard bands to analyze both transient and steady-state regions and extract complex features (e.g., more complex than inherent variable analysis (UVA)).

[0065] The guard bands of the present disclosure can be used to detect anomalies or defects that occur within trace data but may not follow typical anomaly shapes such as spikes or fluctuations. Traditional guard bands can have output quality issues based on false positives (e.g., within transient events), stretching at pre- and post-transition boundaries, trace-to-trace variability being treated as anomalies, and any violations being treated as equal without being profiled or quantified. The guard bands of the present disclosure can address these shortcomings. The guard bands of the present disclosure can be provided alone or combined with other analytical capabilities such as semi-automated feature extraction (SFE).

[0066] A guard band can be a channel defined for a data stream that is meant to identify a region of commonality (e.g., acceptable values) according to position within the data stream. A data stream is often the value of a sensor during some event, such as a manufacturing run. A data stream is often trace data, which is a series of data values ​​presented and arranged according to time (e.g., the x-axis is time). The ordering may not be based on time (e.g., it may be an indication of a number of things, such as the number of products manufactured or the number of errors logged). The term "acceptable" depends on the application environment and purpose of the guard band analysis. For example, "acceptable" may mean "not abnormal" or "not defective." A guard band channel can have upper and lower limits. In some embodiments, these limits can be calculated along the duration of the data stream using statistical methods. For example, a channel can represent ±3σ of the values ​​of multiple traces of a particular sensor, where the variance is calculated at each time value of the data stream. Smoothing techniques (e.g., time-of-day averaging) can make the channel more noise-resistant and smooth.

[0067] FTA guardbanding can be used for anomalous traces with minor variations in transient segments. FTA guardbanding can be used to analyze multiple traces or trace segments from a particular sensor across multiple runs. FTA guardbanding can establish upper and lower limits (e.g., 3σ) that indicate normal ranges or channels for sensor data over time. This solution identifies and profiles deviations from the guardbands. FTA capabilities can be used complementary with SFE to provide a comprehensive analysis as input to a fingerprint library. Guardbands can be associated with a single sensor (UVA) or across multiple sensors (multivariate analysis (MVA)). In the case of MVA, the sensor value (e.g., y-axis) can be a metric representing some combination of sensors involved in the MVA.

[0068] In some embodiments, the guard band may include multiple guard bands (e.g., a warning guard band that is completely contained within the error guard band). In some embodiments, a first zone is a normal region regardless of other parameters, a second zone is a region whose classification as good or bad is based on analysis of other parameters, and a third zone is a defective region regardless of other parameters.

[0069] In some embodiments, the x-axis parameter of the guard band, or the parameter that defines the ordering aspect of the data, is time (e.g., granularly related to the sensor reading rate). In some embodiments, the substrate number within the process tool can correspond to the substrate process.

[0070] In some embodiments, a guard band violation definition can be an anomaly, a defect, a warning, an event trigger, and / or a prediction. The determination and interpretation of a guard band violation can be related to the purpose of the guard band (e.g., defect or anomaly detection). A single data point outside the guard band channel can indicate a defect. A probability distribution function can be used to define the area, duration, and magnitude of the portion of the trace that contains the violation area. These violation areas can be evaluated (e.g., violation area attribute values) to interpret whether there is a defect.

[0071] 2A-2D are flowcharts of methods 200A-D associated with generating guard bands, according to certain embodiments. Figure 2A is a flowchart of a method 200A associated with generating guard bands, Figure 2B is a flowchart of a method 200B associated with using guard bands, Figure 2C is a flowchart of a method 200C associated with generating guard bands via machine learning, and Figure 2D is a flowchart of a method 200D associated with using guard bands via machine learning.

[0072] 2A , in some embodiments, at block 202, processing logic identifies trace data associated with fabrication of substrates (e.g., good substrates) through the substrate processing system having characteristic values ​​that meet a threshold. The trace data includes a separate set of sensor data over time for each of the substrates. In some embodiments, the trace data includes sensor data from multiple different sensors (e.g., different types of sensor data) for each of the substrates.

[0073] At block 204, processing logic determines allowable dispersion types (e.g., guard bands) based on the trace data.

[0074] In some embodiments, the processing logic generates guard bands based on the trace data. The guard bands may include upper and lower limits for defect detection (e.g., anomaly detection). In some embodiments, to form the guard bands, an average of all of the trace data is generated, and then an offset (e.g., 3σ) from the average is used as the upper and lower limits. Conventionally, the upper and lower limits are equally spaced from the average of the trace data.

[0075] Since the trace data is for a good substrate, any variation from the guard band formed by the trace data is an acceptable type of dispersion.

[0076] In some embodiments, block 204 includes processing logic that determines trace-to-trace variance in the trace data at block 210 (see FIGS. 5A-5B). In some embodiments, an acceptable variance type includes time-shifted trace-to-trace variance. The trace data may include sensor values ​​(e.g., y-axis) versus time (e.g., x-axis). Due to different times at which sensor data recording begins, some sets of sensor data over time may be misaligned (e.g., time shift on the x-axis). In some embodiments, the processing logic tracks differences between traces and autonomously adjusts guard band variance horizontally when normal time-shift variance is found. This prevents false positives when strategy steps or traces shift slightly. This increases robustness and reduces false positives.

[0077] In some embodiments, block 204 includes processing logic that determines, at block 212, an upper limit that is different from the lower limit based on the trace data (see FIG. 5C). In some embodiments, the allowable variance types include a first allowable variance type for forming an upper limit of the guard band and a second allowable variance type for forming a lower limit of the guard band. The first allowable variance type (e.g., amount) may be different from the second allowable variance type (e.g., amount). The variation on each side of the guard band is calculated separately (e.g., during a signal transition) to reduce false positives.

[0078] In some embodiments, block 204 includes processing logic that determines, at block 214, the position-dependent distribution of the trace data (see FIG. 5D ). Parameters and weights (e.g., duration, level, area, concatenation of consecutive violations, etc.) and other guard band settings can be adjusted depending on the position within the guard band (e.g., along the x-axis). Examples include: 1) adjusting the guard band along the x-axis to be more / less sensitive or adaptive to different behaviors along the process; 2) adjusting the guard band depending on signal characteristics (e.g., more conservative in areas where the guard band changes rapidly in the y-axis and more aggressive in areas where the guard band value is relatively constant); 3) adjusting the guard band depending on warnings detected along the guard band (e.g., adjusting distribution parameters after the first spike is detected to be more or less sensitive to future spike features on that trace); etc. This improves guard band performance and further enables the incorporation of subject matter expertise into guard band analysis.

[0079] In some embodiments, block 204 includes processing logic that determines, at block 216, a trace segmentation of the trace data (see FIG. 5E). In some embodiments, the acceptable variance type is further based on the segmentation of the portion of the trace data (e.g., associated with values ​​exceeding a threshold change, value changes within a threshold). For example, a feature (e.g., within a threshold) may trigger a wider guard band for the next segment of the guard band. In some embodiments, the acceptable variance type is further based on extraction of features from the trace data.

[0080] In some embodiments, block 204 includes processing logic that performs feature extraction (e.g., semi-automated feature extraction (SFE)) of the trace data at block 218. For example, guard band parameters can be varied for different segments or different features. This improves guard band performance and overall analysis by allowing different capabilities (e.g., guard bands, feature extraction, and / or trace segmentation) to be combined.

[0081] In some embodiments, the trace data includes sensor data from different types of sensors, and the allowable type of variance is via MVA. An MVA metric combines values ​​from multiple sensors into a single metric (e.g., the first principal component in a principal component analysis (PCA)). MVA techniques can be used to apply guard band parameters across multiple guard bands. Examples include: 1) determining violations and violation distributions according to violation characteristics across two or more traces that may or may not occur simultaneously; and / or 2) motion states associated with guard bands for a signal based on that signal relative to one or more other signals. This can improve guard band performance, further enable the incorporation of subject matter expertise into guard band analysis, and address correlation across multiple signals.

[0082] In some embodiments, the features include one or more of continuous features (e.g., slope and flat surface), user-defined features (e.g., concatenation of slope and gradient), intermittent features such as slope, flat surface, and FTA features, x-offset, y-offset, shape, length, distortion correction, etc.

[0083] At block 205, processing logic generates guard bands based on the allowable variance types. For example, the guard bands may allow for time shifting (e.g., in the x-axis), different upper and lower bounds, multivariate (e.g., multivariate) analysis, segmentation of portions of the trace data, extraction of features of the trace data, etc. In some embodiments, the guard bands are generated by training a machine learning model, as shown in FIG. 2C.

[0084] At block 208, processing logic performs corrective actions associated with the substrate processing system based on the guard bands. Block 208 may include comparing the additional trace data to the guard bands to determine whether to perform corrective actions (see, e.g., FIG. 2B).

[0085] 2B , in some embodiments, at block 222, processing logic identifies trace data associated with substrate fabrication through a substrate processing system (e.g., the same substrate processing system as in FIG. 2A ). The trace data may be associated with substrates for which it is not known whether characteristic data for the substrate meets a threshold (e.g., a good substrate) or does not meet a threshold (e.g., a bad substrate). The trace data includes a separate set of sensor data over time for each of the substrates. In some embodiments, the trace data includes sensor data from multiple different sensors (e.g., different types of sensor data) for each of the substrates.

[0086] At block 224, processing logic compares the trace data to guard bands generated based on the allowable dispersion types (see, e.g., block 206 of FIG. 2A). The guard bands may have upper and lower bounds on the data points over time.

[0087] At block 226, processing logic determines that one or more data points of the trace data are not within the guard bands. The one or more data points may include at least one data point above an upper limit and / or at least one data point below a lower limit (e.g., the data points do not match data points of a substrate having characteristic values ​​that meet a threshold).

[0088] At block 228, processing logic executes corrective actions associated with the substrate processing system. In some embodiments, the corrective actions include providing an alert, interrupting substrate processing equipment, inspecting the substrate, scrapping the substrate, updating manufacturing parameters, etc. In some embodiments, the performance of the corrective action is specific to the type or number of data points that fall outside the guard bands.

[0089] 2C, in some embodiments, at block 242, processing logic identifies historical trace data associated with the production of a substrate through a substrate processing system. In some embodiments, a substrate (e.g., a good substrate) has a characteristic value that meets a threshold. Block 242 may be similar to block 202 of FIG. 2A. In some embodiments, a substrate (e.g., a bad substrate) has a characteristic value that does not meet a threshold.

[0090] In some embodiments, processing logic identifies historical performance data associated with the historical trace data at block 244. In some examples, the historical performance data indicates whether a substrate is a good substrate or a bad substrate (e.g., all substrates have characteristic values ​​that meet a threshold, all substrates have characteristic values ​​that do not meet a threshold).

[0091] At block 246, processing logic trains a machine learning model with data inputs including historical trace data (e.g., target outputs including historical performance data) to generate a trained machine learning model that indicates guard bands associated with acceptable variance types. Training the machine learning model allows the guard bands to be based on the acceptable variance types described in block 204 of FIG. 2A.

[0092] The trained machine learning model can be used to determine whether additional trace data satisfies the guard bands (see, for example, FIG. 2D).

[0093] 2D, in some embodiments, at block 262, processing logic identifies trace data associated with substrate fabrication through a substrate processing system (e.g., the same substrate processing system as in FIG. 2C). The trace data may be associated with substrates for which it is not known whether characteristic data for the substrate meets a threshold (e.g., a good substrate) or does not meet a threshold (e.g., a bad substrate). Block 262 may be similar to block 242 of FIG. 2B.

[0094] At block 264, processing logic provides the trace data as input to a trained machine learning model (e.g., the trained machine learning model of diagram 246 in FIG. 2C ) associated with the guard bands generated based on the allowable variance types.

[0095] At block 266, processing logic receives output from the trained machine learning model indicating predicted data.

[0096] At block 268, processing logic determines, based on the prediction data, that one or more data points of the trace data are not within the guard bands of the trained machine learning model. The one or more data points may be above an upper limit of the guard band or below a lower limit of the guard band.

[0097] At block 270, processing logic performs corrective actions associated with the substrate processing system based on the predictive data. Block 270 may be similar to block 228 of Figure 2B.

[0098] 3A-3D are flowcharts of methods associated with guardband violation profiling, according to certain embodiments: FIG. 3A is a flowchart of a method 300A associated with determining a guardband violation shape characterization to classify guardband violation data points; FIG. 3B is a flowchart of a method 300B associated with classifying guardband violation data points based on the guardband violation shape characterization; FIG. 3C is a flowchart of a method 300C associated with training a machine learning model to classify guardband violation data points; and FIG. 3D is a flowchart of a method 300D associated with classifying guardband violation data points using the trained machine learning model.

[0099] 3A, in some embodiments, processing logic identifies trace data associated with fabrication of substrates (e.g., good substrates) through a substrate processing system having characteristic values ​​that meet a threshold value at block 302. Block 302 may be similar to block 202 of FIG.

[0100] At block 304, processing logic identifies guard bands associated with the trace data. The guard bands may be generated based on method 200A of Figure 2A or method 200C of Figure 2C.

[0101] At block 306, processing logic determines guard band violation data points of the trace data based on the guard bands. Guard band violation data points include data points of the trace data that are above an upper limit of the guard band and / or data points of the trace data that are below a lower limit of the guard band.

[0102] In some embodiments, the trace data is from different types of sensors and the determination of guard band violation data points is made via multivariate (eg, multivariate) analysis.

[0103] In some embodiments, determining guard band violation data points includes segmenting portions of the trace data associated with value changes that exceed a threshold change, hi some embodiments, determining guard band violation data points includes extracting features from the trace data.

[0104] At block 308, processing logic determines a guardband violation shape characterization based on the guardband violation data points (see Figures 6A-6E).

[0105] In some embodiments, the guard band violation shape characterization is a weighted combination of one or more of guard band violation duration (e.g., the number of sequential guard band violation data points that are outside the guard band limits), guard band violation magnitude (e.g., how much the guard band violation data points are above the upper limit or below the lower limit), guard band violation area (e.g., the area between a line passing through the guard band violation data points and the guard band limit), guard band violation position (e.g., the location of the guard band violation data points relative to the guard band limit), and / or guard band violation intermittency (e.g., how frequently the guard band violation data points exceed the upper guard band limit and / or the lower guard band limit).

[0106] In some embodiments, determining the guardband violation shape characterization includes concatenating consecutive violations into a single violation. By analyzing the behavior between consecutive violations, one can better understand whether the violations are related. Typical analysis can include the time between violations, the level of return to normal between violations, and the similarity of factors contributing to consecutive violations (e.g., x-shift). Concatenating consecutive violations improves guardband performance by identifying systemic issues rather than guardband violations. This reduces variability in guardband violation reporting and allows for the incorporation of subject matter expertise into guardband analysis.

[0107] Guardband violation shape characterization can be used to characterize any guardband feature in terms of parameters related to duration, level, area, etc. These parameters can be weighted to better capture specific violation types.

[0108] At block 310, processing logic performs corrective actions associated with the substrate processing system based on the guard band violation shape characterization. Block 310 may include classifying additional guard band violation data points of the additional trace data based on the guard band violation shape characterization and determining whether to perform corrective actions (e.g., see FIG. 3B).

[0109] 3B, in some embodiments, at block 322, processing logic identifies trace data associated with substrate fabrication through a substrate processing system (e.g., the same substrate processing system as in FIG. 3A). The trace data may be associated with substrates for which it is not known whether characteristic data for the substrate meets a threshold (e.g., a good substrate) or does not meet a threshold (e.g., a bad substrate). Block 322 may be similar to block 222 of FIG. 2B.

[0110] At block 324, processing logic identifies guard bands associated with the trace data. Block 324 may be similar to block 304 of Figure 3A.

[0111] At block 326, processing logic determines guard band violation data points of the trace data based on the guard bands. Guard band violation data points include data points of the trace data that are above an upper limit of the guard band and / or data points of the trace data that are below a lower limit of the guard band. Block 326 may be similar to block 306 of FIG. 3A.

[0112] At block 328, processing logic identifies a guard band violation shape characterization. The guard band violation shape characterization may be determined by block 308 of FIG. 3A. The guard band violation shape characterization may indicate whether a particular type of guard band violation (e.g., shape, area, duration, magnitude, location, intermittency, etc.) should be classified as anomalous (e.g., take corrective action) or not anomalous (e.g., do not take corrective action).

[0113] At block 330, processing logic determines a classification of the guardband violation data point based on the guardband violation shape characterization. In some embodiments, the classification indicates whether the guardband violation data point is anomalous. In some embodiments, the classification indicates a type of anomaly associated with the guardband violation data point. In some embodiments, the classification indicates a type of corrective action to be performed in connection with the guardband violation data point.

[0114] At block 332, processing logic performs a corrective action associated with the substrate processing system based on the classification. The performing of the corrective action at block 332 may be similar to the performing of the corrective action at block 228 of FIG. 2B.

[0115] 3C, in some embodiments, processing logic identifies trace data associated with fabrication of substrates (e.g., good substrates) through the substrate processing system having characteristic values ​​that meet a threshold at block 342. Block 302 may be similar to block 202 of FIG. 2A, block 242 of FIG. 2C, and / or block 302 of FIG. 3A.

[0116] At block 344, processing logic identifies historical performance data associated with the historical trace data. In some examples, the historical performance data indicates whether the substrates are good or bad (e.g., all substrates have characteristic values ​​that meet a threshold, all substrates have characteristic values ​​that do not meet a threshold). Block 344 may be similar to block 244 of FIG. 2C.

[0117] At block 346, processing logic identifies guard bands associated with the trace data. The guard bands may be generated based on method 200A of Figure 2A or method 200C of Figure 2C. Block 346 may be similar to block 304 of Figure 3B.

[0118] At block 348, processing logic determines past guard band violation data points of the past trace data based on the guard bands. Past guard band violation data points include data points of the past trace data that are above an upper guard band limit and / or data points of the trace data that are below a lower guard band limit. Block 348 may be similar to block 306 of FIG. 3A.

[0119] At block 350, processing logic trains a machine learning model with data inputs (e.g., target outputs including historical performance data) including past guardband violation data points and generates a trained machine learning model associated with a guardband violation shape characterization to classify additional guardband violation data points. The trained machine learning model can be used in accordance with FIG. 3D . In some embodiments, the guardband violation data points and the violation shape characterization (e.g., shape summary statistics) are provided to the trained machine learning model to classify the guardband violation data points. In some embodiments, the guardband violation shape characterization is determined based on the guardband violation data points (e.g., see block 308 of FIG. 3B ).

[0120] 3D, in some embodiments, at block 362, processing logic identifies trace data associated with substrate fabrication through a substrate processing system (e.g., the same substrate processing system as in FIG. 3C). The trace data may be associated with substrates for which it is not known whether characteristic data for the substrate meets a threshold (e.g., a good substrate) or does not meet a threshold (e.g., a bad substrate). Block 362 may be similar to block 342 of FIG. 3B.

[0121] At block 364, processing logic identifies guard bands associated with the trace data. Block 364 may be similar to block 304 of Figure 3A, block 324 of Figure 3B, and / or block 344 of Figure 3C.

[0122] At block 366, processing logic determines guard band violation data points of the trace data based on the guard bands. Guard band violation data points include data points of the trace data that are above an upper limit of the guard band and / or data points of the trace data that are below a lower limit of the guard band. Block 366 may be similar to block 306 of Figure 3A, block 326 of Figure 3B, and / or block 346 of Figure 3C.

[0123] At block 368, processing logic provides the guardband violation data points as input to a trained machine learning model (e.g., trained via block 350 of FIG. 3C ) associated with a guardband violation shape characterization. In some embodiments, the guardband violation data points and the guardband violation shape characterization (e.g., shape summary statistics) are provided as input to the trained machine learning model to classify the guardband violation data points. In some embodiments, the guardband violation shape characterization is determined based on the guardband violation data points (e.g., see block 308 of FIG. 3B ).

[0124] At block 370, processing logic receives output from the trained machine learning model indicating predicted data.

[0125] At block 372, processing logic determines a classification of one or more data points of the trace data based on the prediction data. The classification at block 372 may be similar to the classification at block 330 of FIG. 3B.

[0126] At block 384, processing logic performs corrective actions associated with the substrate processing system based on the classification. Block 384 may be similar to block 332 of Figure 3B.

[0127] 4A-4D are flow diagrams of methods associated with dynamic acceptable areas outside of guard band limits (e.g., time-dependent variations in trace data) in accordance with certain embodiments. Figure 4A is a flow diagram of a method 400A associated with determining an acceptable area outside of the guard band limits, Figure 4B is a flow diagram of a method 400B associated with using and possibly adjusting the acceptable area outside of the guard band limits, Figure 4C is a flow diagram of a method 400C associated with training a machine learning model to determine an acceptable area outside of the guard band limits, and Figure 4D is a flow diagram of a method 400D associated with using and possibly adjusting the acceptable area outside of the guard band limits using the trained machine learning model.

[0128] The trace data for a good wafer may vary over time with acceptable drift, fluctuations, noise, spikes, etc.

[0129] 4A, in some embodiments, processing logic identifies trace data associated with fabrication of substrates (e.g., good substrates) through a substrate processing system having characteristic values ​​that meet a threshold at block 402. Block 302 may be similar to block 202 of FIG. 2A and / or block 302 of FIG. 3A.

[0130] At block 404, processing logic determines dynamic acceptable areas outside the guard band limits based on the trace data.

[0131] In some embodiments, processing logic determines upper and lower guard band limits based on the trace data (e.g., via block 206 of FIG. 2A , block 246 of FIG. 2C , 3σ from the mean of the trace data, etc.) The area between the upper and lower guard band limits is a safe area (e.g., a green area), and data points within this area are considered healthy.

[0132] The processing logic determines acceptable areas outside the guard band limits. The acceptable areas can be warning areas (e.g., yellow areas) where data points are still normal and are used to track the motion state of the substrate processing system. Abnormal areas (e.g., red areas) are outside the acceptable areas, and data points within the abnormal areas are considered positive (e.g., abnormal).

[0133] The acceptable zone may be determined via user input (e.g., 1 sigma outside the guard band limits, 4 angstroms of sensor data variation, 4% sensor data variation, etc.). The acceptable zone may vary over time (e.g., a dynamic acceptable zone). In some embodiments, a predetermined amount of variation in sensor values ​​from the guard band limits may be acceptable for a predetermined amount of time (e.g., a predetermined amount of runs). For example, for 10 runs, a 4 angstrom or 4% change in sensor values ​​from the guard band limits may be acceptable.

[0134] The dynamic acceptable area and / or guard band limits can change over time. For example, for a given amount of runs (e.g., 10 runs), the acceptable area and / or guard band limits can be adjusted by a given amount (e.g., increased by 4 Angstroms, widened by 4%, etc.). As the dynamic acceptable area and / or guard band limits change over time, new acceptable areas can be updated.

[0135] At block 406, processing logic executes corrective actions associated with the substrate processing equipment based on the dynamic acceptable area outside the guard band limits. Execution of the corrective action may be based on the additional trace data being outside the acceptable area (see FIG. 4B).

[0136] 4B, in some embodiments, at block 422, processing logic identifies trace data associated with substrate fabrication through a substrate processing system (e.g., the same substrate processing system as in FIG. 4A). The trace data may be associated with substrates for which it is not known whether characteristic data for the substrate meets a threshold (e.g., a good substrate) or does not meet a threshold (e.g., a bad substrate). Block 422 may be similar to block 222 of FIG. 2B and / or block 322 of FIG. 3B.

[0137] At block 424, processing logic compares the trace data to a dynamic acceptable area outside the guard band limits. The dynamic acceptable area may be determined by block 404 of Figure 4A.

[0138] At block 426, processing logic updates the dynamic acceptable area outside the guard band limits based on the trace data in response to one or more data points of the trace data being within the dynamic acceptable data range.

[0139] In block 428, processing logic executes a corrective action associated with the substrate processing equipment in response to one or more data points of the trace data being outside of a range of dynamic acceptable data. The execution of the corrective action in block 428 may be similar to the execution of the corrective action in block 228 of FIG. 2B and / or block 332 of FIG. 3B.

[0140] 4C, in some embodiments, processing logic identifies trace data associated with fabrication of substrates (e.g., good substrates) through the substrate processing system having characteristic values ​​that meet a threshold at block 442. Block 302 may be similar to block 202 of FIG. 2A, block 242 of FIG. 2C, block 302 of FIG. 3A, block 342 of FIG. 3C, and / or block 402 of FIG. 4A.

[0141] At block 444, processing logic identifies historical performance data associated with the historical trace data. In some examples, the historical performance data indicates whether the substrates are good or bad (e.g., all substrates have characteristic values ​​that meet a threshold, all substrates have characteristic values ​​that do not meet a threshold). Block 444 may be similar to block 244 of FIG. 2C and / or block 344 of FIG. 3C.

[0142] At block 446, processing logic trains a machine learning model with data inputs including historical trace data (e.g., target outputs including historical performance data) to generate a trained machine learning model that indicates dynamic acceptability areas outside the guard band limits. The trained machine learning model can be used by FIG. 4D.

[0143] 4D, in some embodiments, at block 462, processing logic identifies trace data associated with substrate fabrication through a substrate processing system (e.g., the same substrate processing system as in FIG. 4C). The trace data may be associated with substrates for which it is not known whether characteristic data for the substrate meets a threshold (e.g., a good substrate) or does not meet a threshold (e.g., a bad substrate). Block 362 may be similar to block 442 of FIG. 4B.

[0144] At block 464, processing logic provides the trace data as input to a trained machine learning model associated with the dynamic acceptable area outside the guard band limits (e.g., the trained machine learning model of diagram 446 in FIG. 4C ).

[0145] At block 466, processing logic receives output from the trained machine learning model indicating predicted data.

[0146] At block 468, processing logic updates the dynamic acceptable area outside the guard band limits in response to determining, based on the predicted data, that one or more data points of the trace data are within the acceptable area. Block 468 may be similar to block 426 of FIG. 4B.

[0147] In block 470, processing logic performs corrective action associated with the substrate processing system in response to determining, based on the predicted data, that one or more data points of the trace data are outside of an acceptable zone. Block 470 may be similar to block 428 of FIG. 4B.

[0148] 5A-5E show graphs 500A-E of acceptable variance types (see, e.g., FIGS. 2A-2D), according to certain embodiments. 5A-5B show graphs 500A-B of trace-to-trace variation. 5C shows a graph 500C of an upper bound that is different from a lower bound. 5D shows a graph 500D of position-dependent variance. 5E shows a graph 500E of trace segmentation.

[0149] 5A, graph 500A shows trace data 502A-B. Trace data 502A can be sensor data (e.g., from one or more sensors) for fabrication of a first substrate (e.g., one or more first substrates), and trace data 502B can be sensor data (e.g., from the same one or more sensors) for a second substrate (e.g., one or more second substrates). The capture of trace data 502B can occur earlier in the substrate fabrication operation than the capture of trace data 502A, which can cause a shift in the x-direction (e.g., the x-axis is time and the y-axis is sensor value, with trace data 502A rising earlier than trace data 502B). This is referred to as variation (e.g., variance) between the traces.

[0150] Referring to FIG. 5B , graph 500B shows trace data over time associated with the production of many substrates. Guard band 504A, which does not account for trace-to-trace variation, would result in many false positives (e.g., many guard band violating data points that do not correspond to bad substrates). In some examples, line 506 is calculated by taking the average of the trace data, and guard band 504A is created by taking 3σ from line 506, resulting in guard band 504A moving evenly away from line 506 over time. Trace-to-trace variation would cause many false positives for guard band 504A, and simply widening guard band 504A would create many false negatives (e.g., if guard band 504A were widened, deviating data points in the y-direction would not be captured).

[0151] Guard bands 504B that account for trace-to-trace variation are generated by method 200A of FIG. 2A or method 200C of FIG. 2C. Trace data from good substrates is used to determine the type of variation that is acceptable. For example, initial guard bands 504A can be created from trace data from good substrates, and then, because the substrates all have characteristic values ​​that meet the threshold, guard bands 504A are expanded in the x-direction (e.g., expanded horizontally) within the area to form guard bands 504B to accommodate trace-to-trace variation.

[0152] Referring to FIG. 5C, a graph 500C shows an upper limit graph 500C that is different from the lower limit.

[0153] 5B, guard band 504A may have upper and lower limits equally spaced (e.g., 3σ) from a line 506 that passes through the mean of the trace data. Trace data from good wafers may have different amounts of variance above and below line 506. Simply spacing the upper and lower limits of guard band 504A equally apart can result in many false positives.

[0154] 2A or 2C, guard band 504B is generated that accounts for different upper and lower limits. For example, as shown in FIG. 5C, at the beginning of a transition, an upper variance 508A may be greater than a lower variance 508B, and at the end of a transition, a lower variance 508D may be greater than an upper variance 508C.

[0155] Referring to FIG. 5D, a graph 500D shows a graph 500D of position dependent variance.

[0156] 5B, the guard band 504A may have upper and lower bounds (e.g., 3σ) at the same distance over time from a line 506 that passes through the mean of the trace data. Trace data from a good wafer may have different amounts of variance over different portions of time. Having guard bands 504A of the same size over time may result in many false positives.

[0157] 2A or 200C generates guard bands 504B that account for different amounts of variance 510 over time. For example, as shown in FIG. 5D, lower variance 510A may occur in the plateau region (e.g., a smaller distance between the upper and lower limits of guard band 504B for a good substrate), and higher variance 510B may occur in the transition region (e.g., a larger distance between the upper and lower limits of guard band 504B for a good substrate).

[0158] 5E, graph 500E shows a graph 500E of a trace segmentation. As discussed in FIG. 5B, guard band 504A may have upper and lower bounds at the same distance (e.g., 3σ) over time from line 506 that passes through the mean of the trace data. In some embodiments, the change in the trace data is greater than a threshold amount (e.g., see area 512 in FIG. 5E).

[0159] If two boundaries are close to each other, the center position can be used. If one of the boundaries is close to a boundary that has more than a threshold amount of change, that segment boundary can be kept.

[0160] The two segments of the trace data that indicate an abrupt change in the original boundary cannot be removed from the trace data. Based on the segmentation knowledge of the abrupt changes in the segmentation points in area 512, the guard band parameters can be adjusted.

[0161] The abrupt change in area 512 may be one or more of the following:

[0162] 1) A change in the slope of a segment of trace data from negative to positive or positive to negative (e.g., slope(right)*slope(left)<0),

[0163] 2) The slope of the right and / or left segment exceeds a threshold slope (e.g., abs(slope(right))>0.1 or abs(slope(left))>0.1 (normal values)), and / or

[0164] 3) The mean variance (e.g., standard deviation) of two connecting segments is a threshold amount greater than the mean variance (e.g., standard deviation) of the next two segments (e.g., mean(std[right,left])>2*mean(std[right_2,left_2]).

[0165] If two adjacent segments of trace data have 1), 2), and 3) from above, the segment boundary (e.g., data points within area 512) can be considered to be a fixed boundary. In some embodiments, when area 512 is a segment boundary, guard band 504B is generated for data points within area 512 rather than removing those points from the generation of guard band 504B. In some embodiments, when area 512 is a segment boundary, data points within area 512 become the limit of guard band 504B with a smaller allowable variance.

[0166] 6A-6E illustrate guard band violation profiling (see, eg, FIGS. 3A-3D) in accordance with certain embodiments.

[0167] Referring to FIG. 6A, each of graphs 602A-Z shows trace data (e.g., block 302 in FIG. 3A, block 344 in FIG. 3C). Each graph 602A-Z can show trace data for the production of a different substrate, with each substrate being a good substrate (e.g., having characteristic values ​​that meet thresholds). Each graph 602A-Z also shows guard bands, with each graph 602A-Z having several sets of data points that fall outside the guard band limits. These sets of data points are referred to as sets of guard band violation data points 604A-Z (e.g., sets of data points associated with guard band violations). Because the trace data is for a good substrate, the sets of guard band violation data points 604A-Z violate the guard bands but do not indicate a bad substrate (e.g., they are false positives).

[0168] For each set of guard band violation data points 604 of the trace data, parameters 606 are extracted. The parameters 606 may include an area, a duration, a magnitude, etc. In some examples, the area may be the area between the guard band limit and a portion of a line passing through the trace data that is outside the guard band limit. In some examples, the duration may be the amount of time that sequential guard band violation data points are outside the guard band limit. The magnitude may be the magnitude (e.g., y value) of the difference (e.g., in the y direction) between the guard band violation data point and the guard band limit. In some examples, each guard band violation data point may have a corresponding parameter input.

[0169] The parameters 606 and the joint probability density function of the parameters 606 can be used to form a graph 608. The graph 608 can be generated by fitting a multivariate Gaussian distribution (e.g., finding the joint probability density function of the three variables). The set of guardband violation data points 604 can form a guardband violation shape characterization 610 (e.g., a circle on the graph 608) that surrounds the set of guardband violation data points 604 (e.g., good guardband violation data points 614A).

[0170] Graph 608, including guardband violation shape characterization 610, can be used to generate graph 612. Graph 612 displays guardband violation shape characterization 610 separating good guardband violation data points 614A, corresponding to good substrates, from bad guardband violation data points 614B, corresponding to bad substrates.

[0171] In response to the set of guardband violation data points 604 being for a good substrate, the guardband violation shape characterization 610 surrounds (e.g., surrounds) the data points for the good substrate in the graph 608. The guardband violation shape characterization 610 can be used to determine whether a future set of guardband violation data points 604 is a good guardband violation data point 614A corresponding to a good substrate (e.g., within the range of the guardband violation shape characterization 610 on the graph 608, below the guardband violation shape characterization 610 on the graph 612) or a good guardband violation data point 614A corresponding to a non-good substrate (e.g., outside the range of the guardband violation shape characterization 610 on the graph 608, above the guardband violation shape characterization 610 on the graph 612).

[0172] In some embodiments, the set of guardband violation data points 604 used for parameters 606, graph 608, and graph 612 corresponds to a bad substrate (e.g., a substrate having a property value that does not meet a threshold). Guardband violation shape characterization 610 then surrounds (e.g., surrounds) the data points of the bad substrate in graph 608. Guardband violation shape characterization 610 can be used to determine whether a future substrate is bad (e.g., within the range of guardband violation shape characterization 610 on graph 608, below guardband violation shape characterization 610 on graph 612) or not bad (e.g., outside the range of guardband violation shape characterization 610 on graph 608, above guardband violation shape characterization 610 on graph 612).

[0173] In response to the set of guardband violation data points 604 being for a particular type of substrate (e.g., a particular type of bad substrate having a particular characteristic value that does not meet a threshold), the guardband violation shape characterization 610 surrounds (e.g., surrounds) the data points of the particular type of substrate. The guardband violation shape characterization 610 can be used to determine whether a future substrate is a particular type of substrate (e.g., within the range of the guardband violation shape characterization 610 on graph 608, below the guardband violation shape characterization 610 on graph 612) or is not a particular type of substrate (e.g., outside the range of the guardband violation shape characterization 610 on graph 608, above the guardband violation shape characterization 610 on graph 612).

[0174] 6B shows a graph 620 for guard band violation profiling, according to certain embodiments. Graph 620 includes a band average 622 (e.g., average trace data), a guard band limit 624, an offset 626 between the guard band limit 624 and the band average 622, and a guard band violation data point 628. Graph 620 has a gap 638 between a guard band violation data point 628A for a first instance and a guard band violation data point 628B for a second instance.

[0175] The violation duration 630 of a guard band violation data point 628 is the distance (e.g., the duration of the violation in the x-direction) between the first guard band violation data point 628A outside the guard band limits 624 and the last guard band violation data point 628B outside the guard band limits 624.

[0176] The magnitude 632 of the violation (e.g., maximum violation, peak violation) is the distance between the guardband limit 624 and the guardband violation data point 628 (e.g., the guardband violation data point 628 farthest from the guardband limit 624 in the y-direction).

[0177] The violation area 634 is the area between a line passing through the guardband violation data points and the guardband limit 624 (eg, the average value of the violation).

[0178] The violation location 636 can be the location of the maximum violation magnitude 632 (eg, the x value, time value at which the violation occurs).

[0179] The intermittency of a violation may include the joining or splitting of two or more violations.

[0180] The guardband violation shape characterization 610 may be further based on one or more of a violation duration 630, a violation magnitude 632, a violation area 634, and / or a violation location 636. A guardband violation data point 628 corresponding to a violation duration 630, a violation magnitude 632, a violation area 634, and / or a violation location 636 that meets a threshold may be ignored. A guardband violation data point 628 corresponding to a violation duration 630, a violation magnitude 632, a violation area 634, and / or a violation location 636 that meets a threshold may correspond to a substrate type (e.g., a good substrate, a bad substrate, a type of bad substrate, etc.).

[0181] 6C shows a graph 640 for guard band violation profiling, according to certain embodiments. Graph 640 includes a band average 622 (e.g., average trace data), guard band limits 624, an instance of a guard band violation data point 628, a gap 638 between two instances of a guard band violation data point 628, and a violation magnitude 632 between the band average 622 and the last guard band violation data point 628 of the first instance. A line 642 is located between the data point of the band average 622 corresponding to the last guard band violation data point of the first instance, guard band violation data point 628C, and the first guard band violation data point of the second instance, guard band violation data point 628D.

[0182] The guardband violation shape characterization 610 can classify the substrate or trace data as good, bad, or one type of bad based on the number of violations. In response to the length of the line 642 meeting a threshold (e.g., being large enough), the guardband violation data point 628C of the first instance and the guardband violation data point 628D of the second instance are considered to be a single violation; otherwise, the guardband violation data point 628C of the first instance and the guardband violation data point 628D of the second instance are considered to be separate violations.

[0183] 6D illustrates a flow diagram of a method 660 for guard band violation profiling, according to certain embodiments. Classification of guard band violation data points can be performed via multivariate (e.g., multivariate) analysis (e.g., based on sensor data from different types of sensors).

[0184] In some embodiments, trace data from sensors 662A-N is compared separately to corresponding guard bands 664A-N. Guard band violation data points from a comparison of trace data from sensors 662 compared to guard bands 664 via violation probability density function (pdf) 666 are processed by comparing them to a threshold 668 to provide test trace data 670.

[0185] The test trace data provides a score 672 for each sensor 662. Based on the score 672, a score ranking 674 is performed. Based on the score ranking 674, a sensor importance ranking 676 is performed. Based on the sensor importance ranking 676, a composite score 677 is generated. The composite score 677 is compared to a composite threshold 678 to provide a defect detection result 679 (e.g., classification of guard band violation data points as for a good or bad substrate, anomaly detection result).

[0186] 6E illustrates a flow diagram of a method 680 for guardband violation profiling, according to certain embodiments. Guardband violation shape characterization can be based on guardband violation data points and segmentation feature extraction.

[0187] At block 682, trace data is identified. This may be similar to block 302 of Figure 3A and / or block 342 of Figure 3C for generating guard bands. This may be similar to block 322 of Figure 3B and / or block 362 of Figure 3D for using guard bands.

[0188] At block 684, a full trace analysis is performed on the trace data from block 682. For example, area, duration, magnitude, etc. may be determined (see, eg, FIGS. 6A-6C).

[0189] At block 686, a guard band model is generated based on the whole trace analysis of the trace data, which may be similar to block 206 of Figure 2A and / or block 246 of Figure 2C.

[0190] At block 688, abnormal features (e.g., guard band violation data points) are determined by comparing the trace data to a guard band model (e.g., guard bands). This may be similar to block 306 of Figure 3A, block 326 of Figure 3B, block 346 of Figure 3C for training a model, and / or block 366 of Figure 3D. The abnormal features (e.g., guard band violation data points) may be persistent or intermittent.

[0191] At block 690, segmentation feature extraction is performed on the trace data of block 682. This may be similar to Figure 5E (e.g., segment boundaries, such as data points within area 512 of Figure 5E, are considered fixed boundaries). In some embodiments, a guard band model may create anomaly features of block 688 and combine them with features identified in block 690 (e.g., persistent features of block 692 and intermittent features of block 692).

[0192] At block 692, persistent features of the trace data are determined based on the segmented feature extraction of block 690. Persistent features may be features of the trace data that meet a threshold amount of occurrence.

[0193] At block 694, feature parameter correlation of the persistent features of the trace data is performed (e.g., based on subject matter expertise 699, such as user input). Feature parameter correlation may be based on parameters such as magnitude, location, area, duration, etc. (See FIG. 6B). In some embodiments, guard band violation shape characterization may classify guard band violation data points corresponding to persistent features as being associated with a good substrate.

[0194] At block 696, intermittent features of the trace data are determined based on the segmented feature extraction of block 690. Intermittent features may be features of the trace data that do not meet a threshold amount of occurrence.

[0195] At block 698, feature presence correlation is performed based on the intermittent features from block 696, the anomaly features from block 688, and / or subject matter expertise 699. In some embodiments, the guard band violation shape characterization can classify guard band violation data points corresponding to intermittent features as being associated with a bad substrate.

[0196] 7A-7F illustrate dynamic acceptable areas outside the guard band limits according to certain embodiments.

[0197] FIG. 7A illustrates a flow diagram of a method 700 associated with dynamic areas outside the guard band limits.

[0198] At block 702, processing logic identifies baseline trace data. This may be similar to block 302 of Figure 3A and / or block 342 of Figure 3C for generating guard bands. This may be similar to block 322 of Figure 3B and / or block 362 of Figure 3D for using guard bands.

[0199] At block 704, processing logic identifies guard bands based on the trace data. The guard bands may be generated by block 206 of Figure 2A and / or block 246 of Figure 2C.

[0200] At block 706, processing logic performs a multivariate (e.g., multivariate) analysis (MVA) distribution based on the guard band and baseline trace data.

[0201] At block 708, processing logic identifies a probability on the baseline trace data, which may be the probability that the data point is in a safe zone (e.g., zone 736 of graph 734 or graph 742 in FIG. 7B), a warning zone (e.g., zone 738 of graph 734 or graph 742 in FIG. 7B), or an abnormal zone (e.g., zone 740 of graph 734 or graph 742 in FIG. 7B).

[0202] At block 710, processing logic generates an internal threshold value. The internal threshold value may be a line that separates a warning zone (e.g., zone 738 of graph 734 or graph 742 in FIG. 7B) from an abnormal zone (e.g., zone 740 of graph 734 or graph 742 in FIG. 7B).

[0203] At block 712, processing logic generates extended trace data. The extended trace data may be simulated trace data formed by adjusting the baseline trace data of block 702 by one or more of blocks 714-720.

[0204] The expanded trace data may include trace data with minor drift at block 714. The drift may include increasing sensor values ​​in the y-direction of the baseline trace data of block 702.

[0205] The extended trace data may include trace data with minor repetitions at block 716. One or more portions of the baseline trace data of block 702 may be repeated over time (e.g., in the x-direction).

[0206] At block 718, the expanded trace data may include trace data with minor noise (e.g., and / or fluctuations). The baseline trace data of block 702 may be adjusted (e.g., increased and decreased) in the y-direction to mimic the noise.

[0207] The extended trace data may include trace data with minor spikes at block 720. The baseline trace data at block 702 may include peaks and valleys to mimic minor spikes.

[0208] At block 722, processing logic identifies probabilities on the baseline trace data and the extended trace data. Block 722 may be similar to block 708.

[0209] At block 724, processing logic generates an outer threshold value. Block 724 may be similar to block 710.

[0210] 7B shows the dynamic area outside the guard band limits. Graph 730 shows the baseline trace data (e.g., of block 702), and graph 732 shows the extended trace data (e.g., of block 712).

[0211] Graph 734 shows area 736 (e.g., acceptable area, green area), area 738 (e.g., caution area, yellow area), and area 740 (e.g., abnormal area). Graph 734 may be formed based on trace data from a good substrate. All of the data points on graph 734 may fall within area 736 or area 738.

[0212] Graph 742 shows new trace data for a substrate for which it is to be determined whether the substrate is good or bad. Graph 742 has trace data in area 736, area 738, and area 740. Data points in area 740 correspond to bad substrates. Area 738 (e.g., the yellow area, the warning area) should be recalculated based on the data points in area 738.

[0213] FIG. 7C illustrates a flow diagram of a method 744 associated with dynamic areas outside the guard band limits.

[0214] At block 746, processing logic identifies a training set of trace data. The training set may be previous trace data used to prepare substrates by substrate processing equipment.

[0215] At block 748, processing logic identifies new trace data. In some embodiments, the new trace data includes new sensor data associated with creating a new substrate with the same or different substrate processing equipment as in block 746. In some embodiments, the new trace data includes simulated trace data created based on the trace data of block 746 and includes one or more of drift, noise, spikes, fluctuations, etc.

[0216] At block 750, processing logic ranks the new trace data from block 748. The new trace data may be ranked based on its representation (e.g., proximity) of the trace data of block 746. The new trace data may be ranked based on its proximity to each other (e.g., to remove anomalies). Data points of the new trace data may be ranked as good (e.g., green area), acceptable and used to adjust guard bands (e.g., yellow area), or anomalous (e.g., red area). For example, a data point of the new trace data may be within the green area (e.g., no input guard band adjustment), within the acceptable yellow area (e.g., no anomaly but input for GB adjustment), or within the anomalous red area (e.g., anomalous but no input to guard band adjustment).

[0217] At block 752, processing logic selects at least a portion of the new trace data from block 748 (e.g., based on the rankings from block 750). Processing logic may select the highest ranked trace data based on the rankings of block 750.

[0218] At block 754, processing logic updates (e.g., see FIG. 7E) or retrains (e.g., see FIG. 7F) old guard bands based on the training set and / or the trace data selected from block 752. Processing logic can trigger a guard band update when certain criteria are met (e.g., the trace data has noise, there is no drift in the trace data, etc.).

[0219] At block 756, processing logic identifies the new guard bands that were updated or retrained from block 754.

[0220] 7D shows the dynamic area outside the guard band limits. Graph 760 shows new trace data (e.g., from block 748 of FIG. 7C), and graph 762 shows the trace ranking of the trace data in graph 760 (e.g., see block 750 of FIG. 7C).

[0221] Graph 764 shows the selected trace (e.g., from block 752 of FIG. 7C), and graph 766 shows the adapted guard band (e.g., the new guard band from block 756 of FIG. 7C) updated from the selected trace (e.g., the most informative trace) from graph 764.

[0222] 7E illustrates a dynamic area outside the guard band limits for noise. Block diagram 770A illustrates that process data (e.g., trace data) for generating the guard bands is accumulated. In response to processing logic detecting noise (e.g., periodic changes), processing logic updates the guard bands with all previous data.

[0223] Graph 772A shows the initial trace data (eg, the first 60 traces), and graph 774A shows the initial guard band for the trace data of graph 772A.

[0224] Graph 776A shows the trace data (e.g., the first 120 traces), and graph 778A shows the guard band next to the trace data of graph 776A. As shown in graph 778A, the guard band becomes wider over time to accommodate an increase in acceptable noise.

[0225] 7F illustrates a dynamic region outside the guard band limits for drift. Block diagram 770A illustrates that older past data (e.g., previous trace data) is forgotten and only a certain amount (e.g., zero or more) of more recent past data is used to generate the guard band in combination with new process data (e.g., current trace data). In response to processing logic detecting drift (e.g., a change in the sensor value in the y-direction over time), processing logic triggers the forgetting mechanism and updates the guard band with only recent trace data.

[0226] Graph 772B shows the initial trace data (eg, the first 50 traces), and graph 774B shows the initial guard band for the trace data of graph 772B.

[0227] Graph 776B shows the trace data (e.g., the next 50 traces), and graph 778B shows the next guard band for the trace data of graph 776B. As shown in graph 778B, the guard band moves (e.g., increases) in the y-direction over time to accommodate the acceptable drift.

[0228] 8A-8B illustrate guard band adaptation, according to certain embodiments. The guard band adaptation shown in any portion of FIGS. 8A-8B can be used in any of the methods of the present disclosure (e.g., FIGS. 2A-2D, 3A-3D, and / or 4A-4D) to adapt the guard bands to the system's motion conditions (e.g., not to adapt to violations such as fault level shifts).

[0229] 8A illustrates horizontal scaling. Graph 810A shows trace data 812A and trace data 812B. Trace data 812A and 812B may have different scaling in the y direction. As shown in graph 810B, trace data 812A and / or trace data 812B may be subject to horizontal scaling (e.g., horizontal scaling of a maneuver endpoint).

[0230] In some embodiments, trace data 812A and 812B are for a good substrate. By horizontally scaling, more accurate guard bands can be created based on trace data 812A and 812B. By horizontally scaling trace data that may correspond to a good or bad substrate, differences between the trace data and the guard bands can be more accurately identified (e.g., reducing false positives and false negatives).

[0231] 8B illustrates vertical scaling and horizontal distortion correction. Graph 850A shows trace data 812A and trace data 812B with different scaling in the y-direction. Vertical scaling of trace data 812A and 812B of graph 850A can be performed (e.g., amplitude normalization) to generate graph 850B. Horizontal distortion correction of trace data 812A and 812B of graph 850B can be performed (e.g., dynamic time distortion correction) to generate graph 850C.

[0232] Trace distortion and scaling allows out-of-phase and different amplitude contributions to be ignored. Horizontal distortion is applied to ignore out-of-phase contributions and preserve vertical noise. Vertical and horizontal scaling is applied to address region transfer (e.g., applying guard bands to different regions of different magnitudes).

[0233] 9 is a block diagram illustrating a computer system 900, according to certain embodiments. In some embodiments, computer system 900 can be connected to other computer systems (e.g., via a network such as a local area network (LAN), an intranet, an extranet, or the Internet). Computer system 900 can operate in the capacity of a server or a client computer in a client-server environment, or as a peer computer in a peer-to-peer or distributed network environment. Computer system 900 can be provided by a personal computer (PC), a tablet PC, a set-top box (STB), a personal digital assistant (PDA), a cellular phone, a web appliance, a server, a network router, switch, or bridge, or any device capable of executing (sequentially or otherwise) a set of instructions that specify actions to be taken by that device. Furthermore, the term "computer" is intended to include any group of computers that individually or jointly execute a set (or sets) of instructions to perform any one or more of the methods described herein.

[0234] In a further aspect, the computer system 900 may include a processing device 902, a volatile memory 904 (e.g., random access memory (RAM)), a non-volatile memory 906 (e.g., read-only memory (ROM) or electrically erasable programmable ROM (EEPROM)), and a data storage device 918, which may communicate with each other via a bus 908.

[0235] The processing device 902 may be provided by one or more processors, such as a general-purpose processor (e.g., a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a microprocessor implementing other types of instruction sets, or a microprocessor implementing a combination of instruction set types), or a specialized processor (e.g., an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), or a network processor).

[0236] Computer system 900 may further include a network interface device 922 (e.g., coupled to a network 974). Computer system 900 may also include a video display unit 910 (e.g., an LCD), an alphanumeric input device 912 (e.g., a keyboard), a cursor control device 914 (e.g., a mouse), and a signal generating device 920.

[0237] In some embodiments, the data storage device 918 may include a non-transitory computer-readable storage medium 924 (e.g., a non-transitory machine-readable storage medium) that may store instructions 926 encoding any one or more of the methods or functions described herein, the instructions 926 including instructions for encoding the components of FIG. 1 (e.g., the prediction component 114, the model 190 used to predict or detect, etc.) and for performing the methods described herein. These instructions, when executed, may cause the processing device to perform the methods described herein.

[0238] The instructions 926 may also reside, completely or partially, within the volatile memory 904 and / or the processing device 902 during execution thereof by the computer system 900, and thus the volatile memory 904 and the processing device 902 may also constitute machine-readable storage media.

[0239] While the illustrative example shows the computer-readable storage medium 924 as a single medium, the term "computer-readable storage medium" is intended to include a single medium or multiple media (e.g., centralized or distributed databases and / or associated caches and servers) that store one or more sets of executable instructions. The term "computer-readable storage medium" is also intended to include any tangible medium that is capable of storing or encoding a set of instructions, for execution by a computer, that cause a computer to perform any one or more of the methodologies described herein. The term "computer-readable storage medium" is intended to include, but is not limited to, solid-state memory, optical media, and magnetic media.

[0240] The methods, components, and features described herein can be implemented by discrete hardware components or integrated into the functionality of other hardware components, such as ASICS, FPGAs, DSPs, or similar devices. In addition, the methods, components, and features can be implemented by firmware modules or functional circuits within a hardware device. Furthermore, the methods, components, and features can be implemented in any combination of hardware devices and computer program components, or can be implemented within a computer program.

[0241] Unless specifically stated otherwise, terms such as "identify," "generate," "cause," "provide," "receive," "determine," "update," "compare," "train," "dewarp," "scale," "obtain," and the like refer to actions and processes performed or implemented by a computer system that manipulate and transform data represented as physical (electronic) quantities in computer system registers and memory into other data similarly represented as physical quantities in computer system memory or registers or other such information storage, transmission, or display devices. Also, as used herein, terms such as "first," "second," "third," "fourth," and the like are meant to be labels for distinguishing different elements and may not have an ordering meaning according to numerical designations.

[0242] The examples described herein also refer to apparatus for performing the methods described herein. This apparatus may be specially constructed to perform the methods described herein, or may comprise a general-purpose computer system selectively programmed by a computer program stored in the computer system. Such a computer program may be stored in a computer-readable tangible storage medium.

[0243] The methods and examples described herein are not inherently related to any particular computer or other apparatus. Various general-purpose systems can be used in accordance with the teachings described herein, or it may prove convenient to construct more specialized apparatus to perform the methods and / or each of the individual functions, routines, subroutines, or operations described herein. Example structures for a variety of these systems have been set forth in the description above.

[0244] The above description is intended to be illustrative, not limiting. While the present disclosure has been described with reference to particular examples and embodiments, it will be understood that the present disclosure is not limited to the described examples and embodiments. The scope of the present disclosure should be determined with reference to the following claims, along with the full scope of equivalents to which such claims are entitled.

Claims

1. identifying trace data including a plurality of data points, the trace data associated with production of substrates having characteristic values ​​that satisfy a threshold through a substrate processing system; determining guardband violating data points among the plurality of data points of the trace data based on the guardbands; determining a guardband violation shape characterization based on the guardband violation data points, wherein a classification of additional guardband violation data points of the additional trace data is based on the guardband violation shape characterization, and execution of a corrective action associated with the substrate processing system is based on the classification; A method comprising:

2. 2. The method of claim 1, wherein the guard band violation shape characterization is a weighted combination of one or more of guard band violation duration, guard band violation magnitude, guard band violation area, guard band violation location, or guard band violation intermittency.

3. The method of claim 1 , wherein the classification of the additional guard band violation data points comprises concatenating successive or future violations into a single violation based on the guard band violation shape characterization.

4. The method of claim 3 , wherein the linking of the successive or future violations into the single violation is further based on one or more non-violating shapes between two of the successive or future violations.

5. The method of claim 1 , wherein the determination of the guardband violation data points is made via multivariate analysis.

6. The method of claim 1 , wherein the determining the guardband violation data points comprises characterizing the guardband violation data points via multivariate analysis.

7. 2. The method of claim 1, wherein the determination of the guard band violation data points is further based on segmenting portions of the trace data associated with value changes that exceed a threshold change and extracting features from the trace data.

8. 2. The method of claim 1 , wherein the determining the guardband violation shape characterization comprises training a machine learning model using data input including the trace data to form a trained machine learning model associated with the guardband violation shape characterization.

9. identifying additional trace data; determining the additional guardband violating data points of the additional trace data based on the guardbands; and providing the additional guardband violation data points as data inputs to a trained machine learning model; receiving output data from the trained machine learning model, the output data including prediction data; determining the classification of the additional guard band violation data points based on the predicted data; and The method of claim 1 further comprising:

10. identifying trace data comprising a plurality of data points, the trace data associated with fabrication of a substrate through a substrate processing system; determining guardband violating data points among the plurality of data points of the trace data based on the guardbands; determining a classification of the guardband violation data points based on the guardband violation shape characterization, wherein implementation of a corrective action associated with the substrate processing system is based on the classification of the guardband violation data points; A method comprising:

11. 11. The method of claim 10, wherein the guard band violation shape characterization is a weighted combination of one or more of guard band violation duration, guard band violation magnitude, guard band violation area, guard band violation location, or guard band violation intermittency.

12. 11. The method of claim 10, wherein the determining the classification of the guard band violation data points comprises concatenating consecutive or future violations into a single violation based on the guard band violation shape characterization.

13. The method of claim 10 , wherein the determination of the guardband violation data points is made via multivariate analysis.

14. 11. The method of claim 10, wherein the determination of the guard band violation data points is further based on segmenting portions of the trace data associated with value changes that exceed a threshold change and extracting features from the trace data.

15. receiving historical trace data associated with past production of past substrates having past characteristic values ​​that satisfy a threshold value through the substrate processing system; training a machine learning model using data input including the historical trace data to form a trained machine learning model associated with the guardband violation shape characterization; The method of claim 10 further comprising:

16. providing the guardband violation data points as data inputs to a trained machine learning model; receiving output data from the trained machine learning model, the output data including prediction data; determining the classification of the guard band violation data points based on the predicted data; The method of claim 10 further comprising:

17. A non-transitory computer-readable storage medium storing instructions that, when executed, cause a processing device to perform operations, the operations including: identifying trace data including a plurality of data points, the trace data associated with production of substrates having characteristic values ​​that satisfy a threshold through a substrate processing system; determining guardband violating data points among the plurality of data points of the trace data based on the guardbands; determining a guard band violation shape characterization based on the guard band violation data points, wherein a classification of additional guard band violation data points of additional trace data is based on the guard band violation shape characterization, and execution of a corrective action associated with the substrate processing system is based on the classification.

18. 20. The non-transitory computer-readable storage medium of claim 17, wherein the guard band violation shape characterization is a weighted combination of one or more of guard band violation duration, guard band violation magnitude, guard band violation area, guard band violation location, or guard band violation intermittency.

19. 20. The non-transitory computer-readable storage medium of claim 17, wherein the classification of the additional guard band violation data points comprises concatenating consecutive violations into a single violation based on the guard band violation shape characterization.

20. 20. The non-transitory computer-readable storage medium of claim 17, wherein the determination of the guard band violation data points is made via multivariate analysis.

Images