Guard bands in substrate processing systems

Abstract

The method includes identifying trace data including a plurality of data points, the trace data being associated with the fabrication of a substrate having a characteristic value that meets a threshold through a substrate processing system. The method further includes determining a dynamic acceptable region outside the guard band limit based on the trace data. The method further includes performing a corrective action associated with the substrate processing system based on the dynamic acceptable region outside the guard band limit.

Description

Technical Field

[0001] The present disclosure relates to guard bands, and more particularly to guard bands in a substrate processing system.

Background Art

[0002] Products can be fabricated by performing one or more manufacturing processes using manufacturing equipment. For example, a substrate can be fabricated through substrate processing operations using substrate processing equipment. A product having specific characteristics is to be fabricated. Sensor data is monitored in relation to the substrate manufacturing process.

Summary of the Invention

[0003] The following is a simplified summary of the present disclosure to provide a basic understanding of some aspects of the present disclosure. This summary is not an extensive overview of the present disclosure. It is not intended to identify key or critical elements of the present disclosure nor to delineate any scope of either specific embodiments of the present disclosure or any scope of the claims. The sole purpose of this summary is to present some concepts of the present disclosure in a simplified form as a prelude to a more detailed description that is presented later.

[0004] In one aspect of the present disclosure, a method includes identifying trace data including a plurality of data points, the trace data being associated with the fabrication of a substrate having a characteristic value that meets a threshold through a substrate processing system. The method further includes determining a dynamic acceptable region outside the guard band limits based on the trace data. The method further includes performing a corrective action associated with the substrate processing system based on the dynamic acceptable region outside the guard band limits.

[0005] In one aspect of the present disclosure, the method includes identifying trace data comprising a plurality of data points, the trace data being associated with substrate fabrication via a substrate processing system. The method further includes comparing the trace data with an acceptable area outside the guard band limits. The method further includes updating the acceptable area outside the guard band limits based on the trace data in response to one or more data points in the trace data being within the acceptable area. The execution of a corrective action associated with the substrate processing system is based on the fact that at least a portion of the trace data is outside the acceptable area outside the guard band limits.

[0006] In one aspect of the present disclosure, a non-temporary computer-readable storage medium is provided that, when executed, stores instructions causing a processing device to perform an operation, wherein these operations include identifying trace data comprising a plurality of data points, the trace data being associated with the fabrication of a substrate having characteristic values ​​that satisfy a threshold via a substrate processing system. These operations further include determining a dynamically acceptable area outside the guard band limits based on the trace data. These operations further include performing a corrective action associated with the substrate processing system based on the dynamically acceptable area outside the guard band limits.

[0007] The figures in the attached drawings are provided as examples, not limited to those shown in this disclosure. [Brief explanation of the drawing]

[0008] [Figure 1] This block diagram shows an exemplary system architecture according to a specific embodiment. [Figure 2A-2D] This is a flowchart of a method associated with generating a guard band according to a specific embodiment. [Figure 3A-3D] This is a flowchart of a method associated with guard band violation profiling according to a specific embodiment. [Figure 4A-4D]This is a flowchart of a method associated with a dynamically acceptable area outside the guard band limit, according to a specific embodiment. [Figures 5A-5E] This figure shows an acceptable distribution type graph according to a specific embodiment. [Figures 6A-6E] This figure shows guard band violation profiling according to a specific embodiment. [Figures 7A-7F] This figure shows a dynamically acceptable area outside the guard band limit according to a specific embodiment. [Figures 8A-8B] This figure shows the guard band fitting according to a specific embodiment. [Figure 9] This is a block diagram showing a computer system according to a specific embodiment. [Modes for carrying out the invention]

[0009] Techniques related to guard bands in substrate processing systems (e.g., guard band reinforcement, guard band violation profiling, and dynamic regions outside the guard band range) are described herein. Guard bands can be upper and lower thresholds (e.g., acceptable error ranges) before and after a target value. Sensor data can be compared with the guard band to determine whether the sensor data is within the threshold range (e.g., a good substrate) or outside the threshold range (e.g., a defective substrate).

[0010] Products are manufactured by performing one or more manufacturing processes using manufacturing equipment. Products with specific characteristics should be manufactured. For example, substrates are manufactured through substrate processing operations using substrate processing equipment. Substrates that meet specific characteristic values ​​(e.g., dimensions confirmed by measurement data) should be used, and substrates that do not meet these characteristics should be discarded. Sensor data associated with the substrate processing operations is collected over time. Sensor data is monitored for several purposes, including attempting to manufacture substrates that meet specific characteristic values, verifying that the equipment is functioning properly, determining whether the equipment requires current or future repair or replacement, and determining adjustments to equipment parameters to make the process more efficient (e.g., regarding conditions or criteria such as yield or quality, throughput or quantity, cost or lifespan). For the sake of simplicity, this discussion will focus on the purpose of manufacturing substrates that meet specific characteristic values.

[0011] Traditionally, sensor data is summarized across specific policies or policy actions associated with the fabrication of substrates by the equipment. This summary is defined using a set of statistical information, such as mean and variance. These summarized statistics are then compared to set limits, such as the mean between lower and upper limits. If the sensor data falls within the limits, it is estimated that the substrate meets a specific characteristic value; if the sensor data falls outside the limits, it is estimated that the substrate does not meet a specific characteristic value. Limits that are too narrow cause false positives (e.g., inaccurately predicting that the substrate does not meet the characteristic value). Limits that are too broad cause missed detections (e.g., inaccurately predicting that the substrate meets the characteristic value). Variations in sensor data and substrate processing equipment can cause many false positives. Widening the limits to accommodate variations in sensor data and substrate processing equipment can also cause many missed detections. Conventional systems inaccurately label substrates as meeting or not meeting characteristic values, leading to material waste, reduced yield, defective products, increased user time, and increased equipment downtime. Attempting to correct mislabeling incurs extra processing costs, bandwidth, energy consumption, measurement operations, and user time.

[0012] The methods, devices, and systems disclosed herein provide improvements to guard bands in substrate processing systems that address the aforementioned and other shortcomings of conventional solutions.

[0013] In some embodiments, the processing device identifies trace data associated with the fabrication of a substrate having characteristic values ​​that satisfy a threshold (e.g., a good substrate). The trace data may include sets of sensor data associated with the fabrication of different substrates from different types of sensors. In some embodiments, the data is analyzed at the trace level by using guard bands (e.g., not merely providing summary statistics of the sensor over policy or policy operation). Guard bands can provide upper and lower thresholds over the length of the trace data. This offers several advantages over summary statistics (e.g., this disclosure can identify and profile specific violations in the trace data). The processing device generates initial guard bands based on the trace data (e.g., 3σ around the mean of the trace data at each time point in the data, where 3σ is determined by analyzing multiple runs of data at that particular time point). Based on the trace data, the processing device determines an acceptable variance type for the guard bands. Since the trace data is for a good wafer, the acceptable variance type for the guard bands can move (e.g., in the x-direction) by time-shifting from one or more of the sets of sensor data to satisfy the mean of the sensor data. An acceptable variance type may include a guard band upper limit that is a different distance from the mean of the trace data than the guard band lower limit. An acceptable variance type may include a wider guard band limit and a narrower guard band limit in a particular portion of the guard band. The processing device generates a guard band based on the trace data and the acceptable variance type. The processing device compares additional trace data (e.g., from a board whose condition is unknown, whether good or bad) to the guard band and, in response to one or more data points being outside the range of the guard band, the processing device takes corrective action (e.g., discarding the board, interrupting the board processing operation, inspecting the board, etc.).

[0014] In some embodiments, both trace data for good substrates and trace data for bad substrates are used. The guard band is determined or updated based on an improved understanding of the differences between good and bad trace data, and the bad trace data may be a single category or multiple categories representing different states or degrees of defects in the substrate.

[0015] In some embodiments, the processing device identifies trace data associated with the fabrication of a substrate having characteristic values ​​that satisfy a threshold (e.g., a good substrate). Based on the guard band, the processing device determines guard band violation data points in the trace data. Based on the guard band violation data points, the processing device determines guard band violation shape characterization. The processing device compares additional trace data (e.g., from a substrate whose quality is unknown) with the guard band to determine guard band violation data points. Based on the guard band violation shape characterization, the processing device determines the classification of the guard band violation data points (e.g., whether the guard band violation data points correspond to a good wafer or a bad wafer).

[0016] In some embodiments, the processing device identifies trace data associated with the fabrication of a substrate having characteristic values ​​that satisfy a threshold (e.g., a good substrate). Based on the trace data, the processing device determines a dynamically acceptable region outside the guard band limits. The dynamically acceptable region is the area outside the guard bands corresponding to data points of a good substrate, resulting in acceptable noise, acceptable drift, etc. The processing device compares additional trace data (e.g., from a substrate whose good or bad status is uncertain) to the acceptable region outside the guard band limits. In response to one or more data points being outside the acceptable region, the processing device takes corrective action (e.g., discarding the substrate). In response to one or more data points being within the acceptable region, the processing device updates the acceptable region. For example, the updated acceptable region may tolerate additional drift or noise based on the average of the additional trace data processed.

[0017] In some embodiments, the processing device (e.g., performing a guardbanding method) presents trace data runs that have violations, including shape characteristics of those violations, as a result of trace data runs that have been concluded to be within acceptable guardbanding limits, and / or regions where the processing device (e.g., performing a guardbanding method) cannot definitively determine the occurrence of a violation. The processing device (e.g., performing a guardbanding method) can allow a user (e.g., a subject expert) to confirm or invalidate the conclusions and suggestions of the guardbanding method. The processing device (e.g., performing a guardbanding method) can use feedback from the user (e.g., a subject expert) to update the guardbanding limits, other violation assessments, and / or guardbanding characterizations.

[0018] Aspects of the present disclosure provide technical advantages over conventional solutions. The present disclosure has fewer false detections and detection omissions compared to conventional solutions. This enables less material waste, increased yield, fewer defective products, less user time, less equipment downtime, etc. The present disclosure has fewer corrections for mislabeling of substrates compared to conventional solutions. This enables less process processor cost, used bandwidth, energy consumption, measurement operations, user time, etc.

[0019] Some embodiments of the present disclosure are described in relation to substrate processing. In some embodiments, the present disclosure is also applicable to other types of manufacturing processes.

[0020] Some embodiments of the present disclosure are described in relation to monitoring sensor data for the purpose of fabricating a substrate that meets certain characteristic values. In some embodiments, the present disclosure can also monitor sensor data for other purposes, such as confirming that a device is functioning properly, determining to perform repair or replacement (e.g., preventive maintenance) of a device or device component, determining adjustments to device parameters for making a process more effective (e.g., with respect to conditions or criteria such as yield or quality, throughput or quantity, cost or lifespan, etc.).

[0021] Some embodiments of the present disclosure are described in relation to performing corrective actions. In some embodiments, performing corrective actions can include identifying characteristics of trace data as degraded, different, etc.

[0022] Figure 1 is a block diagram showing an exemplary system 100 (exemplary system architecture) according to a particular embodiment. System 100 includes a client device 120, manufacturing equipment 124 (e.g., substrate processing equipment), a sensor 126, measuring equipment 128, a prediction server 112, and a data store 140. The prediction server 112 may be part of the prediction system 110. The prediction system 110 may further include server machines 170 and 180. The prediction system 110 can be used to predict whether an anomaly has occurred, to detect if an anomaly has occurred, etc. (e.g., using guardband technology).

[0023] In some embodiments, the manufacturing equipment 124 (e.g., a cluster tool) is part of a substrate processing system (e.g., an integrated processing system). The manufacturing equipment 124 includes one or more of a controller, a sealed system (e.g., a substrate carrier, a front opening unified pod (FOUP), an auto-teach FOUP, a process kit sealed system, a substrate sealed system, a cassette, etc.), a side storage pod (SSP), an aligner device (e.g., an aligner chamber), a factory interface (e.g., an equipment front end module (EFEM)), a load lock, a transfer chamber, one or more processing chambers, a robotic arm (e.g., disposed within the transfer chamber, within the front interface, etc.). The sealed system, SSP, and load lock attached to the factory interface, and the robotic arm disposed within the factory interface are for transferring contents (e.g., substrates, process kit rings, carriers, certified wafers, etc.) between the sealed system, SSP, load lock, and factory interface. The aligner device is disposed within the factory interface to align the contents. The load lock and processing chamber attached to the transfer chamber, and the robotic arm disposed within the transfer chamber are for transferring contents (e.g., substrates, process kit rings, carriers, certified wafers, etc.) between the load lock, processing chamber, and transfer chamber. In some embodiments, the manufacturing equipment 124 includes components of the substrate processing system. In some embodiments, the manufacturing equipment 124 is used to fabricate one or more products (e.g., substrates, semiconductors, wafers, etc.). In some embodiments, the manufacturing equipment 124 is used to fabricate one or more components used within the substrate processing system.

[0024] Sensor 126 can be coupled to manufacturing equipment 124. Sensor 126 can provide sensor data associated with manufacturing equipment 124 (for example, associated with the manufacturing of a corresponding product such as a substrate by manufacturing equipment 124). Sensor data can be stored as time-dependent measurements (e.g., trace data 142). Trace data 142 may include past trace data 144 and current trace data 146. Trace data 142 can be used for the health of the equipment and / or the health of the product (e.g., product quality). Manufacturing equipment 124 can manufacture products according to a policy or by running runs over a period of time. In some embodiments, trace data 142 may include one or more values ​​from among temperature (e.g., heater temperature), spacing (SP), pressure, high-frequency radio frequency (HFRF), electrostatic chuck (ESC) voltage, current, flow rate, power, voltage, etc. Trace data 142 can be associated with, or indicate, manufacturing parameters such as hardware parameters of the manufacturing equipment 124 (e.g., settings or components, e.g., size, type, etc.) or process parameters of the manufacturing equipment 124. Alternatively, or in addition, data associated with several hardware parameters can be stored as manufacturing parameters. Manufacturing parameters can indicate input settings for the manufacturing device (e.g., heater power, gas flow rate, etc.). Trace data 142 and / or manufacturing parameters can be provided when the manufacturing equipment 124 is performing a manufacturing process (e.g., equipment readings when processing a product). Trace data 142 can be different for each product (e.g., each substrate).

[0025] The measuring instrument 128 can be used to measure the characteristics of a product, such as a substrate (e.g., a processed substrate, a partially processed substrate, etc.). The measuring instrument can incorporate analysis to estimate or better determine the measured values. The measurement data can be included in the performance data 150 along with other performance metrics such as instrument maintenance, yield, etc. The performance data 150 may include historical performance data 152 and current performance data 154. The measurement data and / or performance data 150 may include virtual measurement data, non-virtual measurement data, a mixture of virtual and non-virtual measurement data, etc.

[0026] In some embodiments, trace data 142, performance data 150, and / or manufacturing parameters can be processed (e.g., by a client device 120 and / or a prediction server 112). Processing of trace data 142 may include generating features. In some embodiments, these features are patterns in the trace data 142 or performance data 150 (e.g., gradients, widths, heights, peaks, etc.) or combinations of values ​​from the trace data 142 or performance data 150 (e.g., power derived from voltage and current). Trace data 142 may include features that can be used by the prediction component 114 and / or the client device 120 to perform signal processing and / or to obtain prediction data 168 for taking corrective action. The prediction component 114 may be used to predict whether an anomaly has occurred, to detect if an anomaly has occurred (e.g., using guard banding techniques), etc.

[0027] Each instance (e.g., set) of trace data 142 can correspond to a product (e.g., a substrate), a set of manufacturing equipment 124, the type of substrate produced by the manufacturing equipment 124, and so on. Similarly, each instance of performance data 150 or manufacturing parameters can correspond to a product, a set of manufacturing equipment, the type of substrate produced by the manufacturing equipment, and so on. The data store 140 can further store information that associates sets of different data types, such as sets of trace data, sensor data, measurement data, and / or sets of manufacturing parameters associated with the same product, manufacturing equipment, substrate type, etc.

[0028] In some embodiments, the prediction system 110 can generate prediction data 168 using supervised machine learning (for example, a supervised dataset, performance data 150 including measurement data, and trace data 142 used to train the model 190 associated with good and bad substrates, etc.). In some embodiments, the prediction system 110 can generate prediction data 168 using semi-supervised learning (for example, a semi-supervised dataset, performance data 150 being prediction percentages, and trace data 142 used to train the model 190 associated with good substrates only, etc.). In some embodiments, the prediction system 110 can generate prediction data 168 using unsupervised machine learning (for example, an unsupervised dataset, clustering, clustering based on trace data 142, etc.). In some embodiments, the prediction system 110 can generate prediction data 168 using a model which is one or more of the following: a machine learning model, a statistical model, etc.

[0029] The client device 120, manufacturing equipment 124, sensor 126, measuring equipment 128, prediction server 112, data store 140, server machine 170, and server machine 180 can be connected to each other via network 130 to generate prediction data 168 and execute corrective actions.

[0030] In some embodiments, network 130 is a public network that provides client devices 120 with access to the prediction server 112, the data store 140, and other publicly available computing devices. In some embodiments, network 130 is a private network that provides client devices 120 with access to manufacturing equipment 124, sensors 126, measuring instruments 128, the data store 140, and other privately available computing devices. Network 130 may include one or more wide area networks (WANs), local area networks (LANs), wired networks (e.g., Ethernet networks), wireless networks (e.g., 802.11 networks or Wi-Fi networks), cellular networks (e.g., Long Term Evolution (LTE) networks), routers, hubs, switches, server computers, cloud computing networks, and / or combinations thereof.

[0031] The client device 120 may include computing devices such as personal computers (PCs), laptops, mobile phones, smartphones, tablet computers, netbooks, network-attached televisions ("smart TVs"), network-attached media players (e.g., Blu-ray players), set-top boxes, over-the-top (OTT) streaming devices, and operator boxes. The client device 120 may include a corrective action component 122. The corrective action component 122 may receive user input (for example, via a graphical user interface (GUI) displayed through the client device 120).

[0032] In some embodiments, the corrective action component 122 retrieves trace data 142 associated with the manufacturing equipment 124 (e.g., current trace data 146) (e.g., from a data store 140, etc.) and provides the trace data 142 associated with the manufacturing equipment 124 (e.g., current trace data 146) to the prediction system 110. In some embodiments, the corrective action component 122 stores the trace data 142 in the data store 140, and the prediction server 112 retrieves the trace data 142 from the data store 140. In some embodiments, the prediction server 112 may store the output of a trained machine learning model 190 (e.g., prediction data 168) in the data store 140, and a client device 120 may retrieve its output from the data store 140. In some embodiments, the corrective action component 122 receives instructions for corrective action from the prediction system 110 and implements the corrective action. The client device 120 may include an operating system that allows the user to generate, view, or edit data (for example, instructions associated with the manufacturing equipment 124, corrective actions associated with the manufacturing equipment 124, etc.) one or more of the following:

[0033] In some embodiments, historical performance data 152 corresponds to historical characteristic data of a product (e.g., manufactured using manufacturing parameters associated with historical trace data 144 and stored manufacturing parameters), and predictive data 168 corresponds to predicted characteristic data (e.g., predicted characteristic data of a product that should be manufactured or has been manufactured under conditions recorded by the current trace data 146 and / or manufacturing parameters). In some embodiments, predictive data 168 is predicted measurement data (e.g., virtual measurement data) of a product that should be manufactured or has been manufactured according to conditions recorded as the current trace data 146 and / or manufacturing parameters. In some embodiments, predictive data 168 is an anomaly (e.g., an abnormal product, an abnormal component, an abnormal manufacturing equipment 124, abnormal energy usage, etc.) and an indication of one or more causes of these anomalies. In some embodiments, predictive data 168 is an indication of change or drift over time in some component such as manufacturing equipment 124, sensor 126, or measuring instrument 128. In some embodiments, the predictive data 168 indicates the end of life for components such as manufacturing equipment 124, sensors 126, and measuring instruments 128.

[0034] Implementing a manufacturing process that results in defective products can be costly in terms of time, energy, products, components, manufacturing equipment 124, defect identification, and disposal of defective products. By generating predictive data 168 based on trace data 142 and taking corrective actions based on the predictive data 168, the system 100 can have the technical advantage of avoiding the costs associated with producing, identifying, and disposing of defective products.

[0035] Executing a manufacturing process that results in a failure of a component of the manufacturing equipment 124 can have significant drawbacks in terms of downtime, product damage, equipment damage, and expedited ordering of replacement components. By generating predictive data 168 based on trace data 142 (e.g., manufacturing parameters used or to be used to manufacture a product) and performing corrective actions (e.g., predictive operational maintenance such as component replacement, processing, or cleaning) based on the predictive data 168, the system 100 can have the technical advantage of avoiding one or more costs such as unexpected component failure, unplanned downtime, loss of production rate, unexpected equipment failure, or product scrapping. Monitoring the performance of components (e.g., manufacturing equipment 124, sensor 126, measuring instrument 128, etc.) over time can provide indication of degraded components.

[0036] Manufacturing parameters may be suboptimal for producing a product, which can result in costly consequences such as increased resource (e.g., energy, coolant, gas, etc.) consumption, increased time required to produce the product, increased component failures, and an increased quantity of defective products. By generating predictive data 168 based on the characteristics of trace data 142 and performing corrective actions to update the manufacturing parameters (e.g., setting optimal manufacturing parameters) based on the predictive data 168, the system 100 can have the technical advantage of using optimal manufacturing parameters (e.g., hardware parameters, process parameters, optimal design) to avoid the costly consequences of suboptimal manufacturing parameters.

[0037] Corrective actions may be associated with one or more of the following: computational process control (CPC), statistical process control (SPC) (e.g., SPC on electronic components to determine the process under control, SPC to predict the lifespan of components, SPC for comparison with a 3σ graph, etc.), advanced process control (APC), model-based process control, preventive maintenance, design optimization, updating of manufacturing parameters or manufacturing strategies for current or future manufacturing processes, feedback control, and machine learning modifications.

[0038] In some embodiments, corrective actions include providing an alarm (e.g., a warning to stop or not perform the manufacturing process if predictive data 168 indicates a predicted anomaly, such as a product, component, or manufacturing equipment 124). In some embodiments, corrective actions include scheduling preventive maintenance. In some embodiments, corrective actions include scheduling corrective maintenance. In some embodiments, corrective actions include updating the process policy to fabricate subsequent substrates. In some embodiments, corrective actions may be determined considering the ongoing substrate processing process and may include updating the current process. In some embodiments, corrective actions include compensating for chamber drift associated with manufacturing equipment 124 (e.g., substrate processing equipment). In some embodiments, corrective actions include compensating for sensor drift of sensors associated with manufacturing equipment 124 (e.g., substrate processing equipment). In some embodiments, corrective actions include providing feedback control (e.g., modifying manufacturing parameters in response to predictive data 168 indicating a predicted anomaly). In some embodiments, the corrective action includes providing machine learning (for example, modifying one or more manufacturing parameters based on predictive data 168). In some embodiments, the execution of the corrective action includes updating one or more manufacturing parameters. In some embodiments, one or more corrective actions are performed in relation to components of a substrate processing machine.

[0039] Manufacturing parameters may include hardware parameters (e.g., component replacement, use of specific components, processing chip replacement, firmware updates, etc.) and / or process parameters (e.g., temperature, pressure, flow rate, current, voltage, gas flow, rise rate, etc.). In some embodiments, corrective actions include preventive operational maintenance (e.g., replacement, processing, cleaning, etc. of components of the manufacturing equipment 124). In some embodiments, corrective actions include optimizing the design (e.g., updating manufacturing parameters, manufacturing processes, manufacturing equipment 124, etc. for the optimized product). In some embodiments, corrective actions include updating policies (e.g., putting the manufacturing equipment 124 into idle mode, sleep mode, warm-up mode, etc.).

[0040] Each of the prediction server 112, server machine 170, and server machine 180 may include one or more computing devices such as rack-mount servers, router computers, server computers, personal computers, mainframe computers, laptop computers, tablet computers, desktop computers, graphics processing units (GPUs), and application-specific integrated circuits (ASICs) (for example, tensor processing units (TPUs)).

[0041] The prediction server 112 may include a prediction component 114. In some embodiments, the prediction component 114 can receive current trace data 146 (e.g., received from a client device 120, retrieved from a data store 140) and, based on the current data, generate output (e.g., prediction data 168) for taking corrective action associated with the manufacturing equipment 124. In some embodiments, the prediction component 114 can use one or more trained models 190 to determine the output for taking corrective action based on the current data. In some embodiments, the prediction component 114 predicts that an anomaly has occurred or will occur. In some embodiments, the prediction component 114 predicts that another event has occurred or may occur.

[0042] Model 190 can be a single model or multiple models. Models can be applied sequentially, multiple models can be used simultaneously, appropriate models can be selected based on some metric, combinations of these methods can be used, and so on. Model 190 (or the models included in Model 190) can be machine learning models, including supervised, unsupervised, or semi-supervised machine learning models. Model 190 does not have to be a machine learning model; it can be, for example, a statistical model, a correlation model, etc.

[0043] In some embodiments, a first model 190 is used to generate guard bands (see, for example, Figures 2A to 2D), a second model 190 is used for guard band violation profiling (see, for example, Figures 3A to 3D), and a third model 190 is used to generate dynamic areas outside the range of guard bands (see, for example, Figures 4A to 4D).

[0044] In some embodiments, the data input to Model 190 may include trace data 142 from a single sensor 126. In other embodiments, the data input to Model 190 may include trace data 142 from many sensors 126 exhibiting different characteristic values. The data input may include manufacturing parameters. The features extracted from the trace data 142, the method of feature extraction, the corrective actions and / or predictive data 168 associated with the features, and the method of associating the corrective actions and / or predictive data 168 can all be adapted for the data provided as input.

[0045] In some embodiments, the prediction component 114 receives current trace data 146, provides the current trace data 146 as input to the model 190, and obtains an output from the model 190 showing prediction data 168. In some embodiments, the prediction data 168 shows performance data 150 (e.g., measurement data, yield, etc.). In some embodiments, the prediction data 168 shows corrective actions.

[0046] In some embodiments, Model 190 takes trace data 142 (for example, data indicating policies associated with the trace data 142, components of manufacturing equipment, etc.) as input and produces predictive data 168 as output. Model 190 can be a single model or may include many models. Based on the input data, Model 190 can determine which processes should be executed, or it can indicate to the user which analysis is appropriate for the input data, or a combination of these.

[0047] The datastore 140 and / or can be memory (e.g., random access memory), drives (e.g., hard drives, flash drives), database systems, or other types of components or devices capable of storing data. The datastore 140 and / or can include multiple storage components (e.g., multiple drives or multiple databases) that can span multiple computing devices (e.g., multiple server computers). The datastore 140 can store trace data 142, performance data 150, and prediction data 168. The trace data 142 can include historical trace data 144 and current trace data 146. The trace data can include sensor data time traces over the duration of a manufacturing process, correlations between data and physical sensors, pre-processed data such as averages and composite data, and data showing sensor performance over time (e.g., many manufacturing processes). Manufacturing parameters and performance data 150 can include similar features. Historical trace data 144, manufacturing parameters, and historical performance data 152 can be historical data (e.g., at least a portion of the training model 190). The current trace data 146 can be the current data (for example, at least a portion of the historical data that should be input into the model 190 next) from which the predictive data 168 (for example, to implement corrective actions) should be generated.

[0048] In some embodiments, the prediction system 110 further includes server machines 170 and 180. Server machine 170 includes a dataset generator 172 capable of generating datasets (e.g., a set of data inputs and a set of target outputs) for training, certifying, and / or testing one or more models, such as machine learning models. The models 190 may include one or more machine learning models, or other types of models, such as statistical models. Models incorporating machine learning can be trained using input data and optionally using target output data. Models that do not incorporate machine learning can also be trained. In some embodiments, the dataset generator 172 can divide historical data (e.g., historical trace data 144, manufacturing parameters, or historical performance data 152 stored in data store 140) into a training set (e.g., 60 percent of the historical data), a certification set (e.g., 20 percent of the historical data), and a test set (e.g., 20 percent of the historical data). In some embodiments, the prediction system 110 generates multiple sets of elements (e.g., via prediction components 114). For example, the first set of elements could correspond to a first set of sensor data types corresponding to each of the datasets (e.g., a training set, a certification set, and a test set) (e.g., a first set of sensors, a first combination of values ​​from the first set of sensors, and a first pattern of values ​​from the first set of sensors), and the second set of elements could correspond to a second set of sensor data types corresponding to each of the datasets (e.g., a second set of sensors different from the first set of sensors, a second combination of values ​​different from the first combination, and a second pattern different from the first pattern).

[0049] The server machine 180 includes a training engine 182, an authentication engine 184, a selection engine 185, and / or a test engine 186. The engines (e.g., training engine 182, authentication engine 184, selection engine 185, and test engine 186) can refer to hardware (e.g., circuits, dedicated logic, programmable logic, microcode, processing devices, etc.), software (e.g., instructions executed on processing devices, general-purpose computer systems, or dedicated machines), firmware, microcode, or a combination thereof. The training engine 182 may be capable of training a machine learning model 190 or various machine learning models contained within a model 190 using one or more sets of elements associated with a training set from a dataset generator 172. The training engine 182 can generate multiple trained machine learning models 190, each trained machine learning model 190 corresponding to a distinct set of elements from the training set (e.g., sensor data from a distinct set of sensors). For example, a first trained machine learning model may be trained using all elements (e.g., X1-X5), a second trained machine learning model may be trained using a first subset of elements (e.g., X1, X2, X4), and a third trained machine learning model may be trained using a second subset of elements (e.g., X1, X3, X4, and X5), where the second subset of elements may partially overlap with the first subset of elements. The dataset generator 172 can receive the output of a trained machine learning model (e.g., a model trained to perform a first operation of trace data processing), collect that data into training, authentication, and test datasets, and use these datasets to train a second machine learning model (e.g., a model to be trained to perform a second operation of trace data processing).

[0050] The authentication engine 184 may authenticate the trained machine learning models 190 using the corresponding sets of elements from the authentication set of the dataset generator 172. For example, a first trained machine learning model 190 trained using a first set of elements from the training set can be authenticated using the first set of elements from the authentication set. The authentication engine 184 can determine the accuracy of each of the trained machine learning models 190 based on the corresponding sets of elements from the authentication set. The authentication engine 184 can discard trained machine learning models 190 that have an accuracy that does not meet a threshold accuracy. In some embodiments, the selection engine 185 may select one or more trained machine learning models 190 that have an accuracy that meets a threshold accuracy. In some embodiments, the selection engine 185 may select the trained machine learning model 190 with the highest accuracy among the trained machine learning models 190. In some embodiments, the authentication engine 184 and the selection engine 185 may repeat this process for each machine learning model contained within the model 190.

[0051] In some embodiments, the authentication engine 184 performs verification and / or authentication (e.g., verification and authentication (V&V)). Verification and authentication can be independent procedures used together to determine whether a product, service, or system (e.g., a machine learning model) meets requirements and specifications and intended purposes. Authentication may include assurance that the machine learning model meets the needs of a customer or other identified stakeholder (e.g., acceptance and conformity by an external customer). Verification may include an assessment of whether the machine learning model complies with regulations, requirements, specifications, or imposed conditions (e.g., internal processes).

[0052] The test engine 186 may be able to test the trained machine learning models contained within model 190 using the corresponding sets of elements from the test set of the dataset generator 172. For example, a first trained machine learning model 190, trained using a first set of elements from the training set, can be tested using a first set of elements from the test set. Based on the test set, the test engine 186 can determine which of all the trained machine learning models contained within model 190 has the highest accuracy. The test engine 186 can repeat this process for all the machine learning models contained within model 190.

[0053] Model 190 can refer to model artifacts created by the training engine 182 using a training set containing data inputs and corresponding target outputs (the correct responses for each training input). Patterns in the dataset that map data inputs to target outputs (correct responses) can be discovered, and the machine learning model is provided with mappings that capture these patterns. The machine learning model can use one or more of the following: support vector machines (SVMs), radial basis functions (RBFs), clustering, supervised machine learning, semi-supervised machine learning, unsupervised machine learning, k-nearest neighbors algorithm (k-NN), linear regression, random forests, and neural networks (e.g., artificial neural networks).

[0054] The prediction component 114 can provide the current trace data 146 to the model 190, run the machine learning model 190 trained on the input, and obtain one or more outputs. The prediction component 114 can determine (e.g., extract) prediction data 168 from the output of the model 190, and can determine (e.g., extract) confidence data from the output indicating the level of confidence that the prediction data 168 is an accurate predictor of the process associated with the input data for products made or to be made using the manufacturing equipment 124 with the current trace data 146 and / or manufacturing parameters. The prediction component 114 may also determine a confidence range associated with a predicted event, such as a Remnant Life (RUL) window including upper and lower limits. The prediction component 114 or the corrective action component 122 can use the confidence data to determine, based on the prediction data 168, whether and / or when to trigger a corrective action associated with the manufacturing equipment 124.

[0055] Confidence data may include or indicate a confidence level regarding whether the prediction data 168 is an accurate prediction for products associated with at least a portion of the input data. For example, the confidence level is a real number between 0 and 1, where 0 indicates no confidence that the prediction data 168 is an accurate prediction for products processed according to the input data, and 1 indicates absolute confidence that the prediction data 168 accurately predicts the characteristics of products processed according to the input data. In response to confidence data indicating a confidence level below a threshold level for a given number of cases (e.g., a percentage of cases, a frequency of cases, an occurrence frequency, a total number of cases, etc.), the prediction component 114 (e.g., based on current trace data 146, manufacturing parameters, current performance data 154, etc.) can be used to retrain the model 190.

[0056] For illustrative purposes only, not limiting, aspects of this disclosure describe training one or more machine learning models 190 using historical data (e.g., historical trace data 144, historical performance data 152) and inputting current data (e.g., current trace data 146) into one or more trained machine learning models 190 to determine predictive data 168. In other embodiments, a heuristic model or a rule-based model is used to determine predictive data 168 (e.g., without using trained machine learning models). The predictive component 114 can monitor historical trace data 144 and historical performance data 152.

[0057] In some embodiments, the functions of the client device 120, prediction server 112, server machine 170, and server machine 180 can be provided by fewer machines. For example, in some embodiments, server machines 170 and 180 can be integrated into a single machine, and in some other embodiments, server machine 170, server machine 180, and prediction server 112 can be integrated into a single machine. In some embodiments, the client device 120 and prediction server 112 can be integrated into a single machine.

[0058] In general, functions described in one embodiment as being performed by the client device 120, prediction server 112, server machine 170, and server machine 180 can, where appropriate, also be performed by the prediction server 112 in other embodiments. In addition, functions attributable to a particular component can also be performed by different or multiple components working together. For example, in some embodiments, the prediction server 112 can determine corrective actions based on the prediction data 168. In another example, the client device 120 can determine the prediction data 168 based on the output from a trained machine learning model.

[0059] In addition, the functionality of a particular component can be performed by different or multiple components working together. One or more of the prediction server 112, server machine 170, or server machine 180 can be accessed as a service provided to other systems or devices via an appropriate application programming interface (API).

[0060] In some embodiments, “User” can represent a single individual. However, other embodiments of this disclosure also include “Users” that are entities managed by multiple users and / or automated sources. For example, a set of individual users integrated as a group of administrators can be considered a “User.”

[0061] Embodiments of this disclosure can be applied to data quality assessment, feature enhancement, model evaluation, virtual instrumentation (VM), predictive maintenance (PdM), limit optimization, anomaly or defect detection, anomaly or defect classification, and the like.

[0062] Figures 2A-2D, 3A-3D, and 4A-4D are flowcharts of methods 200A-D, 300A-D, and 400A-D associated with guard bands according to specific embodiments. In some embodiments, methods 200A-D, 300A-D, and 400A-D are executed by processing logic including hardware (e.g., circuits, dedicated logic, programmable logic, microcode, processing devices, etc.), software (e.g., instructions executed on processing devices, general-purpose computer systems, or dedicated machines), firmware, microcode, or a combination thereof. In some embodiments, methods 200A-D, 300A-D, and 400A-D are executed at least partially by a prediction system 110. In some embodiments, methods 200A-D, 300A-D, and 400A-D are executed at least in part by one or more of the following: a prediction system 110 (e.g., a prediction server 112, a prediction component 114), a client device 120 (e.g., a corrective action component), a manufacturing device 124, and / or a measuring device 128. In some embodiments, a non-temporary storage medium stores instructions that cause a processing device (e.g., a prediction system 110, a server machine 180, a prediction server 112, etc.) to execute one or more of the methods 200A-D, 300A-D, and 400A-D when executed by the processing device.

[0063] For the sake of simplicity, methods 200A-D, 300A-D, and 400A-D are illustrated and described as a series of operations. However, the operations according to this disclosure can be performed in various orders and / or simultaneously, along with other operations not presented and described herein. Furthermore, in some embodiments, not all shown operations are necessarily performed to carry out methods 200A-D, 300A-D, and 400A-D according to the subject matter disclosed. In addition, it will be understood and recognized by those skilled in the art that methods 200A-D, 300A-D, and 400A-D can also be represented, alternatively, through a state diagram or events as a series of correlated states.

[0064] In some embodiments, Figures 2A–2D and 3A–3D illustrate sequential data guard band analysis for improved defect diagnosis, defect classification, and prediction (e.g., sequential guard band analysis for defect detection and classification (FDC)). In some embodiments, this disclosure provides an improved analysis of sequential data streams (e.g., trace data) using full trace analysis (FTA) of guard band defect diagnosis to provide improved detection, classification, and prediction, including reduced false alarms (e.g., false detections) and false alarms (e.g., false detections). Conventional sensor trace analysis may have false alarms and false detections (e.g., fail to adequately capture normal variation, fail to fit the model over time, fail to address policy endpoints, fail to address phase-shifted traces, fail to transmit the model to different regions, etc.), resulting in the conventionally low adoption rate of guard bands. Conventional summary statistics may miss portions of the trace data (e.g., transient regions). This disclosure can be applied to address false positives and false negatives, and to provide robustness over time and flexibility to different regions (see, for example, Figures 8A–8B). This disclosure uses guard bands to analyze both transient and steady-state regions and to extract complex features (e.g., more complex than eigenvariate analysis (UVA)).

[0065] The guard bands of this disclosure can detect anomalies or defects occurring within trace data, but they do not necessarily conform to typical anomaly shapes such as spikes or fluctuations. Conventional guard bands may have output quality issues based on false positives (e.g., within transient events), stretching at the boundaries before and after transitions, variability between traces treated as anomalies, and all violations being treated as equal without profiling or quantification. The guard bands of this disclosure can overcome these shortcomings. The guard bands of this disclosure can be provided independently or in combination with other analytical capabilities such as semi-automated feature extraction (SFE).

[0066] A guard band can be a channel defined over a data stream, meaning it identifies a region of commonality (e.g., acceptable values) based on its position within the data stream. A data stream is often sensor values ​​between events, such as a fabrication run. A data stream is often trace data, a series of data values ​​presented and arranged over time (e.g., the x-axis is time). Ordering may not be time-based (e.g., it may be an indication of a number of things, such as the number of products fabricated or the number of errors logged). The term "acceptable" depends on the application and purpose of the guard band analysis. For example, "acceptable" may mean "not abnormal" or "not defective." A guard band channel may have upper and lower limits. In some embodiments, these limits can be calculated along the duration of the data stream using statistical methods. For example, a channel may represent ±3σ of the values ​​of multiple traces of a particular sensor, where the variance is calculated at each time value in the data stream. Smoothing techniques (e.g., time-based averaging) can be used to make the channel more noise-resistant and smoother.

[0067] For anomalous traces, FTA guard banding can be used for minor fluctuations in transient segments. Using FTA guard banding, multiple traces or trace segments from a specific sensor can be analyzed across multiple runs. FTA guard banding can establish upper and lower limits (e.g., 3σ) that indicate a normal range or channel for sensor data over time. This solution identifies and profiles deviations from the guard band. FTA capabilities can be used complementaryly with SFE to provide comprehensive analysis as input to a fingerprint library. Guard bands can be associated with a single sensor (UVA) or across multiple sensors (multivariate analysis (MVA)). In the case of MVA, the sensor value (e.g., y-axis) can be a metric representing any combination of sensors involved in the MVA.

[0068] In some embodiments, the guard band may include multiple guard bands (for example, a warning guard band that is completely contained within an error guard band). In some embodiments, a first area is a normal area regardless of other parameters, a second area is an area whose classification as good or bad is based on an analysis of other parameters, and a third area is a defective area regardless of other parameters.

[0069] In some embodiments, the x-axis parameter of the guard band, or the parameter defining the data ordering pattern, is time (for example, granularly associated with the sensor reading speed). In some embodiments, the substrate number in the process tool may correspond to the substrate process.

[0070] In some embodiments, guard band violation definitions can be anomalies, defects, warnings, event triggers, and / or predictions. The determination and interpretation of guard band violations may be related to the purpose of the guard band (e.g., defect or anomaly detection). A single data point outside the range of a guard band channel may indicate a defect. A probability distribution function can be used to define the area, duration, and size of the portion of the trace containing the violation area. These violation areas can be evaluated (e.g., violation area attribute values) to interpret whether a defect exists.

[0071] Figures 2A to 2D are flowcharts of methods 200A to 2D associated with generating guard bands according to specific embodiments. Figure 2A is a flowchart of method 200A associated with generating guard bands, Figure 2B is a flowchart of method 200B associated with using guard bands, Figure 2C is a flowchart of method 200C associated with generating guard bands via machine learning, and Figure 2D is a flowchart of method 200D associated with using guard bands via machine learning.

[0072] Referring to Figure 2A, in some embodiments, in block 202, the processing logic identifies trace data associated with the fabrication of a substrate (e.g., a good substrate) via a substrate processing system having characteristic values ​​that satisfy a threshold. The trace data includes a separate set of sensor data over time for each substrate. In some embodiments, the trace data includes sensor data (e.g., different types of sensor data) from multiple different sensors for each substrate.

[0073] In block 204, the processing logic determines the acceptable distribution type (for example, a guard band) based on the trace data.

[0074] In some embodiments, the processing logic generates guard bands based on trace data. These guard bands may include upper and lower limits for defect detection (e.g., anomaly detection). In some embodiments, to form the guard bands, the mean of all trace data is generated, and then offsets from the mean (e.g., 3σ) are used as upper and lower limits. Traditionally, the upper and lower limits are equally spaced from the mean of the trace data.

[0075] Since the trace data is from a good substrate, any variation from the guard band formed by the trace data is of an acceptable dispersion type.

[0076] In some embodiments, block 204 includes processing logic in block 210 to determine the variation between traces of trace data (see Figures 5A-5B). In some embodiments, acceptable variance types include variation between traces of time shift. Trace data may include sensor values ​​(e.g., y-axis) against time (e.g., x-axis). Some parts of the set of sensor data over time may be misaligned because the recording of sensor data starts at different times (e.g., time shift on the x-axis). In some embodiments, the processing logic tracks the difference between traces and autonomously adjusts the guardband variance horizontally when a normal time-shift variance is found. This prevents false positives when the policy step or trace shifts slightly. This increases robustness and reduces false positives.

[0077] In some embodiments, block 204 includes processing logic that determines an upper limit different from a lower limit based on trace data in block 212 (see Figure 5C). In some embodiments, the acceptable variance types include a first acceptable variance type for forming the upper limit of the guard band and a second acceptable variance type for forming the lower limit of the guard band. The first acceptable variance type (e.g., quantity) may be different from the second acceptable variance type (e.g., quantity). Variations on each side of the guard band are calculated separately (e.g., during signal transitions) to reduce false detections.

[0078] In some embodiments, block 204 includes processing logic in block 214 to determine the location-dependent variance of the trace data (see Figure 5D). Parameters and weights (e.g., duration, level, region, concatenation of consecutive violations) as well as other guard band settings can be adjusted according to the location within the guard band (e.g., along the x-axis). Examples include: 1) adjusting along the x-axis to correspond to different behaviors along the process so that the guard band senses or adapts more or less; 2) adjusting the guard band according to the characteristics of the signal (e.g., more conservatively in regions where the guard band changes rapidly in the y-direction, and more aggressively in regions where the guard band value is relatively constant); and 3) adjusting the guard band according to warnings detected along the guard band (e.g., adjusting the distribution parameters so that after a first spike is detected, future spike features on that trace are sensed more or less). This improves guard band performance and further enables the incorporation of subject matter expertise into guard band analysis.

[0079] In some embodiments, block 204 includes processing logic in block 216 to determine trace segmentation of trace data (see Figure 5E). In some embodiments, acceptable variance types are further based on segmentation of portions of trace data (e.g., associated with values ​​exceeding a threshold change, or changes in values ​​within a threshold). For example, a feature (e.g., within a threshold) can trigger a wider guard band for the next segment of the guard band. In some embodiments, acceptable variance types are further based on feature extraction from trace data.

[0080] In some embodiments, block 204 includes processing logic in block 218 that performs feature extraction (e.g., semi-automatic feature extraction (SFE)) of trace data. For example, the guard band parameters can be varied for different segments or different features. This improves guard band performance and overall analysis by allowing a combination of different capabilities (e.g., guard banding, feature extraction, and / or trace segmentation).

[0081] In some embodiments, the trace data includes sensor data from different types of sensors, and the acceptable distribution type is via MVA. The MVA metric combines values ​​from multiple sensors into a single metric (e.g., the first principal component in Principal Component Analysis (PCA)). Using the MVA technique, guard band parameters can be applied across multiple guard bands. Examples include 1) determining violations and violation distributions based on violation characteristics across two or more traces that occur (or do not occur) simultaneously, and / or 2) including motion states based on one or more other signals associated with a guard band for a signal. This improves guard band performance, allows for further incorporation of subject matter expertise into guard band analysis, and enables addressing correlations across multiple signals.

[0082] In some embodiments, the features include one or more of the following: persistent features (e.g., slopes and planes), user-defined features (e.g., a combination of slopes and gradients), intermittent features such as slopes, planes, and FTA features, x-offset, y-offset, shape, length, and distortion correction.

[0083] In block 205, the processing logic generates guard bands based on acceptable variance types. For example, guard bands can enable time shifts (e.g., within the x-axis), different upper and lower bounds, multivariate analysis (e.g., multivariate), segmentation of parts of trace data, feature extraction of trace data, and more. In some embodiments, guard bands are generated by training a machine learning model, as shown in Figure 2C.

[0084] In block 208, the processing logic executes corrective actions associated with the substrate processing system based on the guard band. Block 208 may also include comparing additional trace data with the guard band to determine whether to execute corrective actions (see, for example, Figure 2B).

[0085] Referring to Figure 2B, in some embodiments, in block 222, the processing logic identifies trace data associated with substrate fabrication via a substrate processing system (e.g., the same substrate processing system as in Figure 2A). The trace data may be associated with substrates where it is unknown whether the substrate characteristic data meets a threshold (e.g., a good substrate) or does not meet a threshold (e.g., a defective substrate). The trace data includes a separate set of sensor data over time for each substrate. In some embodiments, the trace data includes sensor data from multiple different sensors (e.g., different types of sensor data) for each substrate.

[0086] In block 224, the processing logic compares the trace data with guard bands generated based on acceptable distribution types (see, for example, block 206 in Figure 2A). The guard bands may have upper and lower limits over time relative to the data points.

[0087] In block 226, the processing logic determines that one or more data points in the trace data are not within the range of the guard band. One or more data points may include at least one data point above the upper limit and / or at least one data point below the lower limit (for example, these data points do not match data points on a substrate that have characteristic values ​​that satisfy the threshold).

[0088] Block 228, the processing logic, executes corrective actions associated with the substrate processing system. In some embodiments, corrective actions include providing an alarm, interrupting the substrate processing equipment, inspecting the substrate, discarding the substrate, updating manufacturing parameters, etc. In some embodiments, the performance of the corrective action is specific to the type or number of data points that are outside the range of the guard band.

[0089] Referring to Figure 2C, in some embodiments, in block 242, the processing logic identifies historical trace data associated with the fabrication of the substrate via the substrate processing system. In some embodiments, a substrate (e.g., a good substrate) has characteristic values ​​that satisfy a threshold. Block 242 can be similar to block 202 in Figure 2A. In some embodiments, a substrate (e.g., a defective substrate) has characteristic values ​​that do not satisfy a threshold.

[0090] In some embodiments, in block 244, the processing logic identifies historical performance data associated with past trace data. In some examples, the historical performance data indicates whether a board is a good board or a bad board (for example, all boards have characteristic values ​​that meet a threshold, or all boards have characteristic values ​​that do not meet a threshold).

[0091] In block 246, the processing logic trains a machine learning model with a data input that includes historical trace data (e.g., a target output containing historical performance data) to generate a trained machine learning model that shows guard bands associated with acceptable variance types. By training the machine learning model, the guard bands can be based on the acceptable variance types described in block 204 of Figure 2A.

[0092] A trained machine learning model can be used to determine whether additional trace data meets the guard band requirements (see, for example, Figure 2D).

[0093] Referring to Figure 2D, in some embodiments, in block 262, the processing logic identifies trace data associated with substrate fabrication via a substrate processing system (e.g., the same substrate processing system as in Figure 2C). The trace data can be associated with substrates where it is unknown whether the substrate characteristic data meets a threshold (e.g., a good substrate) or does not meet a threshold (e.g., a defective substrate). Block 262 can be similar to block 242 in Figure 2B.

[0094] In block 264, the processing logic provides trace data as input to a trained machine learning model associated with guard bands generated based on acceptable variance types (for example, the trained machine learning model in Figure 246 of Figure 2C).

[0095] In block 266, the processing logic receives an output from the trained machine learning model that shows the predicted data.

[0096] In block 268, the processing logic determines, based on the prediction data, that one or more data points in the trace data are not within the range of the guard bands of the trained machine learning model. One or more data points may be above the upper limit of the guard band or below the lower limit of the guard band.

[0097] In block 270, the processing logic executes corrective actions associated with the substrate processing system based on the predicted data. Block 270 can be similar to block 228 in Figure 2B.

[0098] Figures 3A to 3D are flowcharts of methods associated with guard band violation profiling according to a particular embodiment. Figure 3A is a flowchart of method 300A associated with determining guard band violation shape characterization and classifying guard band violation data points; Figure 3B is a flowchart of method 300B associated with classifying guard band violation data points based on guard band violation shape characterization; Figure 3C is a flowchart of method 300C associated with training a machine learning model to classify guard band violation data points; and Figure 3D is a flowchart of method 300D associated with classifying guard band violation data points using the trained machine learning model.

[0099] Referring to Figure 3A, in some embodiments, in block 302, the processing logic identifies trace data associated with the fabrication of a substrate (e.g., a good substrate) via a substrate processing system having characteristic values ​​that satisfy a threshold. Block 302 can be similar to block 202 in Figure 2A.

[0100] In block 304, the processing logic identifies the guard band associated with the trace data. The guard band can be generated based on method 200A in Figure 2A or method 200C in Figure 2C.

[0101] In block 306, the processing logic determines the guard band violation data points of the trace data based on the guard bands. Guard band violation data points include data points of the trace data that exceed the upper limit of the guard bands and / or data points of the trace data that fall below the lower limit of the guard bands.

[0102] In some embodiments, the trace data comes from different types of sensors, and the determination of guard band violation data points is performed via multivariate (e.g., multivariate) analysis.

[0103] In some embodiments, determining guard band violation data points involves segmenting portions of trace data associated with value changes that exceed threshold changes. In some embodiments, determining guard band violation data points involves extracting features from trace data.

[0104] In block 308, the processing logic determines the guard band violation shape characteristics based on the guard band violation data points (see Figures 6A to 6E).

[0105] In some embodiments, guard band violation shape characterization is one or more weighted combinations of the following: guard band violation duration (e.g., the number of sequential guard band violation data points is outside the range of the guard band limits), guard band violation magnitude (e.g., how much the guard band violation data points exceed the upper limit or how much they fall below the lower limit), guard band violation area (e.g., the area between the line passing through the guard band violation data points and the guard band limits), guard band violation location (e.g., the location of the guard band violation data points relative to the guard band limits), and / or intermittency of guard band violation (e.g., how often the guard band violation data points exceed the upper and / or lower guard band limits).

[0106] In some embodiments, determining guard band violation shape characterization involves concatenating consecutive violations into a single violation. Analyzing the behavior between consecutive violations allows for a better understanding of whether a violation is involved. Typical analyses may include the time between violations, the level of return to normal between violations, and the similarity of factors contributing to consecutive violations (e.g., x-direction shift). Concatenating consecutive violations improves guard band performance by identifying system problems rather than guard band violations. This reduces fluctuations in guard band violation reporting and allows for the incorporation of subjective expertise into guard band analysis.

[0107] Using guard band violation shape characterization, any guard band feature can be characterized in terms of parameters such as duration, level, and area. These parameters can be weighted to better capture specific violation types.

[0108] In block 310, the processing logic executes corrective actions associated with the substrate processing system based on the guard band violation shape characterization. Block 310 may also include classifying additional guard band violation data points in the additional trace data and determining whether to execute corrective actions based on the guard band violation shape characterization (see, for example, Figure 3B).

[0109] Referring to Figure 3B, in some embodiments, in block 322, the processing logic identifies trace data associated with substrate fabrication via a substrate processing system (e.g., the same substrate processing system as in Figure 3A). The trace data can be associated with substrates where it is unknown whether the substrate characteristic data meets a threshold (e.g., a good substrate) or does not meet a threshold (e.g., a defective substrate). Block 322 can be similar to block 222 in Figure 2B.

[0110] In block 324, the processing logic identifies the guard band associated with the trace data. Block 324 can be similar to block 304 in Figure 3A.

[0111] In block 326, the processing logic determines the guard band violation data points of the trace data based on the guard bands. Guard band violation data points include data points of the trace data that exceed the upper limit of the guard bands and / or data points of the trace data that fall below the lower limit of the guard bands. Block 326 can be similar to block 306 in Figure 3A.

[0112] In block 328, the processing logic identifies the guard band violation shape characterization. The guard band violation shape characterization can be determined by block 308 in Figure 3A. The guard band violation shape characterization can indicate whether a particular type of guard band violation (e.g., shape, area, duration, size, location, intermittency, etc.) should be classified as abnormal (e.g., take corrective action) or not abnormal (e.g., do not take corrective action).

[0113] In block 330, the processing logic determines the classification of the guard band violation data point based on the guard band violation shape characterization. In some embodiments, the classification indicates whether the guard band violation data point is abnormal or not. In some embodiments, the classification indicates the type of abnormality associated with the guard band violation data point. In some embodiments, the classification indicates the type of corrective action to be taken in relation to the guard band violation data point.

[0114] In block 332, the processing logic executes corrective actions associated with the substrate processing system based on the classification. The execution of corrective actions in block 332 can be similar to the execution of corrective actions in block 228 in Figure 2B.

[0115] Referring to Figure 3C, in some embodiments, in block 342, the processing logic identifies trace data associated with the fabrication of a substrate (e.g., a good substrate) via a substrate processing system having characteristic values ​​that satisfy a threshold. Block 302 may be similar to block 202 in Figure 2A, block 242 in Figure 2C, and / or block 302 in Figure 3A.

[0116] In block 344, the processing logic identifies historical performance data associated with past trace data. In some examples, the historical performance data indicates whether a board is a good board or a bad board (for example, all boards have characteristic values ​​that meet a threshold, or all boards have characteristic values ​​that do not meet a threshold). Block 344 can be similar to block 244 in Figure 2C.

[0117] In block 346, the processing logic identifies the guard band associated with the trace data. The guard band can be generated based on method 200A in Figure 2A or method 200C in Figure 2C. Block 346 may be similar to block 304 in Figure 3B.

[0118] In block 348, the processing logic determines past guard band violation data points in the past trace data based on the guard bands. Past guard band violation data points include data points in the past trace data that exceed the upper limit of the guard bands, and / or data points in the trace data that fall below the lower limit of the guard bands. Block 348 can be similar to block 306 in Figure 3A.

[0119] In block 350, the processing logic trains a machine learning model on a data input containing historical guard band violation data points (e.g., a target output containing historical performance data) to generate a trained machine learning model associated with guard band violation shape characterization to classify additional guard band violation data points. The trained machine learning model can be used as shown in Figure 3D. In some embodiments, guard band violation data points and violation shape characterization (e.g., shape summary statistics) are provided to the trained machine learning model to classify the guard band violation data points. In some embodiments, the guard band violation shape characterization is determined based on the guard band violation data points (see, for example, block 308 in Figure 3B).

[0120] Referring to Figure 3D, in some embodiments, in block 362, the processing logic identifies trace data associated with substrate fabrication via a substrate processing system (e.g., the same substrate processing system as in Figure 3C). The trace data can be associated with substrates where it is unknown whether the substrate characteristic data meets a threshold (e.g., a good substrate) or does not meet a threshold (e.g., a defective substrate). Block 362 can be similar to block 342 in Figure 3B.

[0121] In block 364, the processing logic identifies the guard band associated with the trace data. Block 364 can be similar to block 304 in Figure 3A, block 324 in Figure 3B, and / or block 344 in Figure 3C.

[0122] In block 366, the processing logic determines guard band violation data points in the trace data based on the guard bands. Guard band violation data points include data points in the trace data that exceed the upper limit of the guard bands and / or data points in the trace data that fall below the lower limit of the guard bands. Block 366 can be similar to block 306 in Figure 3A, block 326 in Figure 3B, and / or block 346 in Figure 3C.

[0123] In block 368, the processing logic provides guard band violation data points as input to a trained machine learning model associated with guard band violation shape characterization (e.g., trained via block 350 in Figure 3C). In some embodiments, the guard band violation data points and guard band violation shape characterization (e.g., shape summary statistics) are provided as input to the trained machine learning model to classify the guard band violation data points. In some embodiments, the guard band violation shape characterization is determined based on the guard band violation data points (e.g., see block 308 in Figure 3B).

[0124] In block 370, the processing logic receives output from the trained machine learning model that shows the predicted data.

[0125] In block 372, the processing logic determines the classification of one or more data points in the trace data based on the predicted data. The classification in block 372 can be similar to the classification in block 330 in Figure 3B.

[0126] In block 384, the processing logic executes corrective actions associated with the substrate processing system based on the classification. Block 384 can be similar to block 332 in Figure 3B.

[0127] Figures 4A to 4D are flowcharts of methods associated with a dynamic acceptable region outside the guard band limits (e.g., time-dependent variation in trace data) according to a particular embodiment. Figure 4A is a flowchart of method 400A associated with determining the acceptable region outside the guard band limits; Figure 4B is a flowchart of method 400B associated with using and optionally adjusting the acceptable region outside the guard band limits; Figure 4C is a flowchart of method 400C associated with training a machine learning model to determine the acceptable region outside the guard band limits; and Figure 4D is a flowchart of method 400D associated with using and optionally adjusting the acceptable region outside the guard band limits using the trained machine learning model.

[0128] Good wafer trace data may change over time due to acceptable drift, fluctuations, noise, and spikes.

[0129] Referring to Figure 4A, in some embodiments, in block 402, the processing logic identifies trace data associated with the fabrication of a substrate (e.g., a good substrate) via a substrate processing system having characteristic values ​​that satisfy a threshold. Block 302 may be similar to block 202 in Figure 2A and / or block 302 in Figure 3A.

[0130] In block 404, the processing logic determines the dynamically acceptable area outside the guard band limits based on the trace data.

[0131] In some embodiments, the processing logic determines the upper and lower limits of the guard bands based on the trace data (e.g., block 206 in Figure 2A, block 246 in Figure 2C, via 3σ from the mean of the trace data, etc.). The area between the upper and lower limits of the guard bands is a safe area (e.g., the green area), and data points within this area are considered healthy.

[0132] The processing logic determines the acceptable area outside the guard band limit. The acceptable area may be an alarm area (e.g., a yellow area), where the data points are still normal and used to track the motion state of the substrate processing system. An abnormal area (e.g., a red area) is outside the acceptable area, and data points within an abnormal area are considered positive (e.g., abnormal).

[0133] The acceptable range can be determined via user input (e.g., 1σ, 4 angstroms of sensor data variation, 4% sensor data variation, etc., outside the guard band limit). The acceptable range can change over time (e.g., a dynamically acceptable range). In some embodiments, a predetermined amount of variation in sensor values ​​from the guard band limit can be made acceptable for a given amount of time (e.g., a given run). For example, for 10 runs, a change of 4 angstroms or 4% in sensor values ​​from the guard band limit can be made acceptable.

[0134] The dynamic acceptable zone and / or guard band limits can change over time. For example, for a given run length (e.g., 10 runs), the acceptable zone and / or guard band limits can be adjusted by a predetermined amount (e.g., increasing by 4 angstroms, widening by 4%). Because the dynamic acceptable zone and / or guard band limits change over time, the new acceptable zone can be updated.

[0135] In block 406, the processing logic executes corrective actions associated with the substrate processing equipment based on the dynamically acceptable area outside the guard band limit. The execution of corrective actions can be based on the fact that additional trace data is outside the acceptable area (see Figure 4B).

[0136] Referring to Figure 4B, in some embodiments, in block 422, the processing logic identifies trace data associated with substrate fabrication via a substrate processing system (e.g., the same substrate processing system as in Figure 4A). The trace data can be associated with substrates where it is unknown whether the substrate characteristic data meets a threshold (e.g., a good substrate) or does not meet a threshold (e.g., a defective substrate). Block 422 can be similar to block 222 in Figure 2B and / or block 322 in Figure 3B.

[0137] In block 424, the processing logic compares the trace data with the dynamically acceptable region outside the guard band limits. The dynamically acceptable region can be determined by block 404 in Figure 4A.

[0138] In block 426, the processing logic updates the dynamically acceptable area outside the guard band limits based on the trace data, in response to one or more data points in the trace data being within the range of dynamically acceptable data.

[0139] In block 428, the processing logic executes a corrective action associated with the substrate processing equipment in response to one or more data points in the trace data being outside the range of dynamically acceptable data. The execution of the corrective action in block 428 may be similar to the execution of the corrective action in block 228 in Figure 2B and / or block 332 in Figure 3B.

[0140] Referring to Figure 4C, in some embodiments, in block 442, the processing logic identifies trace data associated with the fabrication of a substrate (e.g., a good substrate) via a substrate processing system having characteristic values ​​that satisfy a threshold. Block 302 may be similar to block 202 in Figure 2A, block 242 in Figure 2C, block 302 in Figure 3A, block 342 in Figure 3C, and / or block 402 in Figure 4A.

[0141] In block 444, the processing logic identifies historical performance data associated with past trace data. In some examples, the historical performance data indicates whether a board is a good board or a bad board (for example, all boards have characteristic values ​​that meet a threshold, or all boards have characteristic values ​​that do not meet a threshold). Block 444 can be similar to block 244 in Figure 2C and / or block 344 in Figure 3C.

[0142] In block 446, the processing logic trains a machine learning model using a data input containing historical trace data (e.g., a target output containing historical performance data) to generate a trained machine learning model that shows the dynamically acceptable region outside the guard band limits. The trained machine learning model can be used as shown in Figure 4D.

[0143] Referring to Figure 4D, in some embodiments, in block 462, the processing logic identifies trace data associated with substrate fabrication via a substrate processing system (e.g., the same substrate processing system as in Figure 4C). The trace data can be associated with substrates where it is unknown whether the substrate characteristic data meets a threshold (e.g., a good substrate) or does not meet a threshold (e.g., a defective substrate). Block 362 can be similar to block 442 in Figure 4B.

[0144] In block 464, the processing logic provides trace data as input to a trained machine learning model associated with a dynamically acceptable region outside the guard band limits (for example, the trained machine learning model in Figure 446 of Figure 4C).

[0145] In block 466, the processing logic receives output from the trained machine learning model that shows the predicted data.

[0146] In block 468, the processing logic updates the dynamic acceptable area outside the guard band limit in response to determining, based on the predicted data, that one or more data points in the trace data are within the acceptable area. Block 468 can be similar to block 426 in Figure 4B.

[0147] In block 470, the processing logic, based on the predicted data, determines that one or more data points in the trace data are outside the acceptable range and executes corrective actions associated with the substrate processing system. Block 470 may be similar to block 428 in Figure 4B.

[0148] Figures 5A to 5E show graphs 500A to E of acceptable dispersion types (see, for example, Figures 2A to 2D) according to a particular embodiment. Figures 5A to 5B show graphs 500A to B of variation between traces. Figure 5C shows graph 500C of an upper limit different from the lower limit. Figure 5D shows graph 500D of location-dependent dispersion. Figure 5E shows graph 500E of trace segmentation.

[0149] Referring to Figure 5A, Graph 500A shows trace data 502A-B. Trace data 502A can be sensor data (e.g., from one or more sensors) for the fabrication of a first substrate (e.g., one or more first substrates), and trace data 502B can be sensor data (e.g., from the same one or more sensors) for a second substrate (e.g., one or more second substrates). The acquisition of trace data 502B can be performed earlier in the substrate fabrication operation than the acquisition of trace data 502A, which can cause a shift in the x-direction (e.g., the x-axis is time, the y-axis is sensor value, and trace data 502A spikes earlier than trace data 502B). This is called inter-trace variation (e.g., variance).

[0150] Referring to Figure 5B, Graph 500B shows trace data over time associated with the fabrication of many substrates. A guard band 504A that does not account for variations between traces will result in many false positives (e.g., many guard band violation data points that do not correspond to defective substrates). In some examples, line 506 is calculated by taking the average of the trace data, and guard band 504A is created by taking 3σ from line 506, so that guard band 504A moves equally far from line 506 over time. Variations between traces will cause many false positives in guard band 504A, and simply expanding guard band 504A should create many missed detections (e.g., if guard band 504A is expanded, data points that deviate in the y direction should not be captured).

[0151] Method 200A in Figure 2A or Method 200C in Figure 2C generates a guard band 504B that takes into account the variation between traces. Using trace data from a good substrate, an acceptable type of variation is determined. For example, an initial guard band 504A can be created from trace data from a good substrate, and since the substrate all have characteristic values ​​that satisfy the threshold, the guard band 504A is expanded in the x direction (e.g., horizontally) within the area for forming the guard band 504B to accommodate the variation between traces.

[0152] Referring to Figure 5C, Graph 500C shows a different upper limit than the lower limit.

[0153] As discussed in Figure 5B, the guard band 504A may have upper and lower limits (e.g., 3σ) that are equally spaced from the line 506 through which the mean of the trace data passes. Trace data from a good wafer may have different amounts of dispersion above and below line 506. Simply making the upper and lower limits of the guard band 504A equally spaced can lead to many false positives.

[0154] Method 200A in Figure 2A or Method 200C in Figure 2C generates a guard band 504B that considers different upper and lower limits. For example, as shown in Figure 5C, at the beginning of the transition, an upper variance 508A may occur that is greater than the lower variance 508B, and at the end of the transition, a lower variance 508D may occur that is greater than the upper variance 508C.

[0155] Referring to Figure 5D, Graph 500D shows the position-dependent variance.

[0156] As discussed in Figure 5B, the guard band 504A may have upper and lower limits (e.g., 3σ) at the same distance over time from the line 506 that passes through the mean of the trace data. Trace data from a good wafer may have different amounts of variance at different time points. Having a guard band 504A of the same size over time can lead to many false positives.

[0157] Method 200A in Figure 2A or Method 200C in Figure 2C generates a guard band 504B that takes into account different amounts of dispersion 510 over time. For example, as shown in Figure 5D, a lower dispersion 510A may occur in a flat region (e.g., a smaller distance between the upper and lower limits of the guard band 504B for a good substrate), and a higher dispersion 510B may occur in a transient region (e.g., a larger distance between the upper and lower limits of the guard band 504B for a good substrate).

[0158] Referring to Figure 5E, Graph 500E shows the trace segmentation graph 500E. As discussed in Figure 5B, the guard band 504A may have upper and lower limits (e.g., 3σ) at the same distance over time from the line 506 that passes through the mean of the trace data. In some embodiments, the change in the trace data is greater than the threshold amount (e.g., see area 512 in Figure 5E).

[0159] If two boundaries are close to each other, the central position can be used. If one of the boundaries is close to another boundary with a change greater than a threshold amount, that segment boundary can be maintained.

[0160] Two segments of the trace data showing abrupt changes in the original boundary cannot be removed from the trace data. Based on the segmentation knowledge regarding the abrupt changes in segmentation points in area 512, the parameters of the guard band can be adjusted.

[0161] The rapid changes in area 512 could be one or more of the following:

[0162] 1) Gradient change of a segment of trace data from negative to positive or positive to negative (for example, slope(right)*slope(left)<0),

[0163] 2) The gradient of the right segment and / or the left segment exceeds the threshold gradient (for example, abs(slope(right))>0.1 or abs(slope(left))>0.1 (normal value)), and / or

[0164] 3) The mean variance (e.g., standard deviation) of two connecting segments is greater than the mean variance (e.g., standard deviation) of the next two segments (e.g., mean(std[right,left])>2*mean(std[right_2,left_2]).

[0165] If two adjacent segments of trace data have 1), 2), and 3) above, the segment boundary (for example, a data point within region 512) can be considered a fixed boundary. In some embodiments, if region 512 is a segment boundary, the guard band 504B is generated for those points within region 512, rather than excluding them from the generation of the guard band 504B. In some embodiments, if region 512 is a segment boundary, the data points within region 512 become the limits of the guard band 504B, which have a smaller acceptable variance.

[0166] Figures 6A to 6E show guard band violation profiling according to a specific embodiment (see, for example, Figures 3A to 3D).

[0167] Referring to Figure 6A, trace data is shown for each of graphs 602A–Z (e.g., block 302 in Figure 3A, block 344 in Figure 3C). Each graph 602A–Z can show trace data for the fabrication of a different substrate, and each substrate is a good substrate (e.g., has characteristic values ​​that meet the threshold). Each graph 602A–Z also shows a guard band, and each graph 602A–Z has several sets of data points that are outside the range of the guard band limit. These sets of data points are called the guard band violation data point set 604A–Z (e.g., the set of data points associated with a guard band violation). Since the trace data is for a good substrate, the guard band violation data point set 604A–Z violates the guard band but does not indicate a bad substrate (e.g., these are false positives).

[0168] For each set of guard band violation data points 604 in the trace data, a parameter 606 is extracted. The parameter 606 may include area, duration, magnitude, etc. In some examples, the area may be the area between the guard band limit and the portion of the trace data passing through the trace data that is outside the guard band limit. In some examples, the duration may be the amount of time that sequentially the guard band violation data points are outside the guard band limit. The magnitude may be the magnitude (e.g., the value of y) of the difference (e.g., in the y direction) between the guard band violation data point and the guard band limit. In some examples, each guard band violation data point may have a corresponding parameter input.

[0169] Graph 608 can be formed using parameter 606 and the joint probability density function of parameter 606. Graph 608 can be generated by fitting a multivariate Gaussian distribution (e.g., finding the joint probability density function of three variables). The set of guard band violation data points 604 can form a guard band violation shape characterization 610 (e.g., a circle on graph 608) surrounding the set of guard band violation data points 604 (e.g., good guard band violation data points 614A).

[0170] Graph 612 can be generated using graph 608, which includes the guard band violation shape characterization 610. Graph 12 displays the guard band violation shape characterization 610, which separates good guard band violation data points 614A corresponding to good substrates from bad guard band violation data points 614B corresponding to bad substrates.

[0171] In response that the set of guard band violation data points 604 is for a good substrate, the guard band violation shape characterization 610 surrounds (for example, encloses) the data points for the good substrate in graph 608. The guard band violation shape characterization 610 can be used to determine whether a future set of guard band violation data points 604 is a good guard band violation data point 614A corresponding to a good substrate (for example, within the range of the guard band violation shape characterization 610 on graph 608 and below the guard band violation shape characterization 610 on graph 612) or a good guard band violation data point 614A corresponding to a poor substrate (for example, outside the range of the guard band violation shape characterization 610 on graph 608 and above the guard band violation shape characterization 610 on graph 612).

[0172] In some embodiments, the set of guard band violation data points 604 used for parameter 606, graph 608, and graph 612 corresponds to a defective substrate (e.g., a substrate with characteristic values ​​that do not meet a threshold). In this case, the guard band violation shape characterization 610 surrounds (e.g., encloses) the data points of the defective substrate in graph 608. The guard band violation shape characterization 610 can be used to determine whether a future substrate is defective (e.g., within the range of the guard band violation shape characterization 610 on graph 608 and below the guard band violation shape characterization 610 on graph 612) or not (e.g., outside the range of the guard band violation shape characterization 610 on graph 608 and above the guard band violation shape characterization 610 on graph 612).

[0173] In response to the set of guard band violation data points 604 being for a specific type of substrate (for example, a specific type of defective substrate having specific characteristic values ​​that do not meet the threshold), the guard band violation shape characterization 610 surrounds (for example, encloses) the data points for the specific type of substrate. The guard band violation shape characterization 610 can be used to determine whether a future substrate is a specific type of substrate (for example, within the range of the guard band violation shape characterization 610 on graph 608 and below the guard band violation shape characterization 610 on graph 612) or not (for example, outside the range of the guard band violation shape characterization 610 on graph 608 and above the guard band violation shape characterization 610 on graph 612).

[0174] Figure 6B shows a graph 620 for guard band violation profiling according to a particular embodiment. Graph 620 includes a band mean 622 (e.g., mean trace data), a guard band limit 624, an offset 626 between the guard band limit 624 and the band mean 622, and a guard band violation data point 628. Graph 620 has a gap 638 between the guard band violation data point 628A for the first case and the guard band violation data point 628B for the second case.

[0175] The violation duration 630 of a guard band violation data point 628 is the distance between a first guard band violation data point 628A outside the range of the guard band limit 624 and the last guard band violation data point 628B outside the range of the guard band limit 624 (for example, the duration of the violation in the x direction).

[0176] The magnitude of the violation 632 (e.g., the maximum violation, the peak of the violation) is the distance between the guard band limit 624 and the guard band violation data point 628 (e.g., the guard band violation data point 628 furthest from the guard band limit 624 in the y-direction).

[0177] The violation area 634 is the area between the line passing through the guard band violation data point and the guard band limit 624 (for example, the average value of violations).

[0178] The violation location 636 can be the location of the largest violation magnitude 632 (for example, the x-value and time value where the violation occurs).

[0179] The discontinuity of the violations may include the combination or division of two or more violations.

[0180] The guard band violation shape characterization 610 can further be based on one or more of the violation duration 630, violation size 632, violation area 634, and / or violation location 636. Guard band violation data points 628 corresponding to violation duration 630, violation size 632, violation area 634, and / or violation location 636 that satisfy the threshold can be ignored. Guard band violation data points 628 corresponding to violation duration 630, violation size 632, violation area 634, and / or violation location 636 that satisfy the threshold can correspond to the type of substrate (e.g., good substrate, bad substrate, type of bad substrate, etc.).

[0181] Figure 6C shows a graph 640 for guard band violation profiling according to a particular embodiment. Graph 640 includes a bandwidth mean 622 (e.g., mean trace data), a guard band limit 624, instances of guard band violation data points 628, gaps 638 between two instances of guard band violation data points 628, and the magnitude of the violation 632 between the bandwidth mean 622 and the last guard band violation data point 628 of the first instance. Line 642 lies between the last guard band violation data point 628C of the first instance and the data point of the bandwidth mean 622 corresponding to the first guard band violation data point 628D of the second instance.

[0182] The guard band violation shape characterization 610 can classify the substrate or trace data as good, bad, or a kind of bad based on the number of violations. Depending on the length of the line 642 that satisfies the threshold (e.g., being sufficiently large), the guard band violation data point 628C in the first case and the guard band violation data point 628D in the second case are considered a single violation; otherwise, the guard band violation data point 628C in the first case and the guard band violation data point 628D in the second case are considered separate violations.

[0183] Figure 6D shows a flowchart of Method 660 for guard band violation profiling according to a specific embodiment. Classification of guard band violation data points can be performed via multivariate (e.g., multivariate) analysis (e.g., based on sensor data from different types of sensors).

[0184] In some embodiments, trace data from sensors 662A-N are compared separately with the corresponding guard bands 664A-N. Test trace data 670 is provided by comparing the guard band violation data points from the comparison of trace data from sensors 662 compared with guard bands 664 via a violation probability density function (pdf) 666 with a threshold 668.

[0185] The test trace data provides a score 672 for each sensor 662. Based on the score 672, score ranking 674 is performed. Based on the score ranking 674, sensor importance ranking 676 is performed. Based on the sensor importance ranking 676, a composite score 677 is generated. The composite score 677 is compared to a composite threshold 678 to provide defect detection results 679 (e.g., classification of guard band violation data points as good or bad substrates, anomaly detection results).

[0186] Figure 6E shows a flowchart of method 680 for guard band violation profiling according to a specific embodiment. Guard band violation shape characterization can be performed based on guard band violation data points and segmented feature extraction.

[0187] In block 682, trace data is identified. This can be similar to block 302 in Figure 3A and / or block 342 in Figure 3C for generating guard bands. This can be similar to block 322 in Figure 3B and / or block 362 in Figure 3D for using guard bands.

[0188] In block 684, a full trace analysis is performed on the trace data from block 682. For example, it can determine the area, duration, size, etc. (see, for example, Figures 6A to 6C).

[0189] In block 686, a guard band model is generated based on a full trace analysis of the trace data. This can be similar to block 206 in Figure 2A and / or block 246 in Figure 2C.

[0190] In block 688, anomalous features (e.g., guard band violation data points) are determined by comparing the trace data with a guard band model (e.g., guard bands). This can be similar to block 306 in Figure 3A, block 326 in Figure 3B, block 346 in Figure 3C for training the model, and / or block 366 in Figure 3D. The anomalous features (e.g., guard band violation data points) may be persistent or intermittent.

[0191] In block 690, segmented feature extraction is performed on the trace data of block 682. This can be similar to Figure 5E (for example, segment boundaries such as data points within area 512 in Figure 5E are considered fixed boundaries). In some embodiments, a guard band model can generate anomalous features in block 688, which can be combined with features identified in block 690 (for example, persistent and intermittent features of block 692).

[0192] In block 692, persistent features of the trace data are determined based on the segmented feature extraction in block 690. Persistent features can be trace data features that satisfy the threshold amount of occurrence.

[0193] In block 694, feature parameter correlation of persistent features in the trace data is performed (for example, based on subjective expertise 699, such as user input). Feature parameter correlation can be performed based on parameters such as size, location, area, and duration (see Figure 6B). In some embodiments, guard band violation shape characterization can classify guard band violation data points corresponding to persistent features as being associated with a good substrate.

[0194] In block 696, intermittent features of the trace data are determined based on the segmented feature extraction in block 690. Intermittent features can be defined as trace data features that do not meet the threshold amount for occurrence.

[0195] In block 698, feature presence correlation is performed based on intermittent features from block 696, anomalous features from block 688, and / or subject matter expertise 699. In some embodiments, guard band violation shape characterization can classify guard band violation data points corresponding to intermittent features as being associated with a faulty substrate.

[0196] Figures 7A to 7F show the dynamically acceptable area outside the guard band limit according to a specific embodiment.

[0197] Figure 7A shows a flowchart of method 700 associated with the dynamic area outside the guard band limit.

[0198] In block 702, the processing logic identifies baseline trace data. This can be similar to block 302 in Figure 3A and / or block 342 in Figure 3C for generating guard bands. This can be similar to block 322 in Figure 3B and / or block 362 in Figure 3D for using guard bands.

[0199] In block 704, the processing logic identifies guard bands based on trace data. Guard bands can be generated by block 206 in Figure 2A and / or block 246 in Figure 2C.

[0200] In block 706, the processing logic performs a multivariate analysis (MVA) distribution based on the guard band and baseline trace data.

[0201] In block 708, the processing logic identifies probabilities on the baseline trace data. These probabilities may be the probability that a data point is in a safe area (e.g., area 736 in graph 734 or graph 742 in Figure 7B), an alarm area (e.g., area 738 in graph 734 or graph 742 in Figure 7B), or an abnormal area (e.g., area 740 in graph 734 or graph 742 in Figure 7B).

[0202] In block 710, the processing logic generates an internal threshold. The internal threshold can be a line that separates an alarm area (for example, area 738 in graph 734 or graph 742 in Figure 7B) from an abnormal area (for example, area 740 in graph 734 or graph 742 in Figure 7B).

[0203] In block 712, the processing logic generates extended trace data. The extended trace data can be simulated trace data formed by adjusting the baseline trace data of block 702 with one or more of blocks 714 to 720.

[0204] In block 714, the extended trace data may include trace data with slight drift. The drift may include increasing the sensor value in the y direction of the baseline trace data in block 702.

[0205] In block 716, the extended trace data may include trace data with minor repetitions. One or more portions of the baseline trace data in block 702 can be repeated over time (for example, in the x-direction).

[0206] In block 718, the extended trace data may include trace data with minor noise (e.g., and / or variation). The baseline trace data in block 702 may be adjusted (e.g., increased and decreased) in the y direction to mimic noise.

[0207] In block 720, the extended trace data may include trace data with minor spikes. The baseline trace data in block 702 may include peaks and valleys to mimic minor spikes.

[0208] In block 722, the processing logic identifies probabilities on the baseline trace data and the extended trace data. Block 722 can be similar to block 708.

[0209] In block 724, the processing logic generates an external threshold. Block 724 can be similar to block 710.

[0210] Figure 7B shows the dynamic area outside the guard band limits. Graph 730 shows baseline trace data (for example, block 702), and graph 732 shows extended trace data (for example, block 712).

[0211] Graph 734 shows areas 736 (e.g., acceptable areas, green areas), 738 (e.g., warning areas, yellow areas), and 740 (e.g., abnormal areas). Graph 734 can be formed based on trace data from a good substrate. All data points on Graph 734 can be located within areas 736 or 738.

[0212] Graph 742 shows new trace data for a substrate to be determined to be good or bad. Graph 742 has trace data in areas 736, 738, and 740. Data points in area 740 correspond to bad substrates. Area 738 (e.g., the yellow area, alarm area) should be recalculated based on the data points in area 738.

[0213] Figure 7C shows a flowchart of method 744 associated with the dynamic area outside the guard band limits.

[0214] In block 746, the processing logic identifies a training set of trace data. The training set can be previous trace data used to manufacture a circuit board by a circuit board processing machine.

[0215] In block 748, the processing logic identifies new trace data. In some embodiments, the new trace data includes new sensor data associated with fabricating a new substrate using the same or different substrate processing equipment as in block 746. In some embodiments, the new trace data includes simulated trace data created based on the trace data of block 746, and includes one or more of the following: drift, noise, spikes, fluctuations, etc.

[0216] In block 750, the processing logic ranks the new trace data from block 748. The new trace data can be ranked based on representing (e.g., being close to) the trace data in block 746. The new trace data can be ranked based on their proximity to each other (e.g., removing anomalies). The data points of the new trace data can be ranked as good (e.g., in the green area), acceptable and used for guard band adjustment (e.g., in the yellow area), or abnormal (e.g., in the red area). For example, the data points of the new trace data can be within the green area (e.g., no input guard band adjustment), within the acceptable yellow area (e.g., no anomalies but with input for GB adjustment), or within the abnormal red area (e.g., abnormal but without input for guard band adjustment).

[0217] In block 752, the processing logic selects at least a portion of the new trace data from block 748 (for example, based on the ranking from block 750). Based on the ranking in block 750, the processing logic may select the highest-ranked trace data.

[0218] In block 754, the processing logic updates (e.g., see Figure 7E) or retrains (e.g., see Figure 7F) old guard bands based on trace data selected from the training set and / or block 752. The processing logic can trigger a guard band update when certain criteria are met (e.g., the trace data is noisy, the trace data has no drift, etc.).

[0219] In block 756, the processing logic identifies a new guard band that has been updated or retrained from block 754.

[0220] Figure 7D shows the dynamic region outside the guard band limits. Graph 760 shows new trace data (e.g., from block 748 in Figure 7C), and Graph 762 shows the trace ranking of the trace data in Graph 760 (e.g., see block 750 in Figure 7C).

[0221] Graph 764 shows the selected trace (for example, from block 752 in Figure 7C), and Graph 766 shows the adapted guard band (for example, the new guard band in block 756 in Figure 7C) updated from the selected trace (for example, the most useful trace) in Graph 764.

[0222] Figure 7E shows the dynamic region outside the guard band limits for noise. Block diagram 770A shows that process data (e.g., trace data) is accumulated to generate the guard band. In response to the processing logic detecting noise (e.g., periodic changes), the processing logic updates the guard band with all past data.

[0223] Graph 772A shows the initial trace data (e.g., the first 60 traces), and Graph 774A shows the initial guard bands for the trace data in Graph 772A.

[0224] Graph 776A shows trace data (e.g., the first 120 traces), and Graph 778A shows the subsequent guard bands relative to the trace data in Graph 776A. As shown in Graph 778A, the guard bands widen over time to accommodate an increase in acceptable noise.

[0225] Figure 7F shows the dynamic region outside the guard band limits for drift. Block diagram 770A shows that older historical data (e.g., previous trace data) is forgotten, and only a certain amount (e.g., zero or more) of more recent historical data is combined with newer process data (e.g., current trace data) to generate the guard band. In response to the processing logic detecting drift (e.g., a change in the y-direction of the sensor value over time), the processing logic triggers the forgetting mechanism and updates the guard band with only the most recent trace data.

[0226] Graph 772B shows the initial trace data (e.g., the first 50 traces), and Graph 774B shows the initial guard bands for the trace data in Graph 772B.

[0227] Graph 776B shows trace data (e.g., the next 50 traces), and Graph 778B shows the next guard band for the trace data in Graph 776B. As shown in Graph 778B, the guard band moves in the y-direction over time (e.g., rises) to accommodate acceptable drift.

[0228] Figures 8A to 8B show guard band fitting according to a particular embodiment. The guard band fitting shown in any part of Figures 8A to 8B can be used in any of the methods of this disclosure (for example, Figures 2A to 2D, Figures 3A to 3D, and / or Figures 4A to 4D) to fit the system's motion state (but not to fit violations such as defect level shifts).

[0229] Figure 8A shows a horizontal scaling change. Graph 810A shows trace data 812A and trace data 812B. Trace data 812A and 812B can have different scaling in the y direction. As shown in Graph 810B, trace data 812A and / or trace data 812B can undergo a horizontal scaling change (for example, a horizontal scaling change at the policy endpoint).

[0230] In some embodiments, trace data 812A and 812B are for a good substrate. By changing the horizontal magnification, a more accurate guard band can be created based on the trace data 812A and 812B. By changing the horizontal magnification of the trace data, which can correspond to good or bad substrates, the difference between the trace data and the guard band can be identified more accurately (for example, reducing false detections and missed detections).

[0231] Figure 8B shows vertical scaling and horizontal distortion correction. Graph 850A shows trace data 812A and trace data 812B with different scaling in the y direction. By performing vertical scaling on trace data 812A and 812B in graph 850A, graph 850B can be generated (e.g., amplitude is normalized). By performing horizontal distortion correction on trace data 812A and 812B in graph 850B, graph 850C can be generated (e.g., dynamic time distortion correction).

[0232] Trace distortion correction and scaling can ignore factors of different amplitudes with phase shifts. Horizontal distortion correction is applied to ignore phase shift factors and maintain vertical noise. Vertical and horizontal scaling is applied to address region shifts (e.g., applying guard bands to different regions, such as different policies).

[0233] Figure 9 is a block diagram showing a computer system 900 according to a particular embodiment. In some embodiments, the computer system 900 can be connected to other computer systems (for example, via a network such as a local area network (LAN), intranet, extranet, or internet). The computer system 900 can operate as a server or client computer in a client-server environment, or as a peer computer in a peer-to-peer or distributed network environment. The computer system 900 can be provided by a personal computer (PC), tablet PC, set-top box (STB), personal digital assistant (PDA), cellular telephone, web device, server, network router, switch or bridge, or any device capable of executing (sequentially or otherwise) a set of instructions specifying actions to be taken by such device. Furthermore, the term “computer” includes any group of computers that individually or collectively execute one or more sets of instructions in order to perform any one or more of the methods described herein.

[0234] In a further embodiment, the computer system 900 may include a processing device 902, a volatile memory 904 (e.g., random access memory (RAM)), a non-volatile memory 906 (e.g., read-only memory (ROM) or electrically erasable programmable ROM (EEPROM)), and a data storage device 918, which can communicate with each other via a bus 908.

[0235] The processing device 902 can be provided by one or more processors, such as a general-purpose processor (e.g., a composite instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a microprocessor that implements other types of instruction sets, or a microprocessor that implements a combination of instruction set types) or a specialized processor (e.g., an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), or a network processor).

[0236] The computer system 900 may further include a network interface device 922 (for example, connected to a network 974). The computer system 900 may also include a video display unit 910 (for example, an LCD), an alphanumeric input device 912 (for example, a keyboard), a cursor control device 914 (for example, a mouse), and a signal generation device 920.

[0237] In some embodiments, the data storage device 918 may include a non-temporary computer-readable storage medium 924 (e.g., a non-temporary machine-readable storage medium) capable of storing instructions 926 that encode one or more of the methods or functions described herein, the instructions 926 of which encode components of Figure 1 (e.g., predictive component 114, model 190, etc., used for prediction or detection) and instructions for performing the methods described herein. When these instructions are executed, a processing device can be caused to perform the methods described herein.

[0238] Instruction 926 may also reside entirely or partially in the volatile memory 904 and / or processing device 902 while it is being executed by the computer system 900, so that the volatile memory 904 and processing device 902 may also constitute a machine-readable storage medium.

[0239] In the descriptive example, computer-readable storage medium 924 is shown as a single medium, but the term “computer-readable storage medium” includes a single or multiple mediums that store one or more sets of executable instructions (for example, a centralized or distributed database, and / or associated caches and servers). The term “computer-readable storage medium” also includes any tangible medium capable of storing or encoding a set of instructions that cause a computer to execute one or more of the methods described herein for execution by a computer. The term “computer-readable storage medium” also includes, but is not limited to, solid memory, optical media, and magnetic media.

[0240] The methods, components, and features described herein can be implemented by individual hardware components or integrated into the functionality of other hardware components such as ASICS, FPGAs, DSPs, or similar devices. In addition, the methods, components, and features can be implemented by firmware modules or functional circuits within hardware devices. Furthermore, the methods, components, and features can be implemented in any combination of hardware devices and computer program components, or within computer programs.

[0241] Unless otherwise specifically stated, terms such as “identify,” “generate,” “cause,” “provide,” “receive,” “determine,” “update,” “compare,” “train,” “correct distortion,” “change magnification,” and “acquire” refer to actions and processes performed or implemented by a computer system that manipulate and convert data represented as physical (electronic) quantities in computer system registers and memory into other data similarly represented as physical quantities in computer system memory or registers or other such information storage, transmission, or display devices. Furthermore, in this specification, terms such as “first,” “second,” “third,” and “fourth” are meant to be labels for distinguishing different elements and do not necessarily have an orderly meaning according to numerical indication.

[0242] The examples described herein also refer to apparatus for performing the methods described herein. Such apparatus may be specifically constructed to perform the methods described herein, or may include a general-purpose computer system selectively programmed by a computer program stored within the computer system. Such a computer program may be stored in a computer-readable tangible storage medium.

[0243] The methods and examples described herein are not inherently related to any particular computer or other device. Various general-purpose systems can be used in accordance with the teachings described herein, or it may be advantageous to construct more specialized devices to perform each of the methods and / or individual functions, routines, subroutines, or operations described herein. Examples of structures for various such systems are described above.

[0244] The above description is intended to be illustrative, not restrictive. While this disclosure has been described by reference to specific examples and embodiments, it should be understood that this disclosure is not limited to the examples and embodiments described. The scope of this disclosure should be defined with reference to the following claims, along with the entire scope of equivalents given thereto.

Claims

1. Identifying guard band limits related to acceptable dispersion types based on past trace data, Identifying trace data containing multiple data points over a certain period, wherein the trace data is associated with the fabrication of a substrate having characteristic values ​​that satisfy a threshold via a substrate processing system. Based on the trace data, a dynamically acceptable area outside the guard band limit is determined, wherein the amount of variation in the trace data from the guard band limit is within a predetermined amount and is within an acceptable range. In response to one or more data points of additional trace data that are outside the dynamically acceptable area, the corrective action associated with the substrate processing system is performed. A method that includes this.

2. The method according to claim 1, wherein the dynamically acceptable area is associated with one or more of the drift or noise associated with the substrate processing system.

3. The determination of the dynamically acceptable area outside the guard band limit includes training a machine learning model with data input including the trace data to generate a trained machine learning model that indicates the dynamically acceptable area outside the guard band limit, The machine learning model determines whether the trace data is outside the range of the dynamically acceptable area, which is outside the range of the guard band limit. The machine learning model updates the dynamically acceptable area if the trace data is within the range of the dynamically acceptable area which is outside the range of the guard band limit. The method according to claim 1, wherein the machine learning model performs the corrective action if the trace data is outside the range of the dynamically acceptable area which is outside the range of the guard band limit.

4. The method according to claim 1, further comprising updating the guard band limit in response to incoming trace data that falls within the range of the dynamically acceptable area, taking into account at least one of the mean, variance, and variability of the incoming trace data.

5. The method according to claim 1, further comprising horizontally correcting distortion in a portion of the trace data without vertical distortion correction, so as to maintain vertical noise while ignoring phase shift factors.

6. The method according to claim 1, further comprising scaling portions of the trace data vertically and horizontally so as to ignore different amplitudes and policy endpoints.

7. The method according to claim 1, wherein the trace data includes historical trace data and simulated trace data, the simulated trace data is generated by applying one or more of the following to the historical trace data: drift, fluctuation, noise, or spikes.

8. The implementation of the aforementioned corrective measures is The aforementioned additional trace data is provided as input to the trained machine learning model, The trained machine learning model receives an output containing predictive data, The method according to claim 1, further comprising determining, based on the prediction data, that at least a portion of the additional trace data is outside the range of the dynamically acceptable area which is outside the range of the guard band limit.

9. A non-temporary computer-readable storage medium that stores instructions for causing a processing device to perform an action when executed, wherein the action is Based on historical trace data, identify guard band limits associated with acceptable dispersion types, Identifying trace data containing multiple data points over a certain period, wherein the trace data is associated with the fabrication of a substrate having characteristic values ​​that satisfy a threshold via a substrate processing system. Based on the trace data, a dynamically acceptable area outside the guard band limit is determined, wherein the amount of variation in the trace data from the guard band limit is within a predetermined amount and is within an acceptable range. A non-temporary computer-readable storage medium, which includes performing corrective actions associated with the substrate processing system in response to one or more data points of additional trace data that are outside the dynamically acceptable area and outside the range of the guard band limits.

10. The non-temporary computer-readable storage medium according to claim 9, wherein the dynamically acceptable area is associated with one or more of the drift or noise associated with the substrate processing system.

11. The determination of the dynamically acceptable area outside the guard band limit includes training a machine learning model with data input including the trace data to generate a trained machine learning model that indicates the dynamically acceptable area outside the guard band limit, The machine learning model determines whether the trace data is outside the range of the dynamically acceptable area, which is outside the range of the guard band limit. The machine learning model updates the dynamically acceptable area if the trace data is within the range of the dynamically acceptable area which is outside the range of the guard band limit. The non-temporary computer-readable storage medium according to claim 9, wherein the machine learning model performs the corrective action if the trace data is outside the range of the dynamically acceptable area which is outside the range of the guard band limit.

12. The non-temporary computer-readable storage medium according to claim 9, wherein the operation further comprises updating the guard band limit in response to incoming trace data within the range of the dynamically acceptable area, taking into account at least one of the mean, variance, and variation of the incoming trace data.

13. The non-temporary computer-readable storage medium according to claim 9, wherein the operation further comprises horizontally correcting distortion on a portion of the trace data without vertical distortion correction, so as to maintain vertical noise while ignoring phase shift factors.

14. The non-temporary computer-readable storage medium according to claim 9, further comprising scaling portions of the trace data vertically and horizontally so as to ignore different amplitudes and policy endpoints.

15. The non-temporary computer-readable storage medium according to claim 9, wherein the trace data includes historical trace data and simulated trace data, the simulated trace data is generated by applying one or more of the following to the historical trace data: drift, fluctuation, noise, or spikes.

16. The execution of the corrective measure is The aforementioned additional trace data is provided as input to the trained machine learning model, The trained machine learning model receives an output containing predictive data, The non-temporary computer-readable storage medium according to claim 9, further comprising determining, based on the prediction data, that at least a portion of the additional trace data is outside the range of the dynamically acceptable area which is outside the range of the guard band limit.

Images