Monitoring and simulating chamber conditions after preventative maintenance

Abstract

The subject matter herein may be embodied in, among other things, a method, a system, and a computer readable storage medium. The method may include a processing device receiving sensor data indicative of a condition of an environment of a processing chamber in which a series of substrates are processed according to a substrate processing procedure. The series of substrates may be processed after a preventive maintenance procedure. The processing device may determine a first set of values ​​based on the sensor data. The set of values ​​may be indicative of a likelihood that an associated substrate will include a first process result that satisfies a threshold condition of the substrate processing procedure. The processing device may further predict a first test result based on the first set of values. The first test result may be indicative of a likelihood that a first substrate processed after processing the series of substrates will include a second process result that satisfies the threshold condition.

Description

[Technical Field]

[0001] This disclosure relates generally to monitoring and simulating chamber conditions. More particularly, this disclosure relates to monitoring and simulating chamber conditions related to chamber seasoning. [Background technology]

[0002] Substrate processing may involve a series of processes to form electrical circuits on semiconductor, e.g., silicon, wafers according to a circuit design. These processes may be performed in a series of chambers. Successful operation of modern semiconductor manufacturing facilities may aim to facilitate a steady flow of wafers moving from one chamber to another while the electrical circuits are formed on the wafers. During the course of performing many substrate processes, conditions in the processing chambers may change, resulting in processed substrates that do not meet the desired conditions and results.

[0003] One such substrate process may include plasma etching. Plasma etching is a process that transfers a pattern from a mask material layer to another layer, such as a conductive or dielectric material layer, beneath the mask, by removing these deposited materials from the wafer surface. Such processes inevitably produce different types of etching by-products, such as silicon oxides and organic polymers, depending on the deposited material and etching chemistry. Some of the by-products deposit on the interior surfaces of the chamber in which the plasma etching process is performed. The deposition of by-products may affect etching performance, such as by depositing particles (e.g., flakes) on the substrate or by reacting with the plasma and affecting the process results. Summary of the Invention [Problem to be solved by the invention]

[0004] To mitigate the effects of etching byproducts, preventative maintenance such as chamber cleaning may be used to periodically remove deposits from the chamber walls. To do this, the chamber is removed from production and a cleaning plasma, e.g., a CF4 + O2 plasma, is introduced into the chamber to clean silicon oxide deposited during silicon etching. This plasma reacts with the deposits, and the products of this reaction are pumped out of the chamber. However, it has been observed that after such chamber cleaning, the clean chamber walls make the chamber unsuitable for immediate production wafer etching. Chamber seasoning is the procedure of etching a series of substrates (e.g., blank silicon wafers) to restore chamber conditions suitable for production substrate processing. After chamber seasoning, a thin layer of silicon oxide coats the chamber walls. The chamber is then returned to production wafer etching until the next chamber cleaning and seasoning is required. [Means for solving the problem]

[0005] A method and system for post-preventive maintenance monitoring and simulation is implemented. In some embodiments, a method performed by a processing device may include receiving sensor data indicative of an environmental condition of a processing chamber processing a series of substrates according to a chamber seasoning procedure. The series of substrates is processed after a preventive maintenance procedure performed on the processing chamber. The processing device may determine a first set of values ​​based on the sensor data. Each value in the first set may be associated with an individual substrate in the series of substrates. The first set of values ​​may indicate a likelihood that the associated substrate will include a first process result that satisfies a threshold condition of the substrate processing procedure. The processing device may predict a first test result based on the first set of values. The first test result may indicate a likelihood that the processed first substrate, after processing the series of substrates, will include a second process result that satisfies the threshold condition. The processing device may prepare the first test result for presentation on a graphical user interface (GUI). The processing device may alter operation of the processing chamber based on the first test result.

[0006] In some embodiments, a method for training a machine learning model to predict a condition of a processing chamber after a preventive maintenance procedure is performed. The method includes generating training data for the machine learning model. Generating the training data may include identifying a first training input having first sensor data. The first sensor data indicates a first state of the first processing chamber after performing the preventive maintenance procedure. Generating the training data may further include identifying a first target output for the first training input. The first target output may indicate whether a substrate processed according to a first substrate processing procedure by the first processing chamber in the first state will result in the first substrate having a first process result that satisfies a first threshold condition associated with the first substrate processing procedure. The method may further include providing training data to train the machine learning model with respect to (i) a set of training inputs including the first training input and (ii) a set of target outputs including the first target output. The trained machine learning model may receive a new input having new sensor data indicating a new state of the new processing chamber. The new output may indicate whether a new substrate processed according to the first substrate processing procedure by the new processing chamber in the new state results in the new substrate having new process results that satisfy the threshold condition.

[0007] In some embodiments, the above-described methods may be stored on a non-transitory machine-readable storage medium including instructions that, when executed by a processing device, cause the processing device to perform operations that may include one or more of the steps described in the above-described methods.

[0008] Aspects and embodiments of the present disclosure will be more fully understood from the following detailed description and the accompanying drawings, which are intended to illustrate aspects and embodiments by way of example and not by way of limitation. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a block diagram illustrating an exemplary system architecture in which embodiments of the present disclosure may function. [Figure 2A] 2A-2B are block diagrams illustrating chamber recovery processes 200A-B in which embodiments of the present disclosure may function. [Figure 2B] 2A-2B are block diagrams illustrating chamber recovery processes 200A-B in which embodiments of the present disclosure may function. [Figure 3] FIG. 2 is a block diagram illustrating a chamber monitoring and maintenance system 200 in which embodiments of the present disclosure may function. [Figure 4] 10 is a graph illustrating a chamber recovery profile according to some embodiments of the present disclosure. [Figure 5] 1 is a flow diagram of an exemplary method for monitoring and predicting conditions in a processing chamber, according to some embodiments of the present disclosure. [Figure 6] FIG. 1 is an exemplary diagram illustrating the training stages of a machine learning system according to some embodiments of the present disclosure. [Figure 7] FIG. 1 illustrates a model training workflow and a model application workflow for chamber monitoring and simulation, according to aspects of the present disclosure. [Figure 8] FIG. 1 is a block diagram of an exemplary computing device that operates in accordance with one or more aspects of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0010] Substrate processing may involve a series of processes to form electrical circuits on semiconductor, e.g., silicon, wafers according to a circuit design. These processes may be performed in a series of chambers. Successful operation of modern semiconductor manufacturing facilities may aim to facilitate a steady flow of wafers moving from one chamber to another while the electrical circuits are formed on the wafers. During the course of performing many substrate procedures, conditions in the processing chambers may change, resulting in processed substrates that do not meet the desired conditions or process results (e.g., critical dimensions, process uniformity, thickness dimensions, etc.).

[0011] Preventive maintenance procedures (e.g., chamber cleaning) are often used as part of a chamber recovery process to restore processing chamber conditions suitable for entering a substrate processing production mode (e.g., processing a large number of substrates). A recovery procedure is often used after a preventive maintenance procedure to prepare the chamber for production mode (e.g., "warming up" the chamber). A common recovery procedure used in the past is process chamber seasoning. Chamber seasoning is a procedure that involves processing a series of substrates (e.g., blank silicon wafers) to restore (e.g., coat the chamber walls) chamber conditions suitable for production substrate processing (e.g., substrates processed in the chamber have process results that meet desired threshold criteria). After chamber seasoning, the chamber may operate in production mode for a period of time until the next preventive maintenance and additional chamber seasoning is required, or until it is recommended to restore the processing chamber conditions for other reasons.

[0012] As previously mentioned, a chamber seasoning procedure involves processing a series of substrates to prepare the chamber for performing a substrate processing procedure, which may include adjusting a set of chamber conditions (e.g., temperature, pressure, vacuum conditions, plasma conditions, etc.) to obtain substrates with target process results (e.g., process results that meet threshold criteria associated with a substrate process recipe).

[0013] Conventionally, after a preventive maintenance procedure, a series of substrates are processed in a processing chamber. After processing of this series of substrates, a test sample is processed and evaluated. The test sample may include substrates processed in the processing chamber and evaluated (e.g., process results are measured) to determine whether the test sample contains process results that meet a threshold condition. If the condition is not met, additional seasoning substrates may be processed, or another test sample may be processed to determine whether the chamber has sufficiently recovered. This process may be repeated until the chamber condition is met or a threshold amount of seasoning wafers has been used. Conventional methods of determining chamber conditions, such as those described above, may require an unnecessary amount of seasoning wafers when using a first amount of seasoning wafers (e.g., "over-seasoning" the chamber), when in fact fewer seasoning wafers are required to restore the chamber to an operable state for performing substrate production. Furthermore, conventional methods may require a delay when returning the chamber to production mode, waiting for the measurement and evaluation of the test samples necessary to determine whether the chamber has sufficiently recovered. The tool or chamber is down (e.g., not operating in production mode), and the time it takes for the recovery procedure is lost productivity. Additionally, conventional methods do not provide in situ chamber condition monitoring, relying on test samples and waiting for evaluation of the test samples during the seasoning process.

[0014] Aspects and implementations of the present disclosure address these and other shortcomings of existing technologies by providing methods and systems of various embodiments that can estimate chamber conditions without relying on test sample inspection results and can provide quantitative information regarding chamber health and recovery after preventative maintenance procedures, enabling simulation of chamber conditions with artificial parameter adjustments. In some embodiments, machine learning models may be utilized to determine chamber conditions and / or status during and / or after seasoning procedures. In some embodiments, the present disclosure enables health checks and preventative measures during chamber seasoning sequences and early detection of abnormal chamber conditions prior to sample testing. One or more of these aspects and / or implementations may result in reduced downtime of processing tools (e.g., processing chambers), enabling greater productivity of manufacturing equipment.

[0015] Exemplary embodiments implement a method and system for post-preventive maintenance monitoring and simulation. In some embodiments, a method implemented by a processing device may include receiving sensor data indicating a state of an environment in a processing chamber processing a series of substrates according to a chamber seasoning procedure. The series of substrates is processed after a preventive maintenance procedure performed on the processing chamber. The processing device may determine a first set of values ​​based on the sensor data. Each value in the first set may be associated with an individual substrate in the series of substrates. The first set of values ​​may indicate a likelihood that the associated substrate will include a first process result that satisfies a threshold condition of the substrate processing procedure. The processing device may predict a first test result based on the first set of values. The first test result may indicate a likelihood that the processed first substrate, after processing the series of substrates, will include a second process result that satisfies the threshold condition. The processing device may prepare the first test result for presentation on a graphical user interface (GUI). The processing device may alter operation of the processing chamber based on the first test result.

[0016] In an exemplary embodiment, a method for training a machine learning model to predict a condition of a processing chamber after a preventive maintenance procedure is performed. The method includes generating training data for the machine learning model. Generating the training data may include identifying a first training input having first sensor data. The first sensor data may be indicative of a first state of the first processing chamber after performing the preventive maintenance procedure. Generating the training data may further include identifying a first target output for the first training input. The first target output may be indicative of whether a substrate processed according to a first substrate processing procedure by the first processing chamber in the first state will result in the first substrate having a first process result that satisfies a first threshold condition associated with the first substrate processing procedure. The method may further include providing training data to train the machine learning model with respect to (i) a set of training inputs including the first training input and (ii) a set of target outputs including the first target output. The trained machine learning model may receive a new input having new sensor data indicative of a new state of the new processing chamber. The new output may indicate whether a new substrate processed according to the first substrate processing procedure by the new processing chamber in the new state results in the new substrate having new process results that satisfy the threshold condition.

[0017] In some embodiments, the methods described above may be stored on a non-transitory machine-readable storage medium including instructions that, when executed by a processing device, cause the processing device to perform operations that may include one or more of the operations described in the methods above.

[0018] 1 is a block diagram illustrating an example system architecture 100 in which embodiments of the present disclosure may function. As shown in FIG. 1, system architecture 100 includes a manufacturing system 102, a metrology system 110, a client device 150, a data store 140, a server 120, and a machine learning system 170. The machine learning system 170 may be part of the server 120. In some embodiments, one or more components of the machine learning system 170 may be fully or partially integrated into the client device 150. The manufacturing system 102, the metrology system 110, the client device 150, the data store 140, the server 120, and the machine learning system 170 may each be hosted on one or more computing devices, including a server computer, a desktop computer, a laptop computer, a tablet computer, a notebook computer, a personal digital assistant (PDA), a mobile communication device, a mobile phone, a handheld computer, or a similar computing device.

[0019] The manufacturing system 102, the metrology system 110, the client device 150, the data store 140, the server 120, and the machine learning system 170 may be coupled to one another via a network 160 (e.g., to perform the methodologies described herein). In some embodiments, the network 160 is a private network that provides each element of the system architecture 100 with access to one another and to other privately available computing devices. The network 160 may include one or more wide area networks (WANs), local area networks (LANs), wired networks (e.g., Ethernet networks), wireless networks (e.g., 802.11 networks or Wi-Fi networks), cellular networks (e.g., Long Term Evolution (LTE) networks), routers, hubs, switches, server computers, and / or any combination thereof. Alternatively, or in addition, any of these elements of the system architecture 100 may be integrated or otherwise coupled together without the use of the network 160.

[0020] Client device 150 may be or include any personal computer (PC), laptop, mobile phone, tablet computer, netbook computer, network-connected television ("smart TV"), network-connected media player (e.g., Blu-ray player), set-top box, over-the-top (OTT) streaming device, operator box, etc. Client device 150 may include a browser 152, applications 154, and / or other tools described and executed by other systems in system architecture 100. In some embodiments, the client device 150 may be able to access the manufacturing system 102, the metrology system 110, the data store 140, the server 120, and / or the machine learning system 170, and may be able to communicate (e.g., send and / or receive) sensor data, processed data, data classification, chamber recovery data (e.g., recovery profiles), process result data instructions, and / or inputs and outputs of various process tools (e.g., the metrology tool 114, the data preparation tool 116, the classifier tool 124, the chamber recovery tool 126, and / or chamber recovery components) at various stages of processing of the system architecture 100, as described herein.

[0021] 1 , the manufacturing system 102 includes process tools 104, process procedures 106, and a process controller 108. The process controller 108 may coordinate the operation of the process tools 104 to perform one or more process procedures 106. For example, the various process tools may include specialized chambers such as etch chambers, deposition chambers (including chambers for atomic layer deposition, chemical vapor deposition, plasma-enhanced atomic layer deposition, or plasma-enhanced chemical vapor deposition), anneal chambers, and / or other similar chambers. In another example, a machine may include a sample transport system (e.g., a selective compliance assembly robot arm (SCARA) robot, a transfer chamber, a front opening pod (FOUP), a side storage pod (SSP), and / or other similar) for transporting samples between machines and process steps.

[0022] The process steps 106, which may also be referred to as process recipes or process steps, may include various specifications for performing operations with the process tool 104. For example, the process steps 106 may include process specifications such as the startup duration of the process operation, the process tool to use for that operation, machine (e.g., chamber) temperatures, flow rates, pressures, etc., deposition sequence, and the like. In another example, a process step may include transfer instructions to transport a sample to additional process steps or to transport a sample for measurement by the metrology system 110. One or more process steps 106 include preventive maintenance procedures and seasoning procedures.

[0023] The process controller 108 may include devices designed to manage and coordinate the operation of the process tool 104. In some embodiments, the process controller 108 is associated with a process recipe, or set of process steps 106 instructions, that when applied as designed, results in a desired process result for the substrate process. For example, the process recipe may be associated with processing a substrate to produce a target process result (e.g., critical dimension, thickness, uniformity criteria, etc.). The process controller 108 may coordinate the execution of preventive maintenance procedures (e.g., chamber cleaning) and chamber recovery procedures (e.g., chamber seasoning) to return the chamber to a production state.

[0024] As shown in FIG. 1 , the metrology system 110 includes a metrology tool 114 and a data preparation tool 116. The metrology tool 114 can include various sensors for measuring process results (e.g., critical dimensions, thickness, uniformity, etc.) within the manufacturing system 102. For example, critical dimensions can be measured using wafers processed within one or more processing chambers. The metrology tool 114 can also include devices for measuring process results of substrates processed using the manufacturing system. For example, process results such as critical dimensions, thickness measurements (e.g., film layers from etching, deposition, etc.) of substrates processed according to process recipes and / or operations executed by the process controller 108 can be evaluated. These measurements can also be used to measure chamber conditions throughout a recovery process, such as during a seasoning procedure that involves processing a series of wafers (e.g., blank wafers).

[0025] The data preparation tool 116 may include process methodologies for extracting features and / or generating synthetic / engineered data related to the data measured by the metrology tool 114. In some embodiments, the data preparation tool 116 may identify correlations, patterns, and / or anomalies in the metrology or process execution data. For example, the data preparation tool 116 may perform feature extraction, in which the data preparation tool 116 uses a combination of measurement data to determine whether a decision criterion is met. For example, the data preparation tool 116 may analyze multiple data points of relevant parameters (e.g., thickness, critical dimensions, defects, plasma conditions, etc.) to determine whether a rapid change occurred during an iteration of a seasoning procedure in a processing chamber. In some embodiments, the data preparation tool 116 performs normalization across various sensor data associated with various process chamber conditions. Normalization may include processing the incoming sensor data so that it appears similar across the various chambers and sensors used to acquire the data.

[0026] In some embodiments, the data preparation tool 116 may perform one or more of a process control analysis, a univariate limit violation analysis, or a multivariate limit violation analysis on the metrology data (e.g., acquired by the metrology tool 114). For example, the data preparation tool 116 may perform statistical process control (SPC) by using a statistical-based methodology to monitor and control the process controller 108. For example, SPC may increase the efficiency and accuracy of preventive maintenance and / or wafer seasoning processes (e.g., by identifying data points that fall within and / or outside of control limits).

[0027] In some embodiments, the processing chamber can be measured throughout the seasoning procedure. In some embodiments, an increasing amount of metrology data is acquired during a given seasoning procedure. For example, additional sensors can be activated and / or currently activated sensors may acquire additional data while or immediately after processing a wafer. In some embodiments, the process controller 108 may trigger measurements by the metrology tool 114 based on operations performed by the process tool 104. For example, the process controller 108 can trigger the activation of one or more process results (e.g., of the metrology tool 114) in response to one or more test wafers being processed during a transition period when the processing chamber has completed a seasoning procedure and is waiting for production wafers to be processed if the process results meet the criteria in the process procedure 106.

[0028] In some embodiments, the extracted features, generated synthetic / engineered data, and statistical analysis may be used in connection with the machine learning system 170 (e.g., to train, validate, and / or test the machine learning model 190). Additionally and / or alternatively, the data preparation tool 116 may output data to the server 120 for use by either the classifier tool 124 and / or the chamber recovery tool 126.

[0029] The data store 140 may be a memory (e.g., random access memory), a drive (e.g., a hard drive, a flash drive), a database system, or another type of component or device capable of storing data. The data store 140 may store one or more pieces of historical data 142, including old sensor data 144 and / or old process result data 146 of previous chamber conditions, as well as process results for substrates processed at related chamber conditions. In some embodiments, the historical data 142 may be used to train, validate, and / or test a machine learning model 190 of the machine learning system 170 (see, e.g., the exemplary methodology of FIG. 6 ).

[0030] The server 120 may include one or more computing devices such as a rack-mounted server, a router computer, a server computer, a personal computer, a mainframe computer, a laptop computer, a tablet computer, a desktop computer, etc. The server 120 may include a classifier tool 124, a chamber recovery tool 126, and a data simulation tool 128.

[0031] The classifier tool 124 receives metrology data (e.g., chamber sensor data) from the process tool 104 throughout the chamber seasoning procedure to determine the status of the chamber. In some embodiments, the classifier tool 124 receives raw sensor data from the process tool 104, while in other embodiments, the raw sensor data is combined with engineered synthetic data from the data preparation tool 116. The classifier tool 124 may process the sensor data to determine a set of values ​​indicating the likelihood that a substrate processed under the associated chamber conditions will result in a substrate having process results that meet the threshold criteria corresponding to the process procedure 106. The classifier tool 124 may classify the state or condition of the processing chamber based on the received sensor data and a machine learning model trained in the machine learning system 170. For example, the state of the processing chamber during the seasoning procedure may be processed to determine whether the chamber meets the conditions for transitioning from a chamber recovery mode to a chamber production mode. In some embodiments, the classifier tool 124 includes a machine learning model that uses process data (e.g., sensor data (e.g., from the process tool 104), synthetic and / or engineered data (e.g., from the process tool 104), and general process parameter values ​​corresponding to the process procedure 106) to determine a set of values ​​that indicate the likelihood that the chamber will meet threshold conditions throughout the progression of the seasoning procedure. As discussed below, the machine learning model may include a bootstrap aggregation model, a random forest tree decision tree model, or a partial least squares regression (PLS) model, among others. The machine learning model may also include ensemble modeling including multiple models that utilize a higher-order confidence model to ultimately classify the received data.

[0032] The chamber recovery tool 126 may receive data from the classifier tool 124 and generate a chamber recovery profile related to the recovery of the processing chamber associated with preventive maintenance and / or seasoning procedures. The chamber recovery tool 126 may determine a preventative maintenance index indicating a change in the likelihood that the chamber will meet the conditions for moving from a recovery mode of operation to a production mode of operation. For example, the chamber recovery tool may identify when chamber conditions meet a threshold condition such that a substrate processed under the chamber conditions will result in a substrate having process results that meet the threshold condition associated with one or more process steps 106. In some embodiments, the chamber recovery tool 126 may use a machine learning model that receives input data from the classifier tool 124 and outputs a preventative maintenance index value related to the likelihood that the chamber will meet the threshold condition for moving to a production mode of operation.

[0033] The data simulation tool 128, in conjunction with the classifier tool 124 and the chamber recovery tool 126, identifies changes in chamber status (e.g., ready to move to production mode, requires more seasoning, requires additional preventative maintenance, etc.) based on artificially adjusting components of sensor data acquired using the manufacturing system 102. The data simulation tool 128 may identify one or more sensor data parameters or combinations of parameters as having a greater impact on the chamber recovery process. The data simulation tool may utilize the classifier tool 124 and the chamber recovery tool 126 to identify the status of the chamber under simulated conditions. The data simulation tool 128 may identify one or more process priorities associated with the chamber recovery procedure. For example, one or more simulation models may determine one or more weights associated with various process parameters (e.g., temperature, pressure, vacuum conditions, etc.).

[0034] As described above, some embodiments of the classifier tool 124, chamber recovery tool 126, and / or data simulation tool 128 may use machine learning models to implement their described methodologies. Those associated machine learning models may have been generated (e.g., trained, validated, and / or tested) using the machine learning system 170. The following exemplary description of the machine learning system 170 is described in the context of using the machine learning system 170 to generate the machine learning model 190 associated with the classifier tool 124. However, it should be noted that this description is purely for illustrative purposes. Similar processing hierarchies and methodologies may be used in generating and executing the machine learning models associated with the classifier tool 124, chamber recovery tool 126, and / or data simulation tool 128, individually and / or in combination with one another, as further discussed in connection with other embodiments.

[0035] The machine learning system 170 may include one or more computing devices, such as a rack-mounted server, a router computer, a server computer, a personal computer, a mainframe computer, a laptop computer, a tablet computer, a desktop computer, etc. The machine learning system 170 may include a chamber status component 194. In some embodiments, the chamber status component 194 may use the historical data 142 to determine whether the status of a processing chamber meets the conditions for transitioning to a production operation mode. In some embodiments, the chamber status component 194 may use a trained machine learning model 190 to determine the status of the chamber. The trained machine learning model 190 may use the historical data to determine the chamber status.

[0036] In some embodiments, trained machine learning models 190 are stored on server machine 192 and / or server 120. Server machine 180 and / or server machine 192 may store one or more recently trained machine learning models (e.g., to perform model execution management). For example, one or more models trained within a period of time (e.g., within the last six months) may be stored on server machine 192, and one or more final models for predicting chamber conditions may be stored on server 120 (e.g., operable with chamber recovery tool 126).

[0037] In some embodiments, machine learning system 170 further includes server machine 172 and server machine 180. Server machines 172 and 180 may be one or more computing devices (e.g., rack-mounted servers, router computers, server computers, personal computers, mainframe computers, laptop computers, tablet computers, desktop computers, etc.), data stores (e.g., hard disks, memory, databases), networks, software components, or hardware components.

[0038] The server machine 172 may include a dataset generator 174 that can generate datasets (e.g., a set of data inputs and a set of target outputs) for training, validating, or testing a machine learning model. The dataset generator 174 may divide the historical data 142 into a training set (e.g., 60 percent of the historical data or any other portion of the historical data), a validation set (e.g., 20 percent of the historical data or some other portion of the historical data), and a test set (e.g., 20 percent of the historical data). In some embodiments, the chamber status component 194 generates multiple sets of training data. For example, one or several sets of training data may include each of these datasets (e.g., a training set, a validation set, and a test set).

[0039] The server machine 180 includes a training engine 182, a validation engine 184, and a testing engine 186. The training engine 182 may be capable of training a machine learning model 190 using one or more of old sensor data 144 and old process result data 146 from the historical data 142 (of the data store 140). In some embodiments, the machine learning model 190 may be trained using the output of one or more of the data preparation tool 116, the classifier tool 124, the chamber recovery tool 126, and / or the data simulation tool 128. For example, the machine learning model 190 may be a hybrid machine learning model (e.g., using the data preparation tool 116) that uses sensor data and / or mechanical features, such as feature extraction, mechanistic modeling, and / or statistical modeling. The training engine 182 may generate multiple trained machine learning models 190, each corresponding to a different set of features from a respective training set.

[0040] The validation engine 184 may determine the accuracy of each of the trained machine learning models 190 based on the corresponding set of features in the respective training set. The validation engine 184 may discard trained machine learning models 190 with accuracies that do not meet a threshold accuracy. The testing engine 186 may determine the trained machine learning model 190 with the highest accuracy of all the trained machine learning models based on the testing set (and optionally the validation set).

[0041] In some embodiments, training data is provided to train the machine learning model 190 such that the trained machine learning model can receive new inputs having new sensor data indicative of the new state of the new processing chamber. The new outputs may indicate whether a new substrate processed according to the first substrate processing procedure by the new processing chamber in the new state will result in the new substrate having new process results that satisfy the threshold condition.

[0042] The machine learning model 190 may refer to a model generated by the training engine 182 using a training set including data inputs and corresponding target outputs (historical results of the process chamber under parameters associated with the target inputs). Patterns in the data set that map the data inputs to the target outputs (e.g., identifying connections between portions of sensor data and resulting chamber statuses) can be found, and mappings that capture these patterns are provided to the machine learning model 190. The machine learning model 190 may use one or more of logistic regression, parsing, decision trees, or support vector machines (SVMs). This machine learning may consist of a single level of linear or nonlinear operations (e.g., SVMs) and / or may be a neural network.

[0043] The chamber status component 194 may provide current data (e.g., current sensor data related to the state of the processing chamber during a seasoning procedure) as input to the trained machine learning model 190, which may run on this input to obtain one or more outputs including a set of values ​​indicating the likelihood that a threshold condition will be met at various points in the seasoning procedure. For example, to determine the optimal amount of substrates to process during the seasoning procedure to prepare the chamber for production use (e.g., without wasting unnecessary seasoned wafers). The chamber status component 194 may be able to identify confidence data from this output indicating the confidence that the status of the processing chamber will meet the threshold condition for moving into production mode at various points in the seasoning procedure. In one non-limiting example, this confidence is a real number between 0 and 1, inclusive, where 0 indicates no confidence in one or more chamber statuses and 1 represents absolute confidence in the chamber status.

[0044] For purposes of illustration and not limitation, aspects of the present disclosure describe training a machine learning model and using the trained machine learning model using information about historical data 142. In other embodiments, heuristic or rule-based models are used to determine chamber status.

[0045] In some embodiments, the functionality of client device 150, server 120, data store 140, and machine learning system 170 may be provided by fewer machines than shown in Figure 1. For example, in some embodiments, server machines 172 and 180 may be combined into a single machine, and in other embodiments, server machines 172, 180, and 192 may be combined into a single machine. In some embodiments, machine learning system 170 may be provided in whole or in part by server 120.

[0046] In general, functions described in one embodiment as being performed by client device 150, data store 140, metrology system 110, manufacturing system 102, and machine learning system 170 may in other embodiments be performed on server 120, where appropriate. Additionally, functions attributed to a particular component may also be performed by different or multiple components operating together.

[0047] In embodiments, a "user" may be described as a single individual. However, other embodiments of the present disclosure encompass a "user" being an entity controlled by multiple users and / or automated sources. For example, a collection of individual users united as a group of administrators may be considered a "user."

[0048] 2A-B are block diagrams illustrating chamber recovery processes 200A-B in which embodiments of the present disclosure may function. The chamber recovery processes 200A-B may include performing preventive maintenance 202 on one or more processing tools. The preventive maintenance 202 may include any number of processes performed on one or more processing tools (e.g., processing chambers) to maintain processing chamber conditions over time. In some embodiments, the preventive maintenance 202 may use proactive preventive maintenance procedures having one or more of the following objectives, among others: minimizing processing equipment downtime, keeping maintenance costs lower (e.g., maintaining minor repairs rather than preventing larger, less frequent repairs), extending the life of processing equipment, improving production quality (e.g., substrate process results), improving processing efficiency (e.g., allowing the tool to spend more time in production mode and reducing processing equipment downtime), and / or maintaining equipment safety and integrity.

[0049] In some embodiments, preventive maintenance 202 includes cleaning (e.g., wet cleaning maintenance) one or more processing chambers (e.g., removing buildup on chamber walls resulting from processing procedures performed in the processing chambers). Preventive maintenance 202 may restore the condition of one or more processing tools to an unseasoned state (e.g., a clean chamber that has not yet undergone a seasoning process). Replacement and / or cleaning of one or more processing tools may result in the processing tool operating in a further improved clean (e.g., raw) state by performing substrate processing procedures using the processing tools. For example, the processing chamber may be returned to an unseasoned state, e.g., a new or clean state after preventive maintenance 202 has been performed.

[0050] 2, the chamber recovery process 200A-B includes chamber initialization. Chamber initialization 204 may include warming up the chamber or starting up one or more systems associated with the processing chamber. For example, chamber initialization 204 may be performed to prepare the processing chamber for receiving and processing a substrate. Chamber initialization may be associated with one or more process steps (e.g., process step 106 of FIG. 1), which may include one or more process parameters (e.g., temperature, pressure, vacuum conditions, etc.).

[0051] As shown in FIG. 2 , chamber recovery process 200A-B includes process chamber seasoning 206. Chamber seasoning 206 is a procedure for processing a series of substrates (e.g., blank silicon wafers) to restore chamber conditions suitable for production substrate processing. For example, in some embodiments, a thin silicon oxide layer covers the chamber walls after chamber seasoning. Seasoning 206 involves processing an initial quantity of seasoned wafers (e.g., 20 or more seasoned wafers). Chamber recovery process 200A utilizes a test sample 208 to determine whether chamber conditions are met for transitioning to production mode, while chamber recovery process 200B utilizes a preventative maintenance (PM) model 212 to determine whether chamber conditions are met for transitioning to production mode.

[0052] In some embodiments, as shown in FIG. 2A , a sample test 208 is performed and evaluated after seasoning the chamber with a first amount of wafers. The test sample may include processing a substrate through the processing chamber under conditions after seasoning the first amount of substrates. The sample test measures the sample and evaluates various process result criteria. For example, the sample test 208 may be evaluated to determine whether the substrate contains process results that meet threshold criteria, such as critical dimensions, process uniformity thresholds, thickness profile requirements, etc. Processing and evaluating the sample test may include the duration of downtime required to process the substrate and evaluate the processed substrate (e.g., measuring process results such as thickness and uniformity conditions across the surface of the substrate).

[0053] In block 210A, the chamber recovery process 200A may determine whether one or more threshold conditions for the processing chamber have been met based on the sample test 208. The threshold conditions may include criteria related to one or more process results of the sample test 208. As previously noted, the sample test 208 may be evaluated to determine whether the substrate contains process results that meet threshold criteria, such as critical dimensions, process uniformity thresholds, thickness profile requirements, etc., any of which may be used as an evaluation criterion to determine whether the chamber meets the conditions for transitioning to a production mode of operation. The processing logic may determine that the chamber's status does not meet the threshold conditions and may further perform additional seasoning 206 and another sample test 208 to determine whether the chamber is properly seasoned (e.g., whether the threshold conditions are met). The process of seasoning 206, performing sample tests 208, and determining whether the threshold conditions for the process chamber have been met based on the most recent sample test (e.g., block 210) may be repeated until the chamber meets the threshold conditions or a threshold amount of seasoned wafers or seasoning iterations have been performed. After determining that the threshold conditions have been met, the chamber recovery process 200A may proceed to initiating a production mode for the processing chamber.

[0054] The chamber recovery process 200B includes performing preventive maintenance (PM), chamber initialization, and initial seasoning 206 as described above. As shown in FIG. 2B , the chamber recovery process 200B then monitors the chamber health and evaluates the status of the processing chamber using a PM model 212. The PM model 212 may receive sensor data indicating the state of the processing chamber during seasoning 206. In some embodiments, the PM model includes one or more aspects and / or features of the classifier tool 124, the chamber recovery tool 126, and / or the data simulation tool 128. The PM model 212 may output a notification indicating the status of the processing chamber. In some embodiments, the notification may include an indication that the chamber has recovered. For example, the PM model 212 may indicate permission to run a sample test 208 (e.g., to verify the results of the PM model 212).

[0055] At block 210, chamber recovery process 200B receives output from PM model 212 and determines additional actions to take in response to the received output. In some embodiments, this notification may include an indication that the chamber has recovered. For example, PM model 212 may indicate permission to run sample test 208 (e.g., to verify the results of PM model 212). PM model 212 may output a notification indicating that the processing chamber is not recovered and that additional seasoning on the chamber is required before entering production mode. PM model 212 may output a notification indicating that the quality of the preventive maintenance does not meet a threshold condition and that additional preventive maintenance is required. In some embodiments, chamber recovery process 200B may determine that the processing chamber conditions are met and that production mode can be initialized without using a sample test. In some embodiments, chamber recovery process 200B may determine that previous preventive maintenance performed was insufficient and that the processing chamber may need to perform additional preventive maintenance rather than performing multiple iterations of seasoning 206 to meet a threshold condition. For example, instead of directing the processing logic to seasoning 206 when a threshold condition is not met, the processing logic may direct the process back to preventive maintenance 202 instead of performing a seasoning that fails to improve the chamber conditions to meet the threshold condition (e.g., wasting a seasoned wafer).

[0056] In some embodiments, chamber recovery process 200B returns the chamber to production mode using a shorter time (e.g., shorter downtime of the processing chamber). For example, chamber recovery process 200B may not require performing sample testing (e.g., processing and evaluating a test wafer) to arrive at a determination that the processing chamber meets one or more threshold conditions. In another example, chamber recovery process 200B may determine that additional preventive maintenance, rather than performing more seasoning of the processing chamber, may be preferable to further restore the chamber's status to meet one or more threshold conditions.

[0057] As will be discussed further in other embodiments, the chamber recovery process 200A may be utilized to generate training data for the PM model 212. Processing chamber sensor data may be tracked throughout the seasoning process 206, and data associated with the sample tests 208 may be utilized to validate the output from the PM model 212. For example, the PM model 212 may determine the status of the processing chamber. Evaluations of sample tests processed under the same chamber conditions may be compared to predictions by the PM model 212 to validate and further refine the PM model 212.

[0058] 3 is a block diagram illustrating a chamber monitoring and maintenance system 300 in which embodiments of the present disclosure may function. The chamber monitoring and maintenance system 300 includes a substrate processing system 302 (e.g., the manufacturing system 102 of FIG. 1). The substrate processing system 302 may include a process tool 304 and chamber data 306 (e.g., acquired using the process tool 304). The process tool 304 may include chamber sensors (e.g., for measuring radio frequency (RF), temperature, gases, etc.) and additional sensors (e.g., optical emission spectroscopy (OES) sensors). The chamber data 306 may include data indicative of a condition of a processing chamber. For example, the process tool 304 may measure and monitor parameters such as RF, temperature, gases, and spectroscopic data.

[0059] In some embodiments, the chamber data 306 is processed by a data preparation tool 308. The data preparation tool 308 can receive chamber data (e.g., raw sensor data) and generate synthetic data associated with various combinations, correlations, and / or artifacts of parameters. The data preparation tool 308 can reduce the dimensionality of the raw sensor data into groups or features. For example, the data preparation tool 308 can generate features including parameter values ​​and / or rates of change of collections of parameter values. In some embodiments, the data preparation tool 308 performs any of partial least squares analysis, principal component analysis, multi-factor dimensionality reduction, nonlinear dimensionality reduction, and / or any combination thereof. In some embodiments, the data preparation tool 308 is designed to detect edges in the chamber data. For example, the data preparation tool 308 includes methodologies aimed at identifying data points that change rapidly and / or have discontinuities, such as the slope of one or more measured chamber parameters of a processing chamber.

[0060] In some embodiments, the data preparation tool 308 performs normalization across various sensor data associated with various process chamber conditions. Normalization may include processing the incoming sensor data so that it looks the same across various chambers and / or sensors used to acquire the data. In some embodiments, the data preparation tool 308 may include aspects and / or features of the data preparation tool 116 of FIG. 1 .

[0061] In some embodiments, the data preparation tool 308 is used to generate input for the classifier model 310. As will be further discussed in other embodiments, the classifier model 310 receives input data (e.g., raw sensor data, synthetic data, output of other models, etc.) for determining the status of the processing chamber (e.g., classifying the chamber into one of a set of categories). In some embodiments, the classifier model receives sensor and metrology data 307 indicating the condition of the processing chamber while processing a series of substrates (e.g., seasoning wafers). The classifier model may output a set of values ​​indicating the likelihood that a substrate processed after a related substrate in the series of substrates will result in a substrate having a process result that meets a threshold condition (e.g., associated with a substrate processing recipe). For example, the classifier model 310 may output a set of probability values, each associated with one or more substrates in the series of substrates (e.g., seasoning wafers). The probability values ​​may indicate the likelihood that the related substrate or a substrate processed after the related substrate, when processed, will result in a substrate having a process result that meets one or more threshold conditions associated with a substrate processing recipe (e.g., a substrate process recipe). As will be discussed further in other embodiments, in some embodiments, the classifier model 310 may include a machine learning model (e.g., trained using FIG. 6 and / or implemented using FIG. 5).

[0062] As shown in FIG. 3 , the chamber monitoring and maintenance system 300 may include a chamber recovery tool 312. The chamber recovery tool 312 may receive output from the classifier model 310. The chamber recovery tool 312 may approximate a chamber recovery profile (e.g., the chamber recovery profile of FIG. 4 ) by determining a set of PM indicators associated with the output of the classifier model 310. In some embodiments, the set of PM indicators is based on an analysis of current data and previous data (e.g., the historical data 142 of FIG. 1 ). The chamber recovery tool 312 may determine a threshold PM indicator that identifies whether chamber conditions are sufficient to pass a sample test, whether more seasoning is needed, or whether additional preventive maintenance is required. PM indicator results for a set of seasoned wafers during a seasoning run can be obtained, and these PM indicator results can enable in situ chamber monitoring by the chamber monitoring and maintenance system 300.

[0063] In some embodiments, the chamber recovery tool 312 determines one or more PM indicators associated with the status of the processing chamber. The chamber recovery tool 312 may perform statistically optimized calculations to determine the mapping between threshold conditions and seasoning wafer amounts and PM indicators. An exemplary methodology for calculating the PM indicators may include the following equation: PM index nth wafer = lnCUMSUM(Y est )*AVERAGE(Y est ) In the above equation, Y est is the classifier model result associated with each seasoning wafer in the series of seasoning wafers during the seasoning run.

[0064] In some embodiments, the chamber recovery tool 312 outputs instructions to the process controller 314 to modify one or more operations associated with the substrate processing system 302. For example, the chamber recovery tool 312 may indicate the end of a seasoning process, and the process controller may modify operational parameters of the substrate processing system to enter a production operating mode. In another example, the chamber recovery tool 312 may indicate that a processing chamber is nearly recovered and that additional seasoning is required to improve the processing chamber conditions sufficiently to enter a production operating mode. In another example, the chamber recovery tool 312 may indicate to the process controller that preventive maintenance quality is insufficient (e.g., does not meet a threshold condition) and instruct the process controller 314 to perform additional preventive maintenance on the substrate processing system 302.

[0065] In some embodiments, the chamber recovery tool 312 approximates the expected chamber conditions during a seasoning run. The chamber recovery tool 312 blends the probabilities received from the classifier model 310 to estimate the conditions of a chamber at a given time. These conditions may be expressed as PM indices. As further discussed in connection with FIG. 4, these PM indices may be defined by logarithmic functions. The chamber recovery tool 312 may determine the amount of substrates to process in a seasoning procedure (e.g., to meet threshold conditions for the processing chamber). For example, these PM indices can be used to determine optimal seasoning timing (e.g., the optimal number of wafers to use in a seasoning process).

[0066] 3, the process controller 314 receives input from the chamber recovery tool 312 and provides instructions to the substrate processing system 302 in performing additional operations. For example, the process controller 314 may instruct the substrate processing system 302 to proceed with additional seasoning of one or more processing chambers. In another example, the process controller 314 may instruct the substrate processing system to finish seasoning one or more processing chambers. In another example, the process controller 314 may instruct the substrate processing system to perform preventive maintenance on one or more processing chambers.

[0067] As shown in FIG. 3 , the chamber recovery tool 312 may output instructions for display on a graphical user interface (GUI). The graphical user interface may output a chamber status prediction on the graphical user interface. In some embodiments, the chamber recovery tool 312 outputs instructions for display on the GUI 316 related to one or more operations to perform on the substrate processing system. For example, the GUI 316 may display a notice to proceed with additional seasoning of one or more processing chambers. In another example, the GUI 316 may display instructions to terminate seasoning of one or more processing chambers. In another example, the GUI 316 may display a notice to perform preventive maintenance on one or more processing chambers.

[0068] FIG. 4 illustrates a graph 400 showing a chamber recovery profile according to some embodiments of the present disclosure. Graph 400 shows a chamber recovery profile (e.g., generated by chamber recovery tool 312). The recovery profile may include a first data series 406, each indicating a PM indicator associated with an individual seasoning wafer 404. The recovery profile may indicate a threshold 408 PM indicator at which processing chamber conditions indicate satisfaction of the chamber recovery process. For example, an intersection between threshold 408 and first data series 406 may indicate an optimal number of seasoning wafers to use in a seasoning run such that the processing chamber is ready for production mode. Specifically, the processing chamber may process a subsequent substrate, which is likely to have process results that meet the threshold conditions of the substrate processing procedure. In some embodiments, first data series 402 may cross threshold 408, indicating the chamber is ready to run in production mode. In other embodiments, other recovery profile data series may not cross the threshold, indicating that the chamber is not ready for production mode and additional preventative maintenance and / or chamber seasoning may be required.

[0069] 5-7 illustrate flow diagrams illustrating exemplary methods 500-700 for training and / or using machine learning models associated with chamber data, according to some embodiments of the present disclosure. For ease of explanation, methods 500-700 are shown and described as a series of operations. However, operations according to the present disclosure may be performed in various orders and / or simultaneously, and may be performed with other operations not shown and described herein. Furthermore, not all illustrated operations may be performed to implement methods 500-700 according to the disclosed subject matter. In addition, those skilled in the art will understand and appreciate that methods 500-700 may also be represented as a series of interrelated states, via state diagrams or events.

[0070] 5 illustrates a flow diagram of an exemplary method 500 for monitoring and predicting processing chamber conditions according to some embodiments of the present disclosure. Method 500 is performed by processing logic, which may include hardware (e.g., circuitry, dedicated logic, etc.), software (e.g., software running on a general-purpose computer system or a dedicated machine, etc.), or any combination thereof. In one embodiment, the method is performed using server 120 and trained machine learning model 190 of FIG. 1, while in other embodiments, one or more blocks of FIG. 5 may be performed by one or more other machines not shown.

[0071] The method 500 may include receiving sensor data (e.g., associated with a processing chamber performing a seasoning procedure) and processing the sensor data using the trained machine learning model 190. The trained model may be configured to generate one or more outputs based on the sensor data that indicate (i) the status of the processing chamber and (ii) a confidence that the status represents the actual status of the processing chamber. The status of the processing chamber may indicate whether the processing chamber is recovered and ready for a production mode of operation, whether additional seasoning on the processing chamber is required, and / or whether additional preventative maintenance on the processing chamber is required.

[0072] At block 502, sensor data indicative of a processing chamber environmental condition is received. The processing chamber condition is measured during a seasoning procedure (e.g., while processing a series of substrates). The sensor data may be raw data or may have been processed using one or more of feature extraction, mechanistic modeling, and / or statistical modeling to prepare the sensor for input to a machine learning model. The sensor data may be indicative of one or more parameters of the processing chamber (e.g., temperature, pressure, vacuum conditions, spectroscopic data, etc.).

[0073] In some embodiments, the sensor data further includes synthetic data or data engineered from the raw sensor data. For example, as described in previous embodiments, various engineering tools can perform feature extraction and / or generate artificial and / or hypothetical parameter combinations. A feature extractor (e.g., data preparation tool 116 in FIG. 1 ) can generate various features by performing various analyses on the raw sensor data, such as process control analysis, univariate limit violation analysis, and / or multivariate limit violation analysis. In some embodiments, the sensor data is normalized across multiple processing chambers and / or process recipes to generate comparable data sets with a common base.

[0074] At block 504, the sensor data is provided as input to a trained machine learning model, and one or more outputs are obtained from the trained machine learning model at block 506. At block 508, a first set of values ​​is determined based on the outputs from the machine learning model, the first set of values ​​indicating the likelihood that an associated substrate contains a first process result that satisfies a threshold condition.

[0075] In some embodiments, the processing logic determines a first amount based on a first set of values. The first value may indicate a threshold number of substrates that, when processed after a preventive maintenance procedure, will result in the processing chamber processing subsequent substrates. The subsequent substrates include second process results that meet threshold conditions (e.g., uniformity requirements, critical dimension criteria, film thickness criteria, etc.) associated with the substrate processing procedure. The threshold number of substrates and the subsequent substrates are processed according to the substrate processing procedure. The first amount represents how many seasoning wafers are required to restore the processing chamber to a production operating mode.

[0076] At block 510, a first test result is predicted using the first set of values. The first test result indicates the likelihood that a processed first substrate will contain a second process result that meets the threshold condition after processing a series of substrates. The first test result may be predicted based on a comparison between a first amount and a second amount (e.g., a difference between the first amount and the second amount). For example, a threshold number of substrates may be compared to a current number of substrates to determine whether a sufficient number of seasoning wafers have been processed to restore the condition of the processing chamber after performing preventive maintenance. In some embodiments, the difference between these identified amounts may be displayed on a GUI (e.g., to indicate over-seasoning or under-seasoning of one or more processing chambers).

[0077] In some embodiments, processing logic determines a recovery profile corresponding to the processing chamber. The recovery profile may include a distribution of indices indicating a change in likelihood that a first processed substrate will include a second process result that satisfies a threshold condition after processing a series of substrates. A first test result may be predicted using the recovery profile. The recovery profile may include a cumulative distribution function associated with a set of values ​​received at one or more outputs of the machine learning model.

[0078] At block 512, the first test results are optionally prepared for presentation on a graphical user interface (GUI). For example, the first test results may include a notification indicating the need to perform additional seasoning on the processing chamber, terminate seasoning, and / or perform preventive maintenance on the processing chamber. At block 514, operation of the processing chamber can be modified based on the first test results. For example, the processing logic may send instructions to one or more process controllers to modify one or more operations of the processing equipment (e.g., continue a seasoning process, terminate a seasoning process, initiate a preventive maintenance procedure, etc.).

[0079] 6 is an exemplary diagram illustrating a training phase of a machine learning system according to some implementations of the present disclosure. Machine learning system 170 may use method 600 to perform at least one of training, validating, or testing a machine learning model according to embodiments of the present disclosure. In some embodiments, one or more operations of method 600 may be performed by dataset generator 174 of server machine 172 described with respect to FIG. 1. Note that aspects of FIG. 6 may be illustrated using components described with respect to FIG. 1.

[0080] At block 602, processing logic initializes a training set. At block 604, processing logic identifies a first data input (e.g., a first training input, a first validation input) including a first sensor (as described with respect to FIGS. 1-5). This first data input may include data indicative of the state of the environment of the processing chamber during the seasoning procedure. In some embodiments, the first data input may include any chamber environment parameter (e.g., temperature, pressure, vacuum conditions, plasma conditions, spectroscopic conditions, etc.). A first target input may be identified from old sensor data 144 of historical data 142 (e.g., stored in data store 140).

[0081] At block 606, processing logic identifies a first target output for one or more of these data inputs (e.g., a first data input). The first target output indicates whether a first substrate processed according to a first substrate processing procedure by a first processing chamber in a first state (e.g., the first data input) results in the first substrate having a first process result that satisfies a first threshold condition associated with the first substrate processing procedure. The first target input may be identified from sensor data 144 of historical data 142 (in data store 140).

[0082] At block 608, processing logic optionally generates mapping data indicating an input / output mapping. This input / output mapping (or mapping data) may be in terms of a data input (e.g., one or more of the data inputs described herein), a target output for the data input (e.g., one or more of the data inputs described herein), a target output for the data (e.g., where the target output identifies an associated chamber status), and an association between the data input and the target output.

[0083] At block 610, processing logic adds the mapping data generated at block 504 to dataset T.

[0084] At block 612, processing logic branches based on whether dataset T is sufficient for at least one of training, validating, or testing machine learning model 190. If so (the "yes" branch), execution proceeds to block 614; if not (the "no" branch), execution returns to block 604. It should be noted that in some embodiments, whether dataset T is sufficient may be determined simply based on the number of input / output mappings in the dataset, while in other embodiments, whether dataset T is sufficient may be determined based on one or more other criteria (e.g., measures of diversity, accuracy, etc. of data examples) in addition to or instead of the number of input / output mappings.

[0085] At block 610, processing logic provides dataset T for training, validating, or testing machine learning model 190. In some embodiments, dataset T is a training set and is provided to training engine 182 of server machine 180 to perform training. In some embodiments, dataset T is a validation set and is provided to validation engine 184 of server machine 180 to perform validation. In some embodiments, dataset T is a test set and is provided to test engine 186 of server machine 180 to perform testing. For example, in the case of a neural network, input values ​​(e.g., numerical values ​​associated with data inputs) of a given input / output mapping are input to the neural network, and output values ​​(e.g., numerical values ​​associated with target outputs) of the input / output mapping are stored at output nodes of the neural network. The connection weights of the neural network are then adjusted according to a learning algorithm (e.g., backpropagation), and the procedure is repeated for the remaining input / output mappings of dataset T. After block 614, the machine learning model (e.g., machine learning model 190) may be at least one of trained using training engine 182 of server machine 180, validated using validation engine 184 of server machine 180, or tested using test engine 186 of server machine 180. The trained machine learning model may be implemented by chamber status component 194 (of server machine 192) to identify a chamber status of a chamber (e.g., the likelihood that a substrate processed in that chamber will have process results that meet a threshold condition) based on the received sensor data.

[0086] In some embodiments, the generated training data set is used to train a machine learning model and / or a physics model. The model may be trained to receive chamber sensor data as input. The model may output a chamber status (e.g., the likelihood that a substrate processed in the chamber will have a process result that meets a threshold condition) based on the received sensor data. In embodiments, the model may be agnostic to individual processing chambers, preventive maintenance performed, and / or process recipes used in production mode. Thus, a model may be generated based on training data items generated based on running a seasoning process on a first processing chamber and then used for a second processing chamber without performing transfer learning to tune the model for the second processing chamber.

[0087] In one embodiment, the trained machine learning model is a regression model trained using regression. An example of a regression model is a regression model trained using linear regression or Gaussian regression. A regression model predicts a value of Y given known values ​​of an X variable. The regression model may be trained using regression analysis, which may include interpolation and / or extrapolation. In one embodiment, parameters of the regression model are estimated using least squares regression. Alternatively, Bayesian linear regression, partial least squares regression, percentage regression, least absolute deviation, nonparametric regression, scenario optimization, and / or metric learning may be performed to train the regression model. In one embodiment, the trained machine learning model includes a support vector machine or other type of machine learning model.

[0088] In one embodiment, the trained machine learning model uses ensemble modeling. Ensemble modeling may involve training multiple models (e.g., two or more related but distinct analytical models) and combining the results of each model into a single score or spread (e.g., improving and reducing the accuracy from any one machine learning model). For example, the trained machine learning model may include a bootstrap aggregation (“bagging”) model. Bagging may include an aggregation phase and a bootstrapping phase for each individual model. Bootstrapping involves sampling or generating a dataset (e.g., using method 600) in which each sample (e.g., sensor data associated with running a seasoning procedure on a process chamber) is selected (e.g., randomly) and replaced within a broader set of samples. As previously noted, multiple models may be generated, each based on a different subset of the broader overall set of training data (e.g., historical data 142). The multiple models may output combined (e.g., aggregated) results to form an overall score (e.g., an overall confidence level) associated with the identified output (e.g., predicted chamber status) of the ensemble of models. In another example, the machine learning model may include a random forest model. The machine learning model utilizes multiple decision trees or multiple types of analytical models designed to predict outputs based on different variables and rules. The random forest model may mix decision trees that analyze different sample data, evaluate different factors, or weight common variables differently. The results of these various decision trees may be converted into a combined result (e.g., a result aggregated through a simple average or additional weighting) to predict the chamber status based on the input sensor data.

[0089] In one embodiment, the trained machine learning model is an artificial neural network (also simply referred to as a neural network). The artificial neural network may be, for example, a convolutional neural network (CNN) or a deep neural network. In one embodiment, processing logic performs supervised machine learning to train the neural network.

[0090] Artificial neural networks typically include a feature representation component with a classifier or recurrent layer that maps features to a target output space. For example, a convolutional neural network (CNN) hosts multiple layers of convolutional filters. Pooling may be performed and nonlinearities may be addressed in lower layers, and a multilayer perceptron is typically added above the lower layers to map the upper layer features extracted by the convolutional layers to a decision (e.g., a classification output). This neural network may be a deep network with multiple hidden layers, or a shallow network with no or a small number (e.g., one or two) of hidden layers. Deep learning is a type of machine learning algorithm that performs feature extraction and transformation using a cascade of multiple layers of nonlinear processing units. Each successive layer uses the output from the previous layer as input. Neural networks may learn supervised (e.g., classification) and / or unsupervised (e.g., pattern analysis). Some neural networks (e.g., deep neural networks) include a hierarchy of layers, with different layers learning different levels of representations corresponding to different levels of abstraction. In deep learning, each level learns to transform its input data into a slightly more abstract and complex representation.

[0091] Training a neural network may be accomplished with supervised learning, which involves feeding a training dataset of labeled inputs through the network, observing its outputs, defining an error (by measuring the difference between the output and the label value), and using techniques such as deep gradient descent and backpropagation to tune the network's weights across all its layers and nodes so that the error is minimized. In many applications, repeating this process across many labeled inputs in the training dataset provides a network that can generate correct outputs when presented with inputs that differ from those present in the training dataset.

[0092] To achieve continuous learning and improvement of the trained machine learning model, the trained machine learning model may be periodically or continuously retrained. The model may generate an output based on an input, an action may be performed based on the output, and a result of the action may be measured. In some examples, the result of the action is measured within seconds or minutes, and in some examples, it takes longer to measure the result of the action. For example, one or more additional processes may be performed before the result of the action can be measured. The action and the result of the action may indicate whether the output was the correct output and / or may indicate the difference between what the output should have been and what it was. Thus, the action and the result of the action may be used to determine a target output that can be used as a label for the sensor measurement. After the result of the action is determined, new training data items may be generated using the input (e.g., sensor data), the output (e.g., chamber status) and target result (e.g., target process result of a substrate) of the trained machine learning model, and the measured actual result (e.g., measured process result of a substrate). The new training data items may then be used to further train the trained machine learning model. In some embodiments, this retraining process may be performed on-tool on the manufacturing system (eg, processing chamber).

[0093] 7 illustrates a model training workflow 705 and a model application workflow 717 for chamber monitoring and simulation according to aspects of the present disclosure. In an embodiment, the model training workflow 705 may be executed on a server, which may or may not include a chamber classifier and / or chamber recovery predictor application, which may be provided with the trained model and which may execute the model application workflow 717. The model training workflow 705 and the model application workflow 717 may be performed by processing logic executed by a processor of a computing device (e.g., server 120 of FIG. 1 ). One or more of these workflows 705, 717 may be implemented, for example, by one or more machine learning modules implemented on the processing device and / or other software and / or firmware executing on the processing device.

[0094] The model training workflow 705 is a workflow for training one or more machine learning models (e.g., deep learning models) to perform one or more tasks, such as determining, predicting, or modifying, related to chamber classification and chamber recovery prediction (e.g., determining that a process chamber is properly seasoned and ready to transition to a production operating mode). The model application workflow 717 is a workflow for applying one or more trained machine learning models to perform tasks, such as determining and / or tuning, on chamber data (e.g., raw sensor data, synthetic data indicating the state of the processing chamber). One or more of these machine learning models may receive process result data (e.g., chamber status and / or recovery profile).

[0095] Various machine learning outputs are described herein. Specific numbers and arrangements of machine learning models are described and illustrated. However, it should be understood that the number and types of machine learning models used, as well as the arrangements of such machine learning models, can be varied to achieve the same or similar end results. Thus, the arrangements of machine learning models described and illustrated are merely examples, and the arrangements should not be construed as limiting.

[0096] In an embodiment, one or more machine learning models are trained to perform one or more of the following tasks. Each task may be performed by a separate machine learning model. Alternatively, a single machine learning model may perform each of the tasks or a subset of the tasks. Additionally or alternatively, different machine learning models may be trained to perform different combinations of these tasks. In one example, one or a few machine learning models may be trained, and the trained machine learning (ML) model is a single shared neural network with multiple shared layers and multiple higher-order separate output layers, each of which outputs a different prediction, classification, identification, etc. The tasks that the trained one or more machine learning models may be trained to perform are: Chamber Classifier - As previously discussed, various input data, such as sensor data, pre-processed data, composite data, etc., indicating the state of a processing chamber during a seasoning process (e.g., processing a series of seasoned wafers), may be received and processed by a chamber classifier. The chamber classifier may output various values ​​corresponding to various seasoned wafers, indicating the likelihood that a substrate process under the corresponding chamber conditions will result in a substrate having process results that meet threshold conditions associated with the substrate process (e.g., substrate and / or manufacturing recipe). b. Chamber Recovery Predictor—The chamber recovery predictor may receive data from the chamber classifier and generate a chamber recovery profile related to the recovery of the processing chamber associated with preventive maintenance and / or seasoning procedures. The chamber recovery predictor may determine a preventive maintenance index indicating a change in the likelihood that the chamber will meet the conditions for moving from a recovery mode of operation to a production mode of operation. For example, the chamber recovery predictor may identify when chamber conditions meet a threshold condition such that a substrate processed under the chamber conditions will result in a substrate having process results that meet the threshold conditions associated with one or more process procedures. In some embodiments, the chamber recovery predictor outputs a preventive maintenance index value related to the likelihood that the chamber will meet the threshold conditions for moving to a production mode of operation.

[0097] One type of machine learning model that may be used to perform some or all of the above tasks is an artificial neural network, such as a deep neural network. Artificial neural networks generally include a feature representation component with a classifier or recurrent layer that maps features to a desired output space. For example, a convolutional neural network (CNN) hosts multiple layers of convolutional filters. Pooling may be performed and nonlinearities may be addressed in lower layers, and above the lower layers, multilayer perceptrons are typically added to map upper-layer features extracted by the convolutional layers to decisions (e.g., classification outputs). Deep learning is a type of machine learning algorithm that performs feature extraction and transformation using a cascade of multiple layers of nonlinear processing units. Each successive layer uses the output from the previous layer as input. Deep neural networks may learn supervised (e.g., classification) and / or unsupervised (e.g., pattern analysis) learning. Deep neural networks include a hierarchy of layers, with different layers learning different levels of representation corresponding to different levels of abstraction. In deep learning, each level learns to transform its input data into slightly more abstract and complex representations. For example, in plasma process tuning, the raw input may be chamber data (e.g., raw sensor chamber measurement parameters such as temperature, pressure, and vacuum conditions within the chamber); a second layer may comprise feature data related to parameters associated with the substrate processing procedure or process recipe; and a third layer may include historical thresholds and chamber status under similar conditions. In particular, the deep learning process can independently learn which features to optimally place at which level. The "deep" in "deep learning" refers to the number of layers through which data is transformed. More precisely, deep learning systems have significant credit assignment path (CAP) depth. A CAP is a chain of transformations from input to output. A CAP describes the potentially causal connections between input and output.For feedforward neural networks, the depth of the CAP may be the depth of the network, or the number of hidden layers plus 1. For recurrent neural networks, where signals may propagate through layers more than once, the CAP depth is potentially infinite.

[0098] In one embodiment, one or more machine learning models are recurrent neural networks (RNNs). RNNs are a type of neural network that includes memory to allow the neural network to capture time dependencies. RNNs can learn input-output mappings that depend on both current and past inputs. RNNs address past and future sensor and / or process result measurements and perform predictions based on this continuous sensor information (e.g., sensor data indicating gradually changing conditions in a processing chamber). RNNs may be trained using a training data set to generate a fixed number of outputs (e.g., to determine a set of substrate processing rates and determine chamber conditions as well as production mode requirements and condition thresholds). One type of RNN that may be used is a long short-term memory (LSTM) neural network.

[0099] Training a neural network may be accomplished with supervised learning, which involves feeding a training dataset consisting of labeled inputs through the network, observing its outputs, defining an error (by measuring the difference between the output and the label value), and using techniques such as deep gradient descent and backpropagation to tune the network's weights across all its layers and nodes so that the error is minimized. In many applications, repeating this process across many labeled inputs in the training dataset provides a network that can generate correct outputs when presented with inputs that differ from those present in the training dataset.

[0100] For the model training workflow 705, a training data set including hundreds, thousands, tens of thousands, hundreds of thousands, or even hundreds of thousands of chamber data 710 (e.g., sensor data indicating relevant processing chamber conditions, synthetic data) should be used to form the training data set. In an embodiment, the training data set may further include relevant process result data 712 (e.g., measured parameters of the substrate (e.g., critical dimensions, uniformity requirements, film thickness results, etc.)) to form the training data set, where each data point may include various labels or classifications of one or more types of useful information. Each case may include, for example, data indicating one or more processing chambers undergoing a seasoning process, as well as relevant process results of the substrate evaluated during and / or after the seasoning process. This data may be processed to generate one or more training data sets 736 for training one or more machine learning models. The machine learning models may be trained, for example, to automate a classifier of the chamber (e.g., the chamber meets a condition for operating in production mode) and / or to predict a threshold amount of substrates when the chamber meets a threshold condition for operating in production mode.

[0101] In one embodiment, generating the one or more training data sets 736 includes performing a chamber recovery process 200A by performing preventive maintenance 202 and seasoning 206 and evaluating sample tests 208 to determine whether the chamber meets threshold conditions (e.g., block 210). One or more labels may be used in various iterations of the chamber recovery process 200A to identify the sample tests 208 and measured chamber conditions as meeting threshold conditions for initiating production mode, initiating additional seasoning, and / or initiating additional preventive maintenance corresponding to the processing chamber. The labels used may depend on how a particular machine learning model is trained. In some embodiments, the chamber data, process results, and / or sample test results may be represented as vectors, and the process rates may be represented as one or more matrices, as described in other embodiments.

[0102] To perform the training, processing logic inputs the training dataset 736 to one or more untrained machine learning models. Before inputting the first input to the machine learning models, the machine learning models may be initialized. Processing logic trains the untrained machine learning models based on the training dataset to generate one or more trained machine learning models that perform the various operations described above.

[0103] Training may be performed by inputting one or more of the chamber data 710 and the process result data 712 into the machine learning model one by one.

[0104] The machine learning model processes this input to generate an output. An artificial neural network includes an input layer consisting of values ​​at the data points. The next layer is called the hidden layer, and each node in the hidden layer receives one or more of these input values. Each node includes parameters (e.g., weights) that it applies to the input values. Thus, each node essentially inputs its input values ​​into a multivariate function (e.g., a nonlinear mathematical transform) to generate an output value. The next layer may be another hidden layer or an output layer. In either case, the nodes in the next layer receive the output values ​​from the nodes in the previous layer, and each node applies a weight to those values ​​and then generates its own output value. This may be performed in each layer. The final layer is the output layer, which has one node for each class, prediction, and / or output that the machine learning model can generate.

[0105] Thus, the output may include one or more predictions or inferences. For example, the output predictions or inferences may include one or more changes to the plasma process data (e.g., changes to one or more plasma exposure times). The processing logic may cause the substrate to be processed using the updated recipe (e.g., with the identified changes) and receive an updated thickness profile. The processing logic may compare the updated thickness profile to a target thickness profile to determine whether a threshold criterion is met (e.g., whether a thickness value measured across the surface of the wafer falls within a target threshold window). The processing logic determines an error (i.e., a classification error or a prediction error) based on the difference between the identified chamber status or recovery prediction and the evaluated process results (e.g., sample test results). The processing logic adjusts the weights of one or more nodes of the machine learning model based on the error. An error term or delta may be determined for each node of the artificial neural network. Based on the error, the artificial neural network adjusts one or more of the artificial neural network parameters for one or more of the nodes of the artificial neural network (e.g., weights for one or more inputs of a node). Parameters may be updated using backpropagation, with the nodes in the top layer updated first, followed by the nodes in the next layer, and so on. An artificial neural network includes multiple layers of "neurons," each of which receives as input values ​​from neurons in the previous layer. The parameters for each neuron include weights associated with the values ​​received from each of the neurons in the previous layer. Adjusting the parameters may therefore involve adjusting the weights assigned to each of the inputs to one or more neurons in one or more layers of the artificial neural network.

[0106] After optimizing the model parameters, model validation may be performed to determine whether the model has improved and to determine the current accuracy of the deep learning model. After one or several training rounds, processing logic may determine whether a stopping criterion has been met. The stopping criterion may be a target accuracy level, a target number of processed images from the training dataset, a target amount of change to the parameters over one or more previous data points, a combination of these, and / or other criteria. In one embodiment, the stopping criterion is met when at least a minimum number of data points have been processed and at least a threshold accuracy has been achieved. This threshold accuracy may be, for example, 70%, 80%, or 90% accuracy. In one embodiment, the stopping criterion is met when the accuracy of the machine learning model stops improving. If the stopping criterion is not met, additional training is performed. If the stopping criterion is met, training may be completed. After the machine learning model is trained, the model may be tested using a preliminary portion of the training dataset.

[0107] After the trained machine learning model(s) 738 are generated, they may be stored in model storage 745 and added to the substrate process rate determination and / or process tuning application. The substrate process rate determination and / or process tuning application may then use the trained ML model(s) 738 and additional processing logic to implement an automatic mode in which manual user input of information is minimized, or in some instances, even eliminated.

[0108] For the apply model workflow 717, according to one embodiment, input data 762 may be input to a process rate determiner 767, which may include a trained machine learning model. Based on the input data 762, the process rate determiner 767 outputs information indicative of the likelihood that an associated substrate will contain a first process result that satisfies a threshold condition of the substrate processing recipe.

[0109] According to one embodiment, input data 762 may be input to a chamber recovery predictor 764, which may include a trained machine learning model. Based on the input data 762, the chamber recovery predictor 764 outputs a chamber recovery profile, a threshold amount of seasoning wafers required for the chamber to meet threshold conditions for operating in a production operating mode, and / or a first test result indicating the likelihood that a first substrate processed after processing a series of substrates will include process results that meet threshold conditions (e.g., associated with a substrate processing recipe).

[0110] 8 shows a block diagram of an exemplary computing device 800 that operates in accordance with one or more aspects of the present disclosure. In various illustrative examples, various components of computing device 800 may represent various components of client device 150, metrology system 110, server 120, data store 140, manufacturing system 102, and machine learning system 170 shown in FIG.

[0111] The exemplary computing device 800 may be connected to other computer devices on a LAN, an intranet, an extranet, and / or the Internet. The computing device 800 may operate in the capacity of a server in a client-server network environment. The computing device 800 may be a personal computer (PC), a set-top box (STB), a server, a network router, switch, or bridge, or any device capable of executing (sequentially or otherwise) a set of instructions that specify actions to be taken by the device. Furthermore, while only an exemplary single computing device is shown, the term "computer" should be taken to include any collection of computers that individually or jointly execute a set (or sets) of instructions to perform any one or more of the methodologies discussed herein.

[0112] The exemplary computing device 800 may include a processing unit 802 (also referred to as a processor or CPU), a main memory 804 (e.g., read-only memory (ROM), flash memory, dynamic random access memory (DRAM), such as synchronous DRAM (SDRAM)), a static memory 806 (e.g., flash memory, static random access memory (SRAM)), and a secondary memory (e.g., a data storage device 818), which may communicate with each other via a bus 830.

[0113] Processing unit 802 represents one or more general-purpose processing devices, such as a microprocessor, a central processing unit, or the like. More specifically, processing unit 802 may be a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a processor implementing other instruction sets, or a processor implementing a combination of instruction sets. Processing unit 802 may also be one or more special-purpose processing devices, such as an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), a network processor, or the like. According to one or more aspects of the present disclosure, processing unit 802 may be configured to execute instructions implementing methods 500, 600 illustrated in FIGS. 5-6 .

[0114] The exemplary computing device 800 may further include a network interface device 708, which may be communicatively coupled to a network 820. The exemplary computing device 800 may further include a video display 810 (e.g., a liquid crystal display (LCD), touch screen, or cathode ray tube (CRT)), an alphanumeric input device 812 (e.g., a keyboard), a cursor control device 814 (e.g., a mouse), and an audio signal generating device 816 (e.g., a speaker).

[0115] The data storage device 818 may include a machine-readable storage medium (or more specifically, a non-transitory machine-readable storage medium) 828 having stored thereon one or more sets of executable instructions 822. According to one or more aspects of the present disclosure, the executable instructions 822 may include executable instructions related to performing the methods 500, 600 shown in FIGS.

[0116] The executable instructions 822 may also reside, completely or at least partially, within the main memory 804 and / or within the processing unit 802 during execution of the executable instructions 822 by the exemplary computing device 800, with the main memory 804 and processing unit 802 also constituting computer-readable storage media. The executable instructions 822 may also be transmitted or received across a network via the network interface device 808.

[0117] 8 illustrates computer-readable storage medium 828 as a single medium, the term "computer-readable storage medium" should be interpreted to include a single medium or multiple media (e.g., centralized or distributed databases and / or associated caches and servers) that store one or more sets of operating instructions. The term "computer-readable storage medium" should also be interpreted to include any medium that can store or encode a set of instructions for execution by a machine, causing the machine to perform any one or more of the methods described herein. Accordingly, the term "computer-readable storage medium" should be interpreted to include, but is not limited to, solid-state memory and optical and magnetic media.

[0118] Some portions of the above detailed descriptions are presented in terms of algorithms and symbolic representations of operations on data bits within a computer memory. These algorithmic descriptions and representations are the means used by those skilled in the data processing arts to most effectively convey the substance of their work to others skilled in the art. An algorithm is here, and generally, conceived to be a self-consistent sequence of steps leading to a desired result. These steps require physical manipulations of physical quantities. Usually, though not necessarily, these quantities take the form of electrical or magnetic signals capable of being stored, transferred, combined, compared, and otherwise manipulated. It is convenient at times, principally for reasons of common usage, to refer to these signals as bits, values, elements, symbols, characters, terms, numbers, or the like.

[0119] It should be borne in mind, however, that all of these and similar terms are associated with the appropriate physical quantities and are merely convenient labels applied to those quantities. Unless otherwise indicated, as will be apparent from the discussion that follows, throughout this description, discussions utilizing terms such as "identify," "determine," "store," "adjust," "produce," "return," "compare," "produce," "stop," "load," "copy," "cast," "exchange," "execute," or other similar terms will be understood to refer to operations and processes of a computer system or similar electronic computing device that manipulate data in the computer system's registers and memory, which are represented as physical (electronic) quantities, and transform such data into other data, also represented as physical quantities, in the computer system's memory or registers or other such information storage, transmission, or display devices.

[0120] Examples of the present disclosure further relate to apparatus for performing the methods described herein. This apparatus may be specially constructed for the required purposes, or it may be a general-purpose computer system selectively programmed by a computer program stored in the computer system. Such computer program may be stored on a computer-readable storage medium, such as, but not limited to, any type of disk, including optical disks, compact disk read-only memories (CD-ROMs) and magneto-optical disks, read-only memories (ROMs), random access memories (RAMs), erasable programmable read-only memories (EPROMs), electrically erasable programmable read-only memories (EEPROMs), magnetic disk storage media, optical storage media, flash memory devices, other types of machine-accessible storage media, or any type of media suitable for storing electronic instructions, each coupled to a computer system bus.

[0121] The methods and displays presented herein are not inherently related to any particular computer or other apparatus. Various general-purpose systems may be used with programs in accordance with the teachings herein, or it may prove convenient to construct more specialized apparatus to perform the required method steps. The required structure for a variety of these systems is described below. Additionally, the scope of the present disclosure is not limited to any particular programming language. It will be understood that a variety of programming languages ​​may be used to implement the teachings of the present disclosure.

[0122] It should be understood that the foregoing description is intended to be illustrative, and not limiting. Many other examples of embodiments will be apparent to those skilled in the art upon reading and understanding the foregoing description. While particular examples have been described in the present disclosure, it will be recognized that the systems and methods of the present disclosure are not limited to the examples described herein, but may be practiced with modifications within the scope of the appended claims. Accordingly, the specification and drawings should be interpreted in an illustrative, and not a limiting, sense. The scope of the present disclosure should, therefore, be determined with reference to the appended claims and the full scope of equivalents to which such claims are entitled.

Claims

1. receiving, by a processing device, sensor data indicative of a condition of an environment of one or more pieces of manufacturing equipment associated with processing a series of substrates according to a substrate processing procedure, the series of substrates being processed after a preventive maintenance procedure has been performed on the one or more pieces of manufacturing equipment; determining, by the processing device, a first set of values ​​based on the sensor data, each value in the first set associated with a substrate in the series of substrates, each value in the first set indicative of a likelihood that the associated substrate contains a first process result that satisfies a threshold condition of the substrate processing recipe; predicting, by the processing device, a first test result based on the first set of values, the first test result being indicative of a likelihood that a processed first substrate after processing the series of substrates will include a second process result that satisfies the threshold condition; and and performing, by the processing unit, at least one of: a) preparing the first test results for presentation on a graphical user interface (GUI); or b) modifying operation of the one or more pieces of manufacturing equipment based on the first test results. A method comprising:

2. determining a first amount based on the first set of values, the first amount indicating a threshold number of substrates, the threshold number of substrates being a number of substrates that, when processed after the preventive maintenance procedure, will result in the one or more manufacturing equipment processing subsequent substrates having second process results that satisfy the threshold condition, and the threshold number of substrates and the subsequent substrates are processed according to the substrate processing procedure. The method of claim 1.

3. the series of substrates includes a second quantity of substrates, and the method further comprises: further comprising comparing the first amount to the second amount, wherein the first test result is predicted based on the comparison between the first amount and the second amount. The method of claim 2.

4. determining a first difference between the first amount and the second amount; and preparing to present a notification on the GUI indicating the first difference; The method of claim 3 further comprising:

5. determining that the one or more manufacturing equipment devices satisfy one or more conditions for a production operating mode based on results of the first test; The method of claim 1 further comprising:

6. determining a recovery profile corresponding to the one or more manufacturing equipment, the recovery profile including a distribution of indices indicating a change in the likelihood that the processed first substrate will include the second process result that satisfies the threshold condition after processing the series of substrates, and the first test result is further predicted using the distribution of indices. The method of claim 1.

7. The method of claim 6 , wherein the recovery profile further comprises a cumulative distribution function associated with the first set of values.

8. using the sensor data as input to a machine learning model; and obtaining one or more outputs of the machine learning model, the one or more outputs indicating the first set of values. The method of claim 1 further comprising:

9. 9. The method of claim 8, wherein the machine learning model comprises at least one of a bootstrap aggregation model, a random forest model, a decision tree model, or a partial least squares regression (PLS) model.

10. 1. A method for computer-aided training of a machine learning model to predict a condition of one or more pieces of manufacturing equipment, the method comprising: generating training data for the machine learning model by the computer; and generating the training data comprises: identifying a first training input having first sensor data indicative of a first state of a first piece of manufacturing equipment after performing a preventive maintenance procedure; and identifying a first target output for the first training input, the first target output indicating whether a first substrate processed according to a first substrate processing procedure by the first manufacturing equipment in the first state will result in the first substrate having a first process result that satisfies a first threshold condition associated with the first substrate processing procedure. wherein the method further comprises: receiving, by the computer, the training data to train the machine learning model with respect to (i) a set of training inputs including the first training inputs and (ii) a set of target outputs including the first target outputs, wherein the trained machine learning model receives new inputs having new sensor data indicative of a new state of new manufacturing equipment, and new outputs indicative of whether a new substrate processed according to the first substrate processing procedure by the new manufacturing equipment in the new state will result in the new substrate having a new process result that satisfies the threshold condition; A method comprising:

11. The method of claim 10 , wherein the first sensor data comprises one or more measurements obtained after the first manufacturing equipment processes a first series of substrates.

12. 11. The method of claim 10, wherein the first sensor data comprises one or more measurements obtained prior to the first manufacturing equipment processing a first series of substrates.

13. The method of claim 10 , wherein each training input in the set of training inputs is mapped to one target output in the set of target outputs.

14. A non-transitory machine-readable storage medium containing instructions that, when executed by a processing device, cause the processing device to perform operations, the operations including: receiving sensor data indicative of a condition of an environment of one or more pieces of manufacturing equipment processing a series of substrates according to a substrate processing procedure, the series of substrates being processed after a preventive maintenance procedure has been performed on the one or more pieces of manufacturing equipment; determining a first set of values ​​based on the sensor data, each value in the first set being associated with a substrate in the series of substrates, each value in the first set being indicative of a likelihood that the associated substrate will contain a first process result that satisfies a threshold condition of the substrate processing recipe; predicting a first test result based on the first set of values, the first test result being indicative of a likelihood that a processed first substrate after processing the series of substrates will include a second process result that meets the threshold condition; and and at least one of a) preparing the first test results for presentation on a graphical user interface (GUI); or b) modifying operation of the one or more pieces of manufacturing equipment based on the first test results.

1. A non-transitory machine-readable storage medium comprising:

15. The operation is determining a first amount based on the first set of values, the first amount indicating a threshold number of substrates, the threshold number of substrates being a number of substrates that, when processed after the preventive maintenance procedure, will result in the one or more manufacturing equipment processing subsequent substrates having second process results that satisfy the threshold condition, and the threshold number of substrates and the subsequent substrates are processed according to the substrate processing procedure.

15. The non-transitory machine-readable storage medium of claim 14.

16. the series of substrates includes a second quantity of substrates, and the operation comprises: further comprising comparing the first amount to the second amount, wherein the first test result is predicted based on the comparison between the first amount and the second amount.

16. The non-transitory machine-readable storage medium of claim 15.

17. The operation is determining a first difference between the first amount and the second amount; and preparing to present a notification on the GUI indicating the first difference; 20. The non-transitory machine-readable storage medium of claim 16, further comprising:

18. The operation is determining a recovery profile corresponding to the one or more manufacturing equipment, the recovery profile including a distribution of indices indicating a change in the likelihood that the processed first substrate will include the second process result that satisfies the threshold condition after processing the series of substrates, and the first test result is further predicted using the distribution of indices.

15. The non-transitory machine-readable storage medium of claim 14.

19. The operation is using the sensor data as input to a machine learning model; and obtaining one or more outputs of the machine learning model, the one or more outputs indicating the first set of values.

15. The non-transitory machine-readable storage medium of claim 14, further comprising:

20. 20. The non-transitory machine-readable storage medium of claim 19, wherein the machine learning model comprises at least one of a bootstrap aggregation model, a random forest model, a decision tree model, or a partial least squares regression (PLS) model.

Images