Method and apparatus for predicting a process metric associated with a process

Abstract

A method, the method comprising: obtaining one or more models configured to predict process metrics of a manufacturing process based on input process data; and based on inputting new process data to the one or more models, using a reinforcement learning framework to evaluate the one or more models and / or model configurations of the one or more models, and determining a performance indication of the one or more models and / or model configurations in predicting the process metrics based on the input new process data.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to EP Application No. 20167729.1 filed on April 2, 2020 and EP Application No. 20170414.5 filed on April 20, 2020, which are incorporated herein by reference in their entirety. Technical Field

[0003] The present invention relates to a computer-implemented method and apparatus for predicting a process metric associated with a process. In particular, it relates to selecting a model configuration for predicting a process metric from a plurality of model configurations. The method and apparatus may be applicable to industrial manufacturing processes, such as semiconductor manufacturing processes. Background Art

[0004] A manufacturing process, such as a process for patterning a semiconductor substrate, often requires adequate registration of features that characterize the manufacturing process in order to monitor whether the process is stable. Examples of such features are key performance indicators (KPIs) associated with the devices used in manufacturing and characteristics of features provided to the substrate. The values ​​of these features are often referred to as process data.

[0005] Process data is crucial as input to predictive models, which are used to infer expected characteristics of a manufacturing process, such as the yield of the process or the quality of a single substrate after processing. Today, such predictive models are often further used to control the process, such as to modify operating parameters of the manufacturing equipment and / or to assist in making rework decisions on a single substrate after undergoing one or more process steps.

[0006] However, current usage of predictive models is rather static and thus prone to losing accuracy over time because a) the features characterizing the manufacturing process often exhibit significant evolution (drifts, jumps), and / or b) the amount of process data is not fixed but increases over time, so that the initially selected predictive model or predictive model configuration is often no longer optimal at a later point in time. Summary of the invention

[0007] The object of the present invention is to overcome the disadvantages of the prior art methods.

[0008] According to a first aspect of the present disclosure, a computer-implemented method is provided for selecting a model configuration for predicting a process metric associated with a process from a plurality of model configurations. The method comprises: obtaining process data, the process data comprising values ​​associated with a plurality of model configurations and values ​​associated with process characteristics; evaluating a performance indicator of each of the plurality of model configurations based on the process data; selecting a model configuration based on the evaluated performance indicator; predicting a process metric using a model having the selected model configuration; and outputting the predicted process metric by a computer.

[0009] The process may include one of the following: industrial processing, industrial manufacturing, weather prediction, traffic management, network management, economic forecasting, medical diagnosis, or any other process in a complex environment.

[0010] A process metric may include a characteristic of a process. A process metric may include one of the following: yield, process quality, probability of process failure, probability of component failure in a process environment, diagnostic status.

[0011] Predicting the process metric may include providing a value associated with the process characteristic as an input to a model having a selected model configuration, and receiving a prediction of the process metric as an output from the model.

[0012] The steps of the method may be performed by one or more processors of a computer system.

[0013] Process data may include values ​​associated with a feature or features of the process and characteristics or features associated with the environment in which the process occurs. The characteristics may represent the state of the environment in which the process occurs. Examples of characteristics are critical dimensions, temperature, power.

[0014] The value associated with the plurality of model configurations may include the current set of model configurations in the environment. The value associated with the plurality of model configurations may include the accuracy of a previously predicted process metric predicted using a previously selected model configuration. That is, the method may be performed repeatedly and using data from previous repetitions. Accuracy may be determined using standard performance measurement techniques, such as an area under a receiver operating characteristic (AUC-ROC) value or precision and recall.

[0015] The performance indicator may represent a measure of the performance (eg, accuracy) of the predictions made by the selected model. The performance indicator may be determined analytically, for example, using an advantage function or temporal differences.

[0016] Using this approach, a different model configuration can be selected if its performance exceeds the performance of the current model configuration. Beneficially, this approach yields the following advantages:

[0017] Process metrics can be predicted with improved accuracy. Specifically, the accuracy can be higher than that achievable using manual intervention, for example because the methods described herein are data driven rather than heuristic. Selecting a model configuration using the methods described herein can be much faster than what can be achieved using manual intervention, thereby allowing real-time updates to the prediction process. Using and applying corrections using the methods described herein can increase the productivity of the process, for example when the process is optimized and downtime is avoided. The methods described herein are also very flexible because the methods can be applied to many environments and processes and can be trained to learn new environments and processes and / or learn changes within environments and processes.

[0018] Additionally, by using the methods described herein, a model selection tool is provided that requires less storage space than successful methods, for example because fewer models need to be stored. Specifically, fewer models need to be stored because the selection of model configurations allows the model to be updated in response to new data, rather than requiring an entirely new model.

[0019] The values ​​associated with a process characteristic may include data associated with how the characteristic changes over time.

[0020] Data associated with the way a feature changes over time may be referred to as time-varying data. Time-varying data may include values ​​associated with features monitored over time. A drift state may be determined based on the time variation of a feature. The drift state may be used to identify a drift event. The drift state may include one of: no drift, sudden drift, gradual drift, incremental drift, recurring concept drift. Alternatively, the drift state may be determined periodically, such as after a selected time period has passed.

[0021] Each model configuration may include one or more of a model type, model parameters, and / or model hyperparameters.

[0022] Model type may represent the type of model architecture and / or the type of algorithm. Model parameters may represent parameters determined during training of the model using training data. Model hyperparameters may represent model parameters that are typically set prior to training. Examples of hyperparameters are greediness, learning rate, number of layers.

[0023] Selecting a model configuration may include selecting a first model having a first model type, first model parameters, and first model hyperparameters. The first model type and / or the first model parameters and / or the first model hyperparameters may be the same as a previous model type, previous model parameters, and previous model hyperparameters of a previously selected model. That is, it may be determined that retaining the same model configuration as previously selected is beneficial. Alternatively, one or more of the first model type, first model parameters, and first model hyperparameters may be different from a previous model type, previous model parameters, and / or previous model hyperparameters.

[0024] Model types can include support vector machines, neural networks, or random forests.

[0025] Selecting a model configuration may include retraining the model to update its model parameters. Specifically, retaining the model may be performed after a drift event. The model may be retrained using values ​​associated with one or more features of the process.

[0026] Selecting a model configuration may be further based on a data set including values ​​associated with features of the process. The data set may include stored values ​​from previous iterations of the method. The data set may include previous measured values ​​associated with features of the process.

[0027] Selecting a model configuration may be further based on the size of the data set or the size of a subset of the data set. The size of the data set may include the number of values ​​in the data set. The subset may include a set of values ​​in the data set associated with a particular feature. If the size of the data set or the size of the subset of the data set exceeds a predetermined threshold, it may be determined that switching from one model type to a different model type is beneficial.

[0028] The method steps may be performed using reinforcement learning. The method steps may be performed as at least part of a model-free reinforcement learning (MFRL) framework.

[0029] The steps of obtaining process data, evaluating performance metrics and selecting a model configuration may be performed using one or more of an action-critic algorithm, an advantage action-critic algorithm, an asynchronous advantage action-critic algorithm, Q-learning with a normalized advantage function, a trust region policy optimization algorithm, a proximal policy optimization algorithm, a double-delayed deep deterministic policy gradient, or a soft action-critic algorithm.

[0030] The steps of obtaining process data and evaluating performance metrics may be performed by one or more processors including a reviewer, and the step of selecting a model configuration is performed by one or more processors including an agent.

[0031] The steps of obtaining process data, evaluating performance metrics, and selecting a model configuration may include: determining a policy function J(θ) that defines a selection method for selecting a model configuration; and / or determining a value function V(θ) that defines an enhancement of the process metric prediction assuming that the policy function is followed until the policy function is completed. That is, the reinforcement learning framework may include a policy-based framework and / or a value-based framework.

[0032] A selection method may be defined to control the way in which a model configuration is selected. The selection method may include an algorithm that may be based on the total reward until the policy is completed. Gradient ascent may be used to optimize the policy function.

[0033] The method may also include determining a correction to the process based on the predicted process metric. The correction may be determined based on the predicted process metric and / or a value associated with a feature of the process. The correction may be determined so as to improve a performance metric of the process. In the method where the correction is determined, the method may be referred to as a method for determining a correction to a process.

[0034] The method may further comprise: applying the correction to the process. The correction may be selectively applied, for example it may be applied only when the predicted process metric passes a threshold. In the method where an adjustment is applied, the method may be referred to as a method for applying an adjustment to a process. Application of the adjustment may improve the process metric of the process.

[0035] The process may include a semiconductor manufacturing process. The semiconductor manufacturing process may be performed on semiconductor manufacturing equipment, such as a lithography apparatus. The semiconductor manufacturing equipment may include, or may be in communication with, a measurement apparatus arranged to measure features of the process.

[0036] The correction may be determined to adjust optical and / or mechanical parameters of a semiconductor manufacturing device arranged to perform a semiconductor manufacturing process, or to change a processing step of a semiconductor manufacturing process. For example, the dose of a particular step of a semiconductor manufacturing process may be changed. In another example, the radiation beam may be adjusted to a different position using optical and / or mechanical adjustments.

[0037] The value associated with the characteristic of the process may be generated by an inspection device arranged to monitor at least a portion of the semiconductor manufacturing process. The inspection device may be referred to as a metrology tool. The inspection device may be arranged to inspect a substrate to measure characteristics of the substrate and / or patterned structures on the substrate, such as overlay error between subsequent layers, line thickness, or critical dimension.

[0038] According to a second aspect of the present disclosure, there is provided a method for training an evaluation model for evaluating performance indications for each of a plurality of model configurations for a model set, the model set having the plurality of model configurations and configured to predict process metrics associated with a process, the method comprising: receiving prediction data related to a predicted process metric predicted by at least one model in the model set, receiving values ​​associated with features of the process, and training the evaluation model based on the received values ​​and the received prediction data.

[0039] The evaluation model may include a policy function used by the review of the reinforcement learning model. That is, the evaluation model may be executed by one or more processors including the review. Training the evaluation model may include updating the policy function based on the received value and the received prediction data.

[0040] According to a third aspect of the present disclosure, there is provided a method for training a selection model for selecting a model configuration from a plurality of model configurations for a model set, the model set having the plurality of model configurations and configured to predict process metrics associated with a process, the method comprising: receiving values ​​associated with characteristics of the process, receiving performance indications indicating performance of a specific predicted process metric predicted using a specific model configuration, and training the selection model based on the received values ​​and the received performance indications.

[0041] The selection model may include a policy function used by an agent of the reinforcement learning model. That is, the selection model may be executed by one or more processors including the agent. Training the selection model may include updating the policy function based on the received value and the received performance indication. More than one performance indication may be received, for example indicating the performance of multiple specific prediction process metrics, each prediction process metric being predicted using a different model configuration.

[0042] According to a fourth aspect of the present disclosure, a machine learning model trained according to the second aspect or the third aspect is provided. The machine learning model may be stored on a computer storage medium.

[0043] According to a fifth aspect of the present disclosure, a method is provided, the method comprising: obtaining one or more models, the one or more of which are configured to predict process metrics of a manufacturing process based on input process data; and based on inputting new process data into the one or more models, using a reinforcement learning framework to evaluate the one or more models and / or model configurations of the models, and determining performance indicators of the one or more models and / or model configurations when predicting process metrics based on the input of the new process data.

[0044] Each model used to predict a process metric for a manufacturing process may have a model configuration.The performance indicator may represent a measure of the performance (eg, accuracy) of the predictions made by the selected model.

[0045] Process metrics can be associated with yield or quality parameters of a manufacturing process.

[0046] A model configuration may include one or more of a model type, model parameters, and / or model hyperparameters.

[0047] The process data and new process data may include features of the manufacturing process. The process data may include a data set, such as a validation set, a training set, or a test set. The process data may include values ​​associated with one or more features and / or one or more characteristics of the manufacturing process associated with a manufacturing environment. The new process data may be process data that has not been previously input to the model. The new process data may be process data measured during the manufacturing process, for example using a metrology tool.

[0048] The manufacturing process may include a process of patterning a semiconductor wafer.

[0049] The performance indication may be based on comparing the predicted process metric to values ​​of the process metric obtained in other ways.

[0050] The method may also include selecting a model and / or a model configuration based on the evaluation.

[0051] Process metrology may indicate the quality of a substrate undergoing a patterning process.

[0052] The process data may include key performance indicators (KPIs) associated with a plurality of functional modules involved in the manufacturing process.

[0053] The one or more models may include at least two machine learning based models, each model having a different architecture.

[0054] The one or more models may include at least a first model based on supervised learning and a second model based on unsupervised learning.

[0055] According to a sixth aspect of the present disclosure, there is provided a computer program comprising instructions, which, when run on a suitable device, are operable to perform the method of the first aspect, the second aspect, the third aspect and / or the fifth aspect.

[0056] According to a seventh aspect of the present disclosure, there is provided a computer storage medium comprising the computer program of the sixth aspect.

[0057] According to an eighth aspect of the present disclosure, there is provided a semiconductor manufacturing device, comprising a processing device and a storage medium comprising instructions, wherein the instructions are operable to execute the method of the first aspect, the second aspect, the third aspect or the fifth aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Embodiments of the invention will now be described, by way of example only, with reference to the accompanying schematic drawings, in which:

[0059] - Figure 1 Illustration of common drift events observed in complex environments;

[0060] - Figure 2 The model selection model is described;

[0061] - Figure 3 depicts a schematic overview of a lithography apparatus;

[0062] - Figure 4 A flow chart depicting steps in a method of selecting a model configuration from a plurality of model configurations for predicting a process metric associated with a process. DETAILED DESCRIPTION

[0063] Machine learning models can be used to predict characteristics in complex environments. Machine learning models can include different types of models, such as artificial neural networks, decision trees, support vector machines, and / or Bayesian networks. Machine learning models may be referred to as models in this article.

[0064] Each model has a set of model parameters whose values ​​are derived via training. Machine learning models can be trained using techniques such as supervised learning, unsupervised learning, or reinforcement learning. In training, training data associated with the environment is provided to the model so that the model can be trained to recognize patterns within the data and update the model parameters. After training, the model can be used to predict characteristics in the environment based on the received data associated with the environment.

[0065] Each model also has a set of hyperparameters whose values ​​or types are typically selected before training. Hyperparameters may include, for example, the learning rate of a model, the depth of a tree in a decision tree model, the number of layers and / or neurons in a neural network, and exploration versus exploitation in a reinforcement learning model.

[0066] When using a model to predict characteristics in a complex environment, the type of model and model hyperparameters can be selected based on the type of environment, the amount of data, and / or the type of data received from the environment. In addition, the model is provided with training data from the environment so that it is trained to make predictions specifically for that environment. Thus, each model is customized for a specific environment.

[0067] Models can be used to predict characteristics in a range of environments and / or processes within such environments. Examples of such environments are industrial processing and / or manufacturing environments, weather systems, road or waterway transportation systems, crowds in public places, computer networks, economic systems such as stock markets, and medical diagnostic systems for human or animal bodies (or their in vitro samples). For example, models can be used to predict characteristics (such as yield, quality, throughput, maintenance status, diagnostic status) associated with industrial processes in industrial manufacturing or processing environments. Models can be used to predict the possibility and / or location of weather types or events (such as hurricanes, heat waves) in local or global weather systems. Models can be used to predict movement in complex environments, such as the movement of animals in herds, the movement of people in crowds, or the movement of traffic on roads. Models can be used to predict elasticity and / or exit in computer networks. Models can be used for medical diagnosis, such as the possibility of predicting medical conditions based on data received from medical devices. Some characteristics in the environment are associated with features that can be measured, such as temperature, concentration, or speed, and can be referred to as features. Some characteristics in the environment are associated with processes occurring in the environment (such as the yield of the process, the possibility of successful diagnosis), so they can be called process metrics.

[0068] The above environments and others are complex and dynamically change over time. Each environment is influenced by multiple internal and / or external stimuli. The relationship between each stimulus and its effect on the environment is often not well understood. Given training data from an environment, a machine learning model can be trained to predict the properties of the environment regardless of whether the model can determine the specific stimuli and / or the specific effects of the stimuli on the environment.

[0069] Many complex environments experience drift, where the mean value of a property changes systematically over time. The occurrence of drift at a specific time or time period can be referred to as a drift event. Figure 1 (a) to 1(d) illustrate common drift events observed in complex environments. Each drift event is illustrated using a measured average of environmental characteristics measured over time. In each case, the measured value changes from a first average value 10 to a second average value 12 during the drift event.

[0070] When the measured value changes from the first average value 10 to the second average value 12, usually discontinuously, it occurs Figure 1 The drift event depicted in (a) is called a sudden population drift. The measured value remains at the second mean value 12 after the transition. In examples, a sudden population drift may occur when there is a discontinuous adjustment to the environment (such as a change in the operating parameters of a manufacturing facility, a volcanic eruption, or a solar event that affects a weather system, etc.).

[0071] When the measured value changes between the first average value 10 and the second average value 12, Figure 1 The second drift event depicted in (b), called a gradual population drift, involves a transition between decreasing periods at the first mean 10 and increasingly longer periods at the second mean 12. This may occur multiple times. The transition between the first mean 10 and the second mean 12 is typically discontinuous. Each time the first mean 10 transitions to the second mean 12, the amount of time the measurement remains at the second mean 12 increases. Figure 1 In the illustrated example, the measured values ​​remain at the second mean value for a first time period 13, a second time period 14, and a third time period 15, wherein the third time period 15 is greater than the second time period 14, and the second time period 14 is greater than the third time period 13. Examples of gradual population drift in a real environment are intermittent faults that increase in severity over time, such as due to accumulation of debris in a device and symptoms of a medical condition that a patient is treating by increasing doses of a drug to which the patient develops tolerance or resistance.

[0072] When the measured value starts from the first average value 10 and gradually changes to the second average value 12, for example, increases or decreases in gradual increments, a Figure 1 The third drift event depicted in (c) is called incremental population drift. That is, the transition is usually continuous over the transition period 16. An example of incremental population drift is a manufacturing device whose temperature gradually increases, resulting in, for example, an incremental increase in yield. Yet another example might be an atmospheric effect caused by the accumulation of contaminants over time.

[0073] When the measured value starts from the first average value 10 and changes to the second average value 12, it occurs Figure 1 The fourth drift event depicted in (d) is called a recurring group concept. The transition between the first average value 10 and the second average value 12 is usually discontinuous. Before the transition to the first average value 10, the measured value remains at the second average value 12 for a fourth time period 17. An example of a recurring group concept in a real environment is an intermittent failure in an industrial machine or a symptom of a medical condition that exhibits redness (or "sudden recurrence").

[0074] The trained machine learning model effectively uses historical data from the environment to learn. For example, the machine learning model may use training data from a time period where the measured values ​​were at a first average value of 10. After a drift event, the measured values ​​are at a second average value of 12. Therefore, the trained model is less able to accurately predict the characteristic after the drift event (i.e., may predict the characteristic with reduced accuracy).

[0075] As described above, a drift event can be associated with a drift in the value of a feature associated with the environment. Such a drift event can be referred to as a feature drift. However, a drift event may be associated with other quantities. For example, a drift event may correspond to a drift in a process metric, such as a reduction in yield. Such a drift event may be referred to as a process drift. In another example, a drift event may correspond to a performance metric associated with a model for predicting a characteristic. That is, the performance of the model prediction can be determined, for example using standard performance measurement techniques (such as the area under the receiver operating characteristic (AUC-ROC) value, or precision and recall), and the determined performance may be improved or reduced in drift events similar to those described above.

[0076] In order to improve the accuracy of predicting the characteristics in the environment after the drift event, it may be beneficial to update the model. For example, the model can be retrained using training data from a time period when the measured value is at the second average value 12. Additionally or alternatively, the model architecture and / or hyperparameters can be updated. This updating process can be referred to as selecting a model configuration, where the model configuration includes a specific model type, architecture, parameters, and hyperparameters.

[0077] Typically, the selection of model configurations is performed by manual intervention, where a human operator manually selects the model configuration. Manual intervention may include a 'heuristic' approach, where the model configuration is selected based on intuition. Heuristic methods are generally less accurate. Key performance indicators (KPIs) can be used to improve heuristic methods, such as by monitoring KPIs so that processes within an environment remain within a desired performance window (i.e., defined as 'adequate' performance). KPIs can use statistical analysis to select model configurations. However, manual intervention (including the use of statistical analysis) is generally unable to understand the complex relationship between model configuration and prediction quality. For example, it may not be possible to understand how changing the learning rate may or may not improve the prediction and / or when retraining to update model parameters may be beneficial). Additionally, manual intervention may not be able to update the model at the speed required for real-time updates. Additionally, manual intervention methods are generally unable to specifically respond to drift events.

[0078] Improved methods and systems for selecting model configurations are described herein. The technology described herein provides an improved way to select or update a model configuration so that predictions of properties in an environment are improved. The technology provides responsive real-time updating of the model, thereby providing predictions responsive to changes in the environment, such as changes caused by drift events. The methods described herein can be fully automated, thereby reducing or eliminating the need for human intervention.

[0079] Figure 2A model selection model 200 is depicted that can be used to select a model configuration. The model selection model 200 includes a reinforcement learning model. The model selection model 200 is used with an environment 20, which includes one of the example environments discussed above or another complex environment.

[0080] Environment 20 is initially in state s at time t t Status t The method comprises a data set D and a model configuration set M. The data set D comprises one or more values ​​associated with an environmental characteristic (e.g., temperature in a weather system, power in an industrial process environment, etc.). The characteristic can be measured, for example, by a measurement device. It should be understood that each data entry item in the data set D can be referred to as a characteristic or a value associated with the characteristic, indicating that the characteristic is digitally represented in the form of a data entry item.

[0081] Through data set D, drift state F can be determined. Drift state F can identify drift events, and optionally identify one or more features of the drift event (e.g., the type of drift event, the amplitude of the drift, etc.). Drift state can be determined by monitoring features over time and identifying the way features change over time. For example, the value associated with a feature may be discontinuously changed to a new value and remain at the new value, indicating a sudden population drift. Any number of features can be monitored to determine drift. Data associated with the way features change over time can be referred to as time-varying data. Such time-varying data and / or drift state can be stored in data set D.

[0082] The environment 20 also includes a set of trained machine learning models that have been trained to predict characteristics in the environment. For example, they may have been trained on historical data in a dataset D. Each machine learning model has a model configuration. Thus, a set of model configurations M is the configuration of each model in the set. The set of models may include any type of model, for example, they may include neural networks and / or reinforcement learning models that are operable to predict characteristics in the environment.

[0083] Each model in the set of models in the environment 20 is operable to predict a characteristic of the environment. At a given time, a model (with an associated model configuration) is selected to predict the characteristic. The purpose of the model selection model is to select the 'best model configuration' for each given time, so that the accuracy of the prediction is improved.

[0084] The environment is activated at time t by agent 22 through action a t Take effect. Action t Including agent 22 to select the model configuration. That is, action a tThe agent 22 may be a retraining action (in which the model parameters of the model are updated) and / or an action that changes model hyperparameters (e.g., increasing the greediness of a reinforcement learning model) and / or selecting a different model (e.g., switching from using a reinforcement learning model for prediction to using a neural network for prediction). By acting on the environment, the agent 22 causes the environment 20 to move from its initial state s t (where the first model configuration is selected) transitions to a new state s t+1 (wherein the second model configuration is selected.) For the sake of completeness, it should be noted that the second model configuration may be equivalent to the first model configuration if it is deemed beneficial to retain the same model configuration. That is, a different model configuration is selected if the performance of the currently used model configuration is exceeded.

[0085] From the initial state s t To the new state s t+1 The transformation returns the reward r t . Reward t is a measure of the performance of an action performed by the agent. For example, the reward r t The accuracy of predictions (e.g., predicted features) made using the selected model configuration may be included. Accuracy may be calculated using standard performance measurement techniques, such as the area under the receiver operating characteristic (AUC-ROC) value or precision and recall.

[0086] The goal of agent 22 is to maximize the total reward of its actions over a trajectory, which is defined as a sequence of states, actions, and rewards (s0, a0, r0, s1, a1, r1, s2, a2, ...) over a time period. The sequence of rewards over a trajectory is controlled by a policy π(θ) which is itself controlled by a set of parameters θ, and has an associated total reward J(θ). Agent 22 aims to optimize the total reward J(θ). In some example embodiments, to compute the optimal total reward J(θ), gradient ascent (or descent) may be used, e.g., to compute Different types of reinforcement learning techniques may be used beneficially. By way of example only, an action-review algorithm, an advantage action-review algorithm, an asynchronous advantage action-review algorithm, Q-learning with a normalized advantage function, a trust region policy optimization algorithm, a proximal policy optimization algorithm, a double-delayed deep deterministic policy gradient, or a soft action-review algorithm may be used. The agent 22 may be trained using an initial training data set. The agent 22 may also be trained based on the data set D and at each action a t The reward r received later t Continuous learning.

[0087] To help the agent 22 learn (and thus select a better model configuration), the model selection model is provided with a critic 24. tAfter having been executed by agent 22, critic 24 receives reward r from environment 20 t The critic 24 also receives the new state s of the environment 20 t+1 The critic 24 is controlled by a policy π(ω), which is itself controlled by a set of parameters ω and has an associated total reward J(ω). The goal of the critic 24 is to optimize its total reward J(ω) and to provide a score to the agent 22 based on the reward returned by the environment 20. Therefore, the critic 24 takes the information c t The agent 22 may be provided with a performance indicator (e.g., a performance indicator) that allows the agent 22 to further optimize its policy π(θ). The performance indicator represents a measure of the performance (e.g., accuracy) of the agent's actions. For example, the performance indicator may represent the accuracy of a prediction using a model configuration selected by the agent 22. The performance indicator may be determined analytically, such as using an advantage function or a time difference.

[0088] Beneficially, such a model selection model provides a means of selecting a model configuration that improves predictions of properties in an environment. By using a model selection model, selection (and therefore improvement of predictions) can be performed without the need for human intervention. By using a model selection model, selection can be performed in real time and / or in response to drift events.

[0089] The use of such a model selection model is described in more detail below, with the example of a specific process of semiconductor manufacturing in an environment including a lithographic apparatus. It should be understood that this is illustrative in nature, and the methods and apparatus described herein can be used in other applications, such as in the environment discussed above.

[0090] A lithographic apparatus is a machine configured to apply a desired pattern to a substrate. A lithographic apparatus may be used, for example, in the manufacture of integrated circuits (ICs). A lithographic apparatus may project a pattern (also commonly referred to as a "design layout" or "design"), for example, at a patterning device (e.g., a mask) onto a layer of radiation-sensitive material (resist) disposed on a substrate (e.g., a wafer).

[0091] In order to project a pattern onto a substrate, a lithographic apparatus can use electromagnetic radiation. The wavelength of the radiation determines the minimum size of a feature that can be formed on a substrate. Typical wavelengths currently used are 365nm (i-line), 248nm, 193nm, and 13.5nm. Compared to lithographic apparatus using, for example, radiation with a wavelength of 193nm, lithographic apparatus using extreme ultraviolet (EUV) radiation with a wavelength in the range of 4 to 20nm (e.g., 6.7nm or 13.5nm) can be used to form smaller features on a substrate.

[0092] Low k1 lithography can be used to process features with sizes smaller than the classical resolution limit of the lithographic apparatus. In this process, the resolution formula can be expressed as CD=k1×λ / NA, where λ is the radiation wavelength employed, NA is the numerical aperture of the projection optics in the lithographic apparatus, CD is the "critical dimension" (usually the minimum feature size printed, but in this case it is half pitch), and k1 is the empirical resolution factor. In general, the smaller the k1, the more difficult it is to reproduce a pattern similar to the shape and size planned by the circuit designer on the substrate to achieve specific electrical functionality and performance. In order to overcome these difficulties, complex fine-tuning steps can be applied to the lithographic projection equipment and / or the design layout. For example, these include but are not limited to optimization of NA, customized illumination schemes, the use of phase-shifted patterning devices, various optimizations of the design layout such as optical proximity correction (OPC, sometimes also referred to as "optical and process correction") in the design layout, or other methods generally defined as "resolution enhancement technology" (RET). These fine-tuning steps can be referred to as corrections. Alternatively, a tight control loop for controlling the stability of the lithographic apparatus can be used to improve the reproduction of patterns at low k1.

[0093] When the difficulties in the above-mentioned lithographic patterning are not resolved, this may result in a reduction in the quality of the patterning process. If the quality of the patterning process is reduced, this may result in a reduction in the quality of the final patterned substrate. Therefore, evaluating the quality of the patterned substrate can give an indication of the quality of the patterning process. In order to test the quality of the patterned substrate, it is possible to test whether the patterned substrate works. The patterned substrate may include a plurality of elements (e.g., dies) that may be tested individually to determine whether the element passes, that is, whether the element works, or whether it fails (does not work). The proportion of working elements on a substrate may be referred to as the yield of the substrate. It is desirable to improve the yield of the lithographic apparatus and the corresponding patterning process to obtain more working elements on a substrate.

[0094] Figure 3 A lithographic apparatus LA is schematically depicted. The lithographic apparatus LA comprises an illumination system (also referred to as an illuminator) IL configured to condition a radiation beam B (e.g. UV radiation, DUV radiation or EUV radiation), a mask support (e.g. a mask table) MT configured to support a patterning device (e.g. a mask) MA and connected to a first positioner PM (configured to accurately position the patterning device MA according to certain parameters), a substrate support (e.g. a wafer stage) WT configured to hold a substrate (e.g. a resist-coated wafer) W and connected to a second positioner PW (configured to accurately position the substrate support according to certain parameters), and a projection system (e.g. a refractive projection lens system) PS configured to project a pattern imparted to the radiation beam B by the patterning device MA onto a target portion C (e.g. comprising one or more dies) of the substrate W.

[0095] In operation, the illumination system IL receives a radiation beam from a radiation source SO (e.g., via a beam delivery system BD). The illumination system IL may include various types of optical components, such as refractive, reflective, magnetic, electromagnetic, electrostatic and / or other types of optical components, or any combination thereof, for directing, shaping and / or controlling the radiation. The illuminator IL may be used to condition the radiation beam B so as to have a desired spatial and angular intensity distribution in its cross-section at the plane of the patterning device MA.

[0096] The term "projection system" PS as used herein should be broadly interpreted as covering various types of projection systems, including refractive, reflective, catadioptric, anamorphic, magnetic, electromagnetic and / or electrostatic optical systems or any combination thereof, as appropriate for the exposure radiation used and / or other factors (such as the use of immersion liquid or the use of a vacuum). Any use of the term "projection lens" herein may be considered synonymous with the more general term "projection system" PS.

[0097] In this document, the terms "radiation" and "beam" are used to cover all types of electromagnetic radiation, including ultraviolet radiation (e.g., wavelengths of 365, 248, 193, 157 or 126 nm) and EUV (extreme ultraviolet radiation, e.g., wavelengths in the range of about 5 to 100 nm).

[0098] The terms "reticle", "mask" or "patterning device" as used herein may be broadly interpreted as referring to a general patterning device that can be used to impart a patterned cross-section to an incoming radiation beam, corresponding to the pattern to be created in a target portion of the substrate. In this context, the term "light valve" may also be used. In addition to classical masks (transmissive or reflective, binary, phase-shifting, hybrid, etc.), other examples of such patterning devices include programmable mirror arrays and programmable LCD arrays.

[0099] The lithographic apparatus LA may be of a type in which at least a portion of the substrate may be covered by a liquid having a relatively high refractive index, such as water, to fill the space between the projection system PS and the substrate W, which is also known as immersion lithography. More information on immersion technology is given in US6952253, which is incorporated herein by reference.

[0100] The lithographic apparatus LA may also be of a type having two or more substrate supports WT (also referred to as "dual stage"). In such a "multi-stage" machine, the substrate supports WT may be used in parallel, and / or steps for preparing a substrate W for subsequent exposure may be performed on a substrate W located on one of the substrate supports WT, while another substrate W on another substrate support WT is used to expose a pattern on another substrate W.

[0101] In addition to the substrate support WT, the lithographic apparatus LA may comprise a measuring table. The measuring table is arranged to hold sensors and / or cleaning means. The sensors may be arranged to measure properties of the projection system PS or properties of the radiation beam B. The measuring table may hold a plurality of sensors. The cleaning means may be arranged to clean a part of the lithographic apparatus, e.g. a part of the projection system PS or a part of a system for providing immersion liquid. The measuring table may be moved under the projection system PS when the substrate support WT is away from the projection system PS.

[0102] In operation, a radiation beam B is incident on a patterning device (e.g. a mask MA) which is held on a mask support T and is patterned by a pattern (design layout) present on the patterning device MA. After traversing the mask MA, the radiation beam B passes through a projection system PS which focuses the beam onto a target portion C of the substrate W. With the aid of a second positioner PW and a position measurement system IF, the substrate support WT can be accurately moved, for example in order to position different target portions C in the path of the radiation beam B at focused and aligned positions. Similarly, the first positioner PM and possibly another position sensor (not shown) are also provided. Figure 1 The patterning device MA (depicted explicitly in FIG. 1 ) can be used to accurately position the patterning device MA relative to the path of the radiation beam B. The patterning device MA and the substrate W can be aligned using the mask alignment marks M1, M2 and the substrate alignment marks P1, P2. Although the illustrated substrate alignment marks P1, P2 occupy dedicated target portions, they can be located in the space between target portions. When the substrate alignment marks P1, P2 are located between target portions C, these are referred to as scribe lane alignment marks.

[0103] Typically, the patterning process in the lithographic apparatus LA is one of the most critical steps in the process, which requires high accuracy in dimensioning and placement of structures on the substrate W. It is desirable to monitor properties associated with the patterning process and / or the substrate W and / or the lithographic apparatus. Such properties may be referred to as features, measured features, or monitored features.

[0104] The lithographic apparatus LA also includes an inspection apparatus IA, which may also be referred to as a metrology tool. It should be noted that in other example arrangements of the lithographic apparatus, the inspection apparatus IA may not be integrated into the lithographic apparatus, but may be part of a larger lithographic system or a stand-alone device. If integrated into the lithographic apparatus LA, the positioning of the inspection apparatus IA may be different from that of the inspection apparatus IA. Figure 3 Different than depicted in .

[0105] The inspection apparatus IA is arranged to inspect a substrate to measure features of the substrate W and patterned structures on the substrate, such as overlay errors between subsequent layers, line thickness, critical dimensions (CD), etc. If errors are detected, corrections can be made, for example to subsequent exposure of the substrate or other processing steps to be performed on the substrate W. The inspection apparatus IA can measure features on a latent image (the image in the resist layer after exposure), or features on a semi-latent image (the image in the resist layer after a post-exposure bake step PEB), or features on a developed resist image (where exposed or unexposed parts of the resist have been removed), or even features on an etched image (after a pattern transfer step such as etching). The inspection apparatus IA will typically monitor the features and store values ​​associated with the monitored features in a data set.

[0106] The properties of the patterning process (e.g., the performance of the patterning process) may also be desired to be determined. Such properties may be referred to as process metrics. Process metrics may be difficult to measure directly. In contrast, process metrics may be evaluated indirectly, such as by evaluating the quality of the final pattern present on the substrate. Process metrics may be evaluated by measuring properties representing process metrics, such as yield (e.g., the proportion of fully functional elements on the substrate). A perfectly patterned substrate (where each die includes functional elements) has a yield of 100%. The yield of a substrate where all dies are composed of non-working elements is 0%. Obtaining a yield of 100% or as close to 100% as possible is the goal of high-quality lithographic patterning. The yield can be measured with an inspection device IA or other related measurement equipment and evaluated using a processing device.

[0107] The lithography apparatus includes a processor P or other processing device. The processor P is operable to predict the properties of a patterning process, such as a process metric of a patterning process. Specifically, the processor P accesses a set of models, each of which is configured to predict the above-mentioned process metrics and each of which has a model configuration. The model set (and its corresponding configuration) is stored in a computer storage device. Each model may be provided with data, such as a value associated with a feature measured by the inspection device IA, which is used as an input to the prediction process. That is, the model can use the features measured by the inspection device IA to predict the process metrics. The data may be current data, or may be stored historical data. Additional data may be provided as input to the prediction process, such as operating parameters of the lithography apparatus LA. The details of such a model will not be described in detail here, but an example of a model for predicting process metrics in a semiconductor manufacturing process is described in more detail in the more properties of the model described in WO2018 / 202361, NL2024627, and European patent application EP19186833.0.

[0108] Advantageously, by using a model to predict process metrics, corrections can be calculated and applied proactively (i.e., before the yield is expected to exceed a threshold) rather than retroactively (i.e., in response to the yield exceeding a threshold). As previously discussed, the semiconductor manufacturing process is a complex environment that often experiences drift events. Therefore, the model selection model (e.g., reference Figure 2 ) is used in conjunction with the lithography apparatus LA to select a model configuration to improve the accuracy of the predictions.

[0109] Figure 4 A flow chart of a method 400 for selecting a model configuration so that the prediction of a process metric can be improved is depicted. The purpose of the method is to select a specific model configuration from a set of model configurations for predicting process metrics. Beneficially, by using this method, a model configuration that provides a more accurate prediction of a process metric can be selected. In a first step 402, process data is obtained. The process data includes one or more values ​​associated with a lithography device LA, a substrate W and / or patterned features and one or more values ​​associated with multiple model configurations. The process data may include, for example, a training set, a validation set and / or a test set of data. In a second step 404, a performance indicator is evaluated for each model configuration in the model configuration set M. Specifically, the performance indicator represents the accuracy of each model configuration for the prediction process for which it has been trained. The performance indicator is based on the process data. For example, the performance indicator can be based on a test set that forms at least a part of the process data.

[0110] In a third step 406, a model configuration is selected based on the evaluated performance indication. The selected model configuration may have the 'best' performance indication, such as the highest accuracy. Alternatively, the evaluated performance indication may be provided to an agent that uses historical information and / or an algorithm with the performance indication to select the model configuration. In a fourth step 408, the model with the selected model configuration (i.e., the selected model) is used to predict the process metric. The selected model typically uses at least a portion of the data set D as input for prediction. The predicted process metric is the output of the selected model. In a fifth step 410, the predicted process metric is output by the computer.

[0111] Specific reference is made herein to the use of process data including training sets, validation sets and / or test sets. The terms training set, validation set and test set will be interpreted as the type of data set that is input to the model in accordance with the manner in which they are used in the art. Typically, a training set is used to tune model parameters. Typically, a validation set is used to tune model hyperparameters. Typically, a test set is used to provide a measure of model performance. However, given that the methods herein provide a method for tuning model parameters and model hyperparameters and evaluating the performance of different model configurations, any type of data set may be selected as process data.

[0112] Method 400 can be used with reinforcement learning techniques, such as Figure 2 The model described selects the model 200. In this example implementation, the reviewer 24 obtains (at step 402) process data. The process data includes the state s of the environment at time t+1. t+1 The process data also includes the actions a performed by the agent 22. t The reward r at the previous time t after (e.g. updating the model parameters of the selected model) t (e.g., an indication of prediction accuracy). That is, the reward is a measure of the accuracy of the previously predicted process metric using the previously selected model configuration. The reviewer 24 then evaluates (in step 404) the performance indicator c based on the obtained process data. t , and the performance indication c t Provided to agent 22.

[0113] Given new information (i.e., performance indicator c t ), the agent 22 selects (in step 406) a model configuration. The model with the selected model configuration is then used to predict (in step 408) a process metric, such as a yield. The process metric is output (in step 410). Based on the method 400, the agent 22 learns in real time to select a more optimal model configuration based on the performance indications received from the critics 24. The method 400 is iterative and can be repeated over time. With each repetition of the method 400 (e.g., at time t, t+1, t+2, ...), the agent 22 and the critic 24 can make adjustments based on the evaluated rewards or performance indications and additional data received from the updated environment (e.g., each action (a t 、a t+1 ...) after the state of environment 20 (s t Additionally, with each repetition of method 400, additional values ​​(eg, measured values ​​and / or predicted values ​​of properties) may be added to data set D, thereby increasing the size of data set D.

[0114] When reinforcement learning techniques are used to perform method 400, the method steps can be performed as at least part of a model-free reinforcement learning (MFRL) framework. The MFRL framework can use a policy-based framework that uses a policy function J(θ) to define a selection method for selecting a model configuration. The selection method controls the manner in which the model configuration is selected. The selection method is typically an algorithm that can be based on the total reward throughout the trajectory, where different selection algorithms use different algorithms and / or trajectories of different lengths and / or different weights for different points in the trajectory. A value function V(θ) may also be used. If the current policy function J(θ) is followed until the trajectory is completed (i.e., the end of the policy function), the value function V(θ) defines an enhancement to the prediction process metric. The performance indicator determined by the critic is a gradient ascent (i.e., ) is automatically derived. With each iteration of method 400, the policy function P(s) is updated, thereby 'optimizing' it to select a model configuration that produces a more accurate prediction.

[0115] The critic 24 may be referred to as an evaluation tool. The policy function of the critic 24 is referred to as an evaluation model and is configured to evaluate performance indications and deliver the performance indications to the agent 22. Training the critic 24 may include initializing and / or updating its policy function. The critic 24 may be initially trained using training data, such as data from a dataset D and / or historical data or model parameters from another model selection model. Additionally or alternatively, the critic 24 may be trained in use, receiving process data and optionally other values ​​from the dataset D and values ​​associated with the model configuration (e.g., rewards), and updating its policy function based thereon. This training may be performed repeatedly each time the method 400 is executed.

[0116] The agent may be referred to as a selection tool. The policy function of the agent 22 is referred to as a selection model and is configured to select a model configuration. Training the agent 22 may include initializing and / or updating its policy function. The agent 22 may initially be trained using training data, such as data from a data set D and / or historical data or model parameters from another model selection model. Additionally or alternatively, the agent 22 may be trained in use, through which it receives performance indications, optionally other values ​​from the data set D and values ​​associated with environmental features, and updates its policy function based thereon. The training may be performed repeatedly each time the method 400 is executed.

[0117] The method 400 may be executed by a computer processing device, for example Figure 3A processor P in the computer processing device has access to a memory in which a set of models is stored. Each model in the set of models is configured to predict a process metric (e.g., yield) and has been trained accordingly. Each model has a model configuration, which includes model parameters (e.g., parameters set by training), model hyperparameters (e.g., number of layers, greediness), and optionally a model type (e.g., neural network, SVM, random forest). Each model in the set typically has access to a data set D of values ​​associated with features of a patterning process, a substrate W, and / or a lithography apparatus LA, such as features monitored by an inspection apparatus IA.

[0118] In a specific example, the data set D specifically contains data related to the drift state of the lithographic apparatus LA. Such data may be referred to as drift data. When a drift event occurs, it is particularly advantageous to use a data set D containing a drift state. The drift data may be determined, for example, by monitoring features associated with the substrate and determining a drift event based on changes in monitored overlap measurements. That is, the drift data may be associated with the way in which the features change over time. For example, overlap measurements may be monitored over time, and if the overlap measurements gradually increase over a time period, an incremental population drift event may be identified.

[0119] The drift data may be included in the process data obtained in step 402 of method 400. In the case of a drift event, through its evaluation of the performance indicator based on the process data, critic 24 may encourage the selection (by agent 22) of a specific model configuration that may make predictions with improved accuracy after such a drift event. Specifically, both critic 24 and agent 22 may learn over time that a specific model configuration may produce more accurate predictions after a corresponding specific drift event.

[0120] In another example, the process data may include a content value indicating the number of values ​​contained in the data set D. The content value may indicate the number of values ​​in the data set D as a whole, or may indicate the number of values ​​associated with a particular property, such as values ​​associated with overlapping measurements. Such process data including content values ​​may be particularly advantageous for selecting a model configuration based at least in part on the content value associated with the property. Specifically, some types of models are more prone to overfitting and are therefore generally more accurate when more training data is available. That is, it may be beneficial to use a first model type (e.g., a support vector machine) when the data set D is small, but as the size of the data set D grows (e.g., as more repetitions of method 400 are performed), it may be beneficial to transition to a second model type (e.g., a neural network).

[0121] Selecting a model configuration may include one of a number of actions. In a first example, the model selected at time t (denoted as M t) has the same model as the one previously selected at time t-1 (denoted as M t-1 That is, the most accurate prediction of the process metric is likely to be made by the same model configuration as the previously selected model (M t-1 ) The same model is performed. This first example is more likely to occur in a stable system, where no drift events occur between time t-1 and time t.

[0122] In the second example, the selected model (M t ) and the previously selected model (M t-1 ) has a different model configuration than the previously selected model M. In the second example, selecting a model configuration includes selecting a model configuration with a different model type. For example, if the previously selected model M t-1 is a support vector machine, then selecting a model configuration may include selecting a new model M having a neural network architecture t The second example may be particularly beneficial in systems that use process data including the above-described content values.

[0123] In the third example, the selected model (M t ) and the previously selected model (M t-1 ) has a different model configuration than the new model M. In the third example, selecting the model configuration includes changing the hyperparameters of the selected model. That is, the new model M t It is usually equivalent to the previously selected model M t-1 , but one or more hyperparameters can be changed. For example, when using a reinforcement learning type model, it may be beneficial to reduce the greediness of the model over time.

[0124] In the fourth example, the selected model (M t ) and the previously selected model (M t-1 ). In a fourth example, selecting a model configuration includes changing one or more model parameters. That is, the selected model is retrained to update its model parameters. The fourth example is particularly advantageous after a drift event because the selected model can be retrained using training data associated with the environmental state after the drift event. Therefore, by retraining the model after a drift event, the process metrics can be determined with improved accuracy after the drift event.

[0125] The method 400 provides predicted process metrics as output. In an example embodiment of the method 400, the output process metrics are used to determine corrections to the lithographic apparatus LA and / or the patterning process. The corrections may also be referred to as adjustments. The corrections are calculated to improve the performance of the lithographic apparatus LA and / or the patterning process. Examples of corrections are adjustments to the optical components of the apparatus, adjustments to the dose provided to the lithographic apparatus, changes to the pattern to be applied.

[0126] This correction may then be applied to the apparatus, which may improve its performance. As a result, process metrics may improve, for example the correction may improve yield. In this example, the lithographic apparatus LA comprises a processor P operable to determine a correction and means for applying said correction. It may be chosen to selectively apply the correction, for example if a predicted process metric exceeds a threshold. In the example, it may be beneficial to apply the correction only when the predicted yield is below 80%. Alternatively, it may be beneficial to apply the correction periodically.

[0127] In another example embodiment, the predicted process metrics are used to generate data indicative of the process. For example, the process metrics can be used to generate a map throughout an area associated with the process. A series of predicted process metrics can be generated for the probability of failure of a patterning process at a given location on a semiconductor wafer. Given the series of predicted process metrics, a map showing the probability of failure for the entire wafer can be generated. Alternative maps can be generated, such as an overlap map showing the predicted overlap of multiple layers after the patterning process.

[0128] In another example implementation, the predicted process metrics are provided as input back to the model, for example, to train the model and thereby improve its predictions.

[0129] Other embodiments of the invention are disclosed in the following numbered item list:

[0130] 1. A computer-implemented method of selecting a model configuration from a plurality of model configurations for predicting a process metric associated with a process, the method comprising:

[0131] obtaining process data, the process data comprising values ​​associated with a plurality of model configurations and values ​​associated with process characteristics;

[0132] evaluating a performance indicator for each of a plurality of model configurations based on the process data;

[0133] selecting a model configuration based on the performance indications of the evaluation;

[0134] Predicting process measures using the model with the selected model configuration;

[0135] A process measure predicted by computer output.

[0136] 2. A method according to clause 1, wherein the value associated with the process characteristic includes data associated with the manner in which the characteristic changes over time.

[0137] 3. A method according to clause 1 or 2, wherein each model configuration includes one or more of a model type, a model parameter and / or a model hyperparameter.

[0138] 4. A method according to item 3, wherein the model type includes a support vector machine, a neural network or a random forest.

[0139] 5. A method according to clause 3 or 4, wherein selecting a model configuration includes: retraining the model to update its model parameters.

[0140] 6. A method according to any preceding clause, wherein selecting a model configuration is further based on a data set comprising values ​​associated with process characteristics.

[0141] 7. A method according to clause 6, wherein selecting the model configuration is also based on the size of the data set or the size of a subset of the data set.

[0142] 8. A method according to any preceding clause, wherein the steps of the method are performed by using reinforcement learning.

[0143] 9. A method according to item 8, wherein the steps of obtaining process data, evaluating the performance metric and selecting the first-level model configuration are performed using one or more of the following: an action-review algorithm, an advantage action-review algorithm, an asynchronous advantage action-review algorithm, Q learning with a normalized advantage function, a trust region policy optimization algorithm, a proximal policy optimization algorithm, a double-delayed deep deterministic policy gradient, or a soft action-review algorithm.

[0144] 10. A method according to any of clauses 8 or 9, wherein the steps of obtaining process data and evaluating performance metrics are performed by one or more processors including a review, and the step of selecting a model configuration is performed by one or more processors including an agent.

[0145] 11. A method according to any of clauses 8 to 10, wherein the steps of obtaining process data, evaluating performance metrics and selecting a model configuration comprise:

[0146] determining a strategy function J(θ) that defines a selection method for selecting a model configuration; and / or

[0147] Assuming the policy function is followed until the policy function is completed, a value function V(θ) defining an enhancement of the process metric prediction is determined.

[0148] 12. The method of any preceding clause, further comprising: determining a correction to the process based on the predicted process metrics.

[0149] 13. The method of clause 12, further comprising: applying the correction to the process.

[0150] 14. A method according to any preceding clause, wherein the process comprises a semiconductor manufacturing process.

[0151] 15. A method according to clause 14 when dependent on clause 12 or 13, wherein the correction is determined to adjust optical and / or mechanical parameters of a semiconductor manufacturing device arranged to perform a semiconductor manufacturing process, or to change a processing step of the semiconductor manufacturing process.

[0152] 16. A method according to clause 14 or 15, wherein the value associated with the process characteristic is generated by an inspection device arranged to monitor at least a part of a semiconductor manufacturing process.

[0153] 17. A method of training an evaluation model for evaluating a performance indicator for each of a plurality of model configurations for a model ensemble, the model ensemble having the plurality of model configurations and configured to predict a process metric associated with a process, the method comprising:

[0154] receiving prediction data related to a prediction process metric predicted by at least one model in the set of models;

[0155] receiving a value associated with a process characteristic;

[0156] The evaluation model is trained based on the received values ​​and the received prediction data.

[0157] 18. A method for training a selection model for selecting a model configuration from a plurality of model configurations for a model ensemble, the model ensemble having the plurality of model configurations and configured to predict a process metric associated with a process, the method comprising:

[0158] receiving a value associated with a process characteristic;

[0159] receiving a performance indication indicating performance of a specific prediction process metric predicted using a specific model configuration;

[0160] A selection model is trained based on the received values ​​and the received performance indications.

[0161] 19. A machine learning model trained according to clause 17 or 18.

[0162] 20. A method comprising:

[0163] obtaining one or more models configured to predict a process metric of a manufacturing process based on input process data; and

[0164] Based on inputting new process data to the one or more models, the one or more models and / or model configurations of the one or more models are evaluated using a reinforcement learning framework, and an indication of performance of the one or more models and / or model configurations in predicting a process metric is determined based on the inputting the new process data.

[0165] 21. The method of clause 20, wherein the process metric is associated with a yield or quality parameter of the manufacturing process.

[0166] 22. A method according to clause 20 or 21, wherein the configuration includes one or more of model type, model parameters and / or model hyperparameters.

[0167] 23. A method according to any of clauses 20 to 22, wherein the process data and the new process data comprise features of the manufacturing process.

[0168] 24. A method according to any one of clauses 20 to 23, wherein the manufacturing process comprises a process of patterning a semiconductor wafer.

[0169] 25. A method according to any of clauses 20 to 24, wherein performance in predicting a process metric is based on comparing the predicted characteristic to an otherwise obtained value of the process metric.

[0170] 26. A method according to any of clauses 20 to 25, further comprising the step of selecting a model and / or a model configuration based on the evaluation.

[0171] 27. The method of clause 26, wherein the process metrics indicate a quality of a substrate subjected to a patterning process.

[0172] 28. A method according to any of clauses 20 to 27, wherein the process data comprises key performance indicators (KPIs) associated with a plurality of functional modules involved in the manufacturing process.

[0173] 29. A method according to any one of clauses 20 to 28, wherein the one or more models include at least two machine learning based models, each model having a different architecture.

[0174] 30. A method according to item 29, wherein the one or more models include at least a first model based on supervised learning and a second model based on unsupervised learning.

[0175] 31. A computer program comprising instructions operable when run on suitable apparatus to perform the method of any of clauses 1 to 18 or 20 to 30.

[0176] 32. A computer storage medium comprising the computer program of clause 31.

[0177] 33. A semiconductor manufacturing apparatus comprising a processing device and a storage medium comprising instructions operable to perform the method of any one of clauses 1 to 18 or 20 to 30.

[0178] Although the use of lithographic equipment in the manufacture of ICs may be specifically cited herein, it should be understood that the lithographic equipment described herein may have other applications. Possible other applications include the manufacture of integrated optical systems, guidance and detection patterns for magnetic domain memories, flat panel displays, liquid crystal displays (LCDs), thin film magnetic heads, etc.

[0179] Although embodiments of the present invention may be specifically cited in the context of lithographic equipment herein, embodiments of the present invention may be used in other equipment. Embodiments of the present invention may form a part of any equipment for mask inspection equipment, measurement equipment, or measurement or processing of objects such as wafers (or other substrates) or masks (or other patterning devices). These devices may be generally referred to as lithographic tools. Such lithographic tools may use vacuum conditions or environmental (non-vacuum) conditions. Additionally, the method described herein may be used in non-lithographic equipment. For example, various environments have been described herein, and within these environments, the method may be used to determine more accurate process metrics.

[0180] The methods described herein have specifically referenced yield prediction as a process metric. However, these methods can be used for prediction of any other process metric. In a semiconductor manufacturing environment, such process metrics may include quality, probability of patterning failure, probability of part failure, diagnostic status. In other environments, other process metrics may be predicted, such as the probability of process failure, such as an estimate of misdiagnosis in an in vitro diagnostic tool.

[0181] The methods described herein refer to methods of selecting a model configuration for predicting a process metric associated with a process from a plurality of model configurations. However, in some example embodiments, it may be beneficial to select more than one model configuration. For example, it may be beneficial to use a first model configuration to predict a first process metric and a second model configuration to predict a second process metric. The first process metric and the second process metric may include the same metric (e.g., yield), or may include different metrics (e.g., yield and probability of partial failure).

[0182] The methods described herein may be implemented as instructions in a computer program. The computer program may be executed on a device, such as an inspection device IA, an inspection device or a lithography device LA. Such a computer program may be stored on a computer storage medium.

[0183] Although specific reference may have been made above to the use of embodiments of the present invention in the context of optical lithography, it will be appreciated that the present invention is not limited to optical lithography and may be used in other applications such as imprint lithography, etching, development (e.g., development of resists such as photoresists), and chemical mechanical polishing where the context permits.

[0184] Although specific embodiments of the present invention have been described above, it is to be understood that the present invention may be practiced in a manner other than that described. The above description is intended to be illustrative, not restrictive. Therefore, it will be apparent to those skilled in the art that modifications may be made to the described invention without departing from the scope of the claims set forth below.

Claims

1. A method for predicting a process metric of a manufacturing process, the method comprising: obtaining one or more models configured to predict a process metric of a manufacturing process based on input process data; as well as Based on inputting new process data to the one or more models, the one or more models and / or model configurations of the one or more models are evaluated using a reinforcement learning framework, and based on inputting the new process data, an indication of performance of the one or more models and / or model configurations in predicting the process metric is determined. 2 . The method of claim 1 , wherein the process metric is associated with a yield or quality parameter of the manufacturing process.

3. The method of claim 1, wherein the configuration comprises one or more of a model type, a model parameter, and / or a model hyperparameter. The method of claim 1 , wherein the process data and new process data include characteristics of the manufacturing process. The method according to claim 1 , wherein the manufacturing process is a patterning process of a semiconductor wafer. 6 . The method of claim 1 , wherein the performance in predicting the process metric is based on comparing the predicted process metric to an otherwise obtained value of the process metric.

7. The method of claim 1, further comprising the step of selecting a model and / or a model configuration based on the evaluation.

8. The method of claim 7, wherein the process metrics are indicative of a quality of a substrate subjected to the manufacturing process, the manufacturing process being a patterning process on a semiconductor wafer.

9. The method of claim 1, wherein the process data comprises: Key performance indicators (KPIs) associated with multiple functional modules involved in the manufacturing process.

10. The method of claim 1, wherein the one or more models include at least two machine learning based models, each model having a different architecture.

11. The method according to claim 10, wherein the one or more models include at least a first model based on supervised learning and a second model based on unsupervised learning.

12. A computer program comprising instructions operable, when executed on a suitable device, to perform the method of claim 1.

13. A computer storage medium comprising a computer program according to claim 12.

14. A semiconductor manufacturing apparatus comprising a processing device and a storage medium comprising instructions operable to perform the method according to claim 1.

Images