Systems and methods for perceptual process control of process metrics

Abstract

This document describes a method that includes: determining a sequence of states of an object, where the states are determined based on processing information associated with the object, and where the sequence of states includes one or more future states of the object; determining a process metric associated with the object based on at least one state within the sequence of states and the one or more future states, where the process metric includes an indication of whether processing requirements for an individual state within the sequence of states are met; and initiating an adjustment to processing based on (1) at least one state within the sequence of states and the one or more future states and (2) the process metric, where the adjustment is configured to enhance the process metric for the individual state within the sequence of states such that final processing requirements for the object are met.

Description

[0001] Cross-reference to related applications

[0002] This application claims priority to European Application No. 20159192.2, filed on February 25, 2020, the entire content of which is incorporated herein by reference. Technical field

[0003] The description herein relates to systems and methods for perceptual process control of process metrics. Background art

[0004] Integrated circuit manufacturing can include a variety of processes such as lithography, etching, deposition, chemical mechanical polishing, ion implantation, and / or other operations. Individual operations can produce parts that meet the manufacturing specifications or parts that are rejected for not meeting these specifications. For example, corrections to manufacturing operations can be made based on whether the part meets the manufacturing specifications.

[0005] A lithographic projection apparatus can be used, for example, in the manufacture of integrated circuits (ICs). In such a case, a patterning device (e.g., a mask) can contain or provide a pattern corresponding to a single layer of the IC ("design layout"), and such a pattern can be transferred onto a target portion (e.g., including one or more dies) on a substrate (e.g., a silicon wafer) by a method such as irradiating the target portion through the pattern on the patterning device, where the target portion has been coated with a layer of radiation-sensitive material ("resist"). Typically, a single substrate contains a plurality of adjacent target portions, and the pattern is transferred successively, one target portion at a time, to the plurality of adjacent target portions by the lithographic projection apparatus. In this type of lithographic projection apparatus, the pattern on the entire patterning device is transferred onto one target portion in one operation. Such a device is commonly referred to as a stepper. In an alternative device, commonly referred to as a step-and-scan device, the projection beam scans over the entire patterning device in a given reference direction ("scan" direction), while the substrate is moved synchronously parallel or anti-parallel to this reference direction. Different portions of the pattern on the patterning device are transferred stepwise onto one target portion. Typically, since the lithographic projection apparatus will have a reduction ratio M (e.g., 4), the speed F of the substrate movement will be 1 / M times the speed at which the projection beam scans the patterning device. More information about lithographic apparatus as described herein can be gathered, for example, from US 6,046,792, which is incorporated herein by reference.

[0006] Before transferring a pattern from a pattern forming device to a substrate, the substrate may undergo various processes such as priming, resist coating, and soft baking. After exposure, the substrate may be subjected to other processes ("post-exposure processes") such as post-exposure bake (PEB), development, hard bake, and measurement / inspection of the transferred pattern. This array of processes serves as the basis for manufacturing a single layer of a device (e.g., an IC). The substrate may then undergo a variety of processes such as etching, ion implantation (doping), metallization, oxidation, chemical mechanical polishing, etc., all of which are intended to finish a single layer of the device. If several layers are required in the device, the entire process or a variant thereof is repeated for each layer. Eventually, there will be devices in each target portion on the substrate. These devices are then separated from each other by techniques such as dicing or sawing. The individual devices may be mounted on carriers, connected to pins, etc.

[0007] Manufacturing semiconductor devices generally involves processing a substrate (e.g., a semiconductor wafer) using several manufacturing processes to form individual features and multiple layers of the device. Deposition, lithography, etching, chemical mechanical polishing, and ion implantation are typically used to fabricate and process these layers and features. Multiple devices may be fabricated on multiple die on the substrate and then the devices are separated into individual devices. This device manufacturing process can be considered a patterning process. The patterning process involves a patterning step (such as optical and / or nanoimprint lithography) using a pattern forming device in a lithography apparatus to transfer the pattern on the pattern forming device to the substrate, but typically the patterning process optionally involves one or more associated pattern processing steps such as resist development by a development apparatus, baking the substrate using a baking tool, etching using the pattern with an etching apparatus, etc. One or more metrology processes are also typically involved in the patterning process.

[0008] As mentioned, lithography is a central step in the manufacture of devices such as ICs, where the patterns formed on the substrate define the functional elements of the device, such as microprocessors, memory chips, etc. Similar lithography techniques are also used to form flat panel displays, microelectromechanical systems (MEMS), and other devices.

[0009] As semiconductor manufacturing processes have continued to progress, over the decades, the sizes of the functional elements have been continuously decreasing, while the number of functional elements such as transistors per device has been steadily increasing, following a trend commonly known as "Moore's Law". In the current state of the art, layers of a device are fabricated using a lithographic projection apparatus that uses irradiation from a deep ultraviolet radiation source and / or an extreme ultraviolet radiation source to project a design layout onto the substrate, thereby creating individual functional elements with dimensions far below 100 nm, i.e., less than half the wavelength of the radiation from the radiation source.

[0010] According to the resolution formula CD = k1 × λ / NA, where λ is the wavelength of the radiation used (currently 248 nm or 193 nm in most cases), NA is the numerical aperture of the projection optics in the lithographic projection apparatus, CD is the "critical dimension" (usually the smallest feature size printed), and k1 is an empirical resolution factor. Such a process of printing features smaller than the classical resolution limit of the lithographic projection apparatus is generally referred to as low-k1 lithography. Generally, the smaller k1 is, the more difficult it becomes to reproduce on the substrate a pattern similar in shape and size to that planned by the designer in order to achieve a specific electrical functionality and performance. To overcome these difficulties, complex fine-tuning steps are applied to the lithographic projection apparatus, the design layout, or the patterning device. These steps include, for example but not limited to, optimization of NA and optical coherence settings, customized illumination schemes, use of phase-shifting patterning devices, optical proximity correction (OPC, sometimes also referred to as "optical and process correction") in the design layout, or other methods generally defined as "resolution enhancement techniques" (RET). Summary of the Invention

[0011] The correction and / or other adjustments to a layer on a wafer are typically based on information from the immediately previous layer of the wafer. Generally, there is no yield optimization across the (wafer) stack. Advantageously, the present system and method use model-free reinforcement learning to determine the relationship between the sequence of states of a wafer and the final yield of the wafer, and to make corrections and / or other adjustments that optimize the yield and / or processing time and / or cost.

[0012] According to an embodiment, there is provided a semiconductor processing method. The method includes determining, by one or more processors, a sequence of states of an object being processed. The states are determined based on processing information associated with the object, and the sequence of states includes one or more future states of the object. The method includes determining, by the one or more processors, a process metric associated with the object based on at least one of the states and the one or more future states within the sequence of states. The process metric includes an indication of whether the processing requirements for the object are met for an individual state within the sequence of states. The method includes initiating, by the one or more processors, an adjustment to the processing based on (1) at least one of the states and the one or more future states within the sequence of states and (2) the process metric. The adjustment is configured to enhance the process metric for the individual state within the sequence of states such that the final processing requirements for the object are met.

[0013] In an embodiment, the sequence of states corresponds to a sequence of processing operations performed on the target. Determining the sequence of states, determining the process metric, and initiating the adjustment includes: determining a policy function P(s), which defines or specifies a processing operation correction for an individual state, equipment for performing the processing operation, and / or one or more process parameters for the processing operation; and / or determining a value function V(s), which defines or specifies the enhancement of the process metric assuming that the policy function is followed until the completion of the sequence of processing operations.

[0014] In an embodiment, the value function defines or specifies an expected process metric for a given state (s).

[0015] In an embodiment, the method is performed for a semiconductor processing environment, and the object being processed is a semiconductor wafer or one or more portions of the semiconductor wafer.

[0016] In an embodiment, the process metric includes one or more of the following: yield, cost of sensor measurements, production volume, indication of a trade-off between yield optimization and measurement density, cost of overlay measurements, or overlay.

[0017] In an embodiment, the process metric includes a reward, and the one or more processors include an agent.

[0018] In an embodiment, the process metric includes yield, and enhancing the process metric for the individual state in the sequence of states such that a final processing requirement for the object is met includes increasing the yield.

[0019] In an embodiment, the process metric and / or the adjustment is determined based on at least two states within the sequence of states.

[0020] In an embodiment, the process metric and / or the adjustment is determined based on multiple states within the sequence of states.

[0021] In an embodiment, initiating the adjustment includes: (1) optimizing the process metric based on the sequence of states and determining the adjustment based on the optimized process metric; and / or (2) prompting a user to make the adjustment.

[0022] In an embodiment, the adjustment includes a correction.

[0023] In an embodiment, the correction is an exposure correction associated with the semiconductor processing process.

[0024] In an embodiment, the adjustment includes where, when, and / or how to measure an indication of the object during one or more processing operations.

[0025] In an embodiment, the sequence of states corresponds to the sequence of processing operations performed on the object, and the adjustment includes one or more of the following: a change in the processing operations performed, a change in the order in which the processing operations are performed, or a change in one or more pieces of equipment used to perform one or more of the processing operations.

[0026] In an embodiment, the adjustment includes a change in one or more process parameters of the one or more processing operations.

[0027] In an embodiment, the one or more process parameters include one or more of the following: dose, focal length, mask design, exposure level, one or more etching parameters, one or more deposition parameters, or one or more measurement parameters.

[0028] In an embodiment, the sequence of states corresponds to the sequence of processing operations performed on the object, and the processing information includes one or more of the following: the value of a measurement of the object performed as part of the processing operation, an indication of which processing operations are performed, an indication of the order of the sequence of processing operations, an indication of which equipment and / or associated machine constants are used in the processing operation, or the processing parameters of the processing operation.

[0029] In an embodiment, determining the sequence of states, determining the process metrics, and initiating the adjustment are performed as at least part of a model-free reinforcement learning (MFRL) framework.

[0030] In an embodiment, the MFRL framework includes one or more of the following: Asynchronous Advantage Actor-Critic algorithm, Q-learning with a normalized advantage function, Trust Region Policy Optimization algorithm, Proximal Policy Optimization algorithm, Twin Delayed Deep Deterministic Policy Gradient, or Soft Actor-Critic algorithm.

[0031] In an embodiment, the method further includes: using the one or more processors to compare a first sequence with a second sequence based on a policy function and a value function associated with a first sequence of one or more processing operations having first process parameters and a second sequence of one or more processing operations having second process parameters.

[0032] In an embodiment, the method further includes: performing, as part of a servo operation phase, determining the sequence of the states, determining the process metrics, and initiating the adjustment; and training the policy function and / or the value function during a training operation phase prior to the servo phase.

[0033] In an embodiment, the training operation phase is performed in a simulated semiconductor processing environment.

[0034] According to another embodiment, there is provided a non-transitory computer-readable medium having instructions thereon that, when executed by a computer, implement the method of any of the embodiments described above.

[0035] According to another embodiment, there is provided a non-transitory computer-readable medium having instructions thereon that, when executed by a computer, cause the computer to perform the following: determining a sequence of states of an object being processed, the states being determined based on processing information associated with the object; determining a process metric associated with the object based on at least one of the states within the sequence of states, the process metric including an indication of whether a processing requirement for the object is satisfied for an individual state within the sequence of states; and initiating an adjustment to a processing process based on (1) at least one of the states within the sequence of states and (2) the process metric, the adjustment being configured to enhance the process metric for the individual state within the sequence of states such that a final processing requirement for the object is satisfied.

[0036] In an embodiment, the sequence of states corresponds to a sequence of processing operations performed on the object, and determining the sequence of the states, determining the process metric, and initiating the adjustment include: determining a policy function P(s) that defines or specifies a processing operation correction for an individual state, an apparatus for performing the processing operation, and / or one or more process parameters for the processing operation; and / or determining a value function V(s) that defines or specifies the enhancement of the process metric assuming that the policy function is followed until completion of the sequence of processing operations.

[0037] In an embodiment, the value function defines or specifies an expected process metric for a given state (s).

[0038] In an embodiment, the computer is associated with a semiconductor processing environment, and the object being processed is a semiconductor wafer or one or more portions of the semiconductor wafer.

[0039] In an embodiment, the process metric includes one or more of the following: yield, cost of sensor measurement, production volume, indication of a trade-off between yield optimization and measurement density, cost of overlapping measurement, or overlap.

[0040] In an embodiment, the process metric includes a reward and the computer includes an agent.

[0041] In an embodiment, the process metric includes yield, and enhancing the process metric for the individual states in the sequence of the states such that a final processing requirement for the object is met includes increasing the yield.

[0042] In an embodiment, the process metric and / or the adjustment is determined based on at least two of the states within the sequence of the states.

[0043] In an embodiment, the process metric and / or the adjustment is determined based on a plurality of the states within the sequence of the states.

[0044] In an embodiment, starting the adjustment includes: (1) optimizing the process metric based on the sequence of the states and determining the adjustment based on the optimized process metric; and / or (2) prompting a user to make the adjustment.

[0045] In an embodiment, the adjustment includes a correction.

[0046] In an embodiment, the correction is an exposure correction associated with a semiconductor processing procedure.

[0047] In an embodiment, the adjustment includes indication of where, when, and / or how to measure the object during one or more processing operations.

[0048] In an embodiment, the sequence of the states corresponds to a sequence of processing operations performed on the object, and the adjustment includes one or more of the following: a change in the processing operation performed, a change in the order of performing the processing operation, or a change in one or more pieces of equipment for performing one or more of the processing operations.

[0049] In an embodiment, the adjustment includes a change in one or more process parameters of the one or more processing operations.

[0050] In an embodiment, the one or more process parameters include one or more of the following: dose, focal length, mask design, exposure level, one or more etching parameters, one or more deposition parameters, or one or more measurement parameters.

[0051] In an embodiment, the sequence of states corresponds to a sequence of processing operations performed on the object, and the processing information includes one or more of the following: a value of a measurement of the object performed as part of the processing operation, an indication of which processing operations are performed, an indication of the order of the sequence of processing operations, an indication of which equipment and / or associated machine constants are used in the processing operation, or processing parameters of the processing operation.

[0052] In an embodiment, determining the sequence of states, determining the process metric, and initiating the adjustment are performed as at least part of a model-free reinforcement learning (MFRL) framework.

[0053] In an embodiment, the MFRL framework includes one or more of the following: an asynchronous advantage actor-critic algorithm, Q-learning with a normalized advantage function, a trust region policy optimization algorithm, a proximal policy optimization algorithm, a twin-delayed deep deterministic policy gradient, or a soft actor-critic algorithm.

[0054] In an embodiment, the sequence of states includes one or more future states of the object, and determining the process metric is based on the sequence of states including the one or more future states; and initiating the adjustment to the processing process is based on the sequence of states including the one or more future states and the process metric.

[0055] In an embodiment, the instructions further cause the computer to compare the first sequence with the second sequence based on a policy function and a value function associated with a first sequence of one or more processing operations having first process parameters and a second sequence of one or more processing operations having second process parameters.

[0056] In an embodiment, the instructions further cause the computer to: perform determining the sequence of states, determining the process metric, and initiating the adjustment as part of a servo operation phase; and train the policy function and / or the value function during a training operation phase prior to the servo phase.

[0057] In an embodiment, the training operation phase is performed in a simulated semiconductor processing environment.

[0058] According to another embodiment, a lithographic apparatus is provided. The apparatus includes: an illumination source and a projection optics configured to image a pattern onto a substrate; and one or more processors configured by machine-readable instructions to perform the following operations: determining a sequence of states of an object being processed, the states being determined based on process information associated with the object; determining a process metric associated with the object based on at least one of the states within the sequence of states, the process metric including an indication of whether a processing requirement for the object is met for an individual state within the sequence of states; and starting an adjustment to a processing process based on (1) at least one of the states within the sequence of states and (2) the process metric, the adjustment being configured to enhance the process metric for the individual state within the sequence of states such that a final processing requirement for the object is met.

[0059] In an embodiment, the adjustment includes a change in a process parameter associated with the illumination source, the projection optics, the pattern, and / or the substrate.

[0060] In an embodiment, the processing process is a semiconductor processing process and the object being processed is a semiconductor wafer or one or more portions of the semiconductor wafer.

[0061] In an embodiment, the process metric includes a yield, and meeting the processing requirement includes increasing the yield.

[0062] In an embodiment, starting the adjustment includes optimizing the process metric based on the sequence of states and determining the adjustment based on the optimized process metric.

[0063] In an embodiment, the adjustment is an exposure correction associated with the lithographic apparatus.

[0064] In an embodiment, determining the sequence of states, determining the process metric, and starting the adjustment are performed as at least a part of a model-free reinforcement learning (MFRL) framework.

[0065] In an embodiment, the sequence of states includes one or more future states of the object, determining the process metric is based on the sequence of states including the one or more future states; and starting the adjustment to the processing process is based on the sequence of states including the one or more future states and the process metric.

[0066] In an embodiment, the sequence of states corresponds to a sequence of processing operations performed on the object, meeting the processing requirements includes enhancing the process metric, and starting the adjustment includes: determining a policy function P(s), which defines or specifies the processing operation correction for individual states, the equipment for performing the processing operation, and / or one or more process parameters for the processing operation; and / or determining a value function V(s), which defines or specifies the enhancement of the process metric assuming that the policy function is followed until the sequence of processing operations is completed.

[0067] According to another embodiment, a semiconductor processing method is provided. The method includes determining, using one or more entity processors, a sequence of states of an object being processed. The states are determined based on processing information associated with the object. The method includes: using the one or more processors, determining a process metric associated with the object based on at least one state within the sequence of states. The process metric indicates the processing quality for individual states within the sequence of states. The method includes: using the one or more processors, starting an adjustment to the processing process based on (1) at least one state within the sequence of states and (2) the process metric. The adjustment is configured to enhance the process metric for the individual states within the sequence of states such that the final processing requirements meet the quality standards. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate one or more embodiments and, together with the description, explain these embodiments. Embodiments of the present invention will now be described by way of example only with reference to the accompanying schematic drawings, in which corresponding reference numerals indicate corresponding parts or components, and in which:

[0069] Figure 1 A block diagram showing the various subsystems of a lithography system according to an embodiment.

[0070] Figure 2 A schematic overview of a lithography unit according to an embodiment is illustrated.

[0071] Figure 3 A schematic representation of overall lithography according to an embodiment is illustrated, which represents the collaboration between three techniques for optimizing semiconductor processing.

[0072] Figure 4 An overview of the operation of the present method for using reinforcement learning for yield-aware process control is illustrated.

[0073] Figure 5The figure illustrates how a decision-making agent according to an embodiment interacts with other elements of a reinforcement learning framework.

[0074] Figure 6 The figure illustrates possible states for different layers of a wafer according to an embodiment.

[0075] Figure 7 The figure illustrates a training operation phase and a servo operation phase of a reinforcement learning framework according to an embodiment.

[0076] Figure 8 The figure illustrates an implementation of a model-free reinforcement learning framework for use in yield-aware overlay control according to an embodiment.

[0077] Figure 9 is a block diagram of an example computer system according to an embodiment.

[0078] Figure 10 is a schematic diagram of a lithographic projection apparatus according to an embodiment.

[0079] Figure 11 is a schematic diagram of another lithographic projection apparatus according to an embodiment.

[0080] Figure 12 is according to an embodiment, Figure 11 a more detailed view of the apparatus in

[0081] Figure 13 is according to an embodiment, Figure 11 and Figure 12 a more detailed view of the source collector module SO of the apparatus of DETAILED DESCRIPTION

[0082] Corrections and / or other adjustments to layers on a wafer are typically based on information from the immediately preceding layer of the wafer. For example, a correction for a subsequent layer can be determined based on a single measurement, such as an overlay associated with the previous layer. Typically, there is no process metric (e.g., yield) optimization across the stack. For example, there is no balance of corrections for variations caused by a particular process and / or tools used during processing of different previous layers in the stack. There is no process adjustment based on whether an earlier processing step (e.g., for one or more layers preceding the immediately preceding layer) has caused the wafer or a portion of the wafer to fail to meet the processing specifications.

[0083] Advantageously, the present system and method use model-free reinforcement learning methods to determine the relationship between a sequence of wafer states and the process metrics of the wafer (e.g., final yield), and make corrections to optimize the process metrics (e.g., yield and / or processing cost). The present system and method include a training phase in which training is performed using a large training data set that includes performance, context, scanner, and yield (or yield proxy) and / or other data. For example, the system and method can then be used to determine an optimal strategy (e.g., a sequence of actions through the stack) for a processed wafer associated with a certain state (e.g., a wafer having a specific processing history, for example), the optimal strategy resulting in the best yield and / or processing process (e.g., in terms of measurement time, materials, etc.) with the lowest cost.

[0084] Although specific reference may be made herein to overlay error correction via an advanced process control (APC) system, the architectures described herein can be applied to other metrology processes such as, but not limited to, alignment processes, focus processes, dose determination, intelligent sampling, and the like.

[0085] Although specific reference may be made herein to the manufacture of ICs, it should be clearly understood that the description herein has many other possible applications. For example, it can be used in the manufacture of integrated optical systems, guiding and detecting patterns for magnetic domain memories, liquid crystal display panels, thin film magnetic heads, and the like. In these alternative applications, those skilled in the art should understand that, in the context of these alternative applications, any use of the terms "reticle", "wafer", or "die" herein should be considered to be interchangeable with the more general terms "mask", "substrate", and "target portion", respectively. Additionally, it should be noted that the systems and methods described herein can have many other possible applications in a number of fields such as, for example, language processing systems, autonomous vehicles, medical imaging and diagnosis, semantic segmentation, denoising, chip design, electronic design automation, and the like. The present system and method can be applied in any field in which model-free reinforcement learning is advantageous.

[0086] In this document, the terms "radiation" and "beam" are used to encompass all types of electromagnetic radiation, including ultraviolet radiation (e.g., having wavelengths of 365 nm, 248 nm, 193 nm, 157 nm, or 126 nm) and extreme ultraviolet radiation (EUV, e.g., having wavelengths in the range of about 5 nm to 100 nm).

[0087] A pattern forming apparatus may include or may form one or more design layouts. A computer-aided design (CAD) process may be utilized to generate the design layout. Such a process is often referred to as electronic design automation (EDA). Most CAD processes follow a set of predefined design rules in order to generate a functional design layout / pattern forming apparatus. These rules are set based on processing and design limitations. For example, the design rules define the space tolerances between devices (such as gates, capacitors, etc.) or interconnects to ensure that the devices or lines do not interact with each other in an undesirable manner. One or more of the design rule limitations may be referred to as “critical dimension” (CD). The critical dimension of a device may be defined as the minimum width of a line or a hole or the minimum space between two lines or two holes. Thus, CD regulates the overall size and density of the designed device. One of the goals in device manufacturing is to faithfully reproduce the original design intent on a substrate (via the pattern forming apparatus).

[0088] As used herein, the term “mask” or “pattern forming device” may be broadly interpreted to mean a general pattern forming device that can be used to impart a patterned cross-section to an incident radiation beam, the patterned cross-section corresponding to a pattern to be created in a target portion of a substrate. In such a context, the term “light valve” may also be used. In addition to classical masks (transmission or reflection; binary, phase-shift, hybrid, etc.), examples of other such pattern forming devices also include programmable mirror arrays. An example of such a device is a matrix-addressable surface having a viscoelastic control layer and a reflective surface. The underlying principle implicit in such a device is (for example): the addressed regions of the reflective surface reflect the incident radiation as diffracted radiation, while the non-addressed regions reflect the incident radiation as non-diffracted radiation. Using an appropriate filter, the non-diffracted radiation can be filtered out from the reflected beam, leaving only the diffracted radiation; in this way, the beam becomes patterned according to the addressing pattern of the matrix-addressable surface. Appropriate electronic means may be used to perform the required matrix addressing. Examples of other such pattern forming devices also include programmable LCD arrays. An example of such a configuration is given in U.S. Patent No. 5,229,872, which is incorporated herein by reference.

[0089] As used herein, the term "projection optics" should be construed broadly to cover various types of optical systems, including (by way of example) refractive optics, reflective optics, apertures, and catadioptric optics. The term "projection optics" may also include components that operate according to any of these design types for jointly or individually guiding, shaping, or controlling a projection radiation beam. The term "projection optics" may include any optical component in a lithographic projection apparatus, regardless of where the optical component is located in the optical path of the lithographic projection apparatus. Projection optics may include optical components for shaping, conditioning, and / or projecting the radiation from the source before it passes through the patterning device, and / or optical components for shaping, conditioning, and / or projecting the radiation after it passes through the patterning device. Projection optics typically excludes or does not include the light source and the patterning device.

[0090] As a brief introduction, Figure 1 FIG. 1 illustrates an exemplary lithographic projection apparatus 10A. The main components are: a radiation source 12A, which may be a deep ultraviolet (DUV) excimer laser source or other type of source, including an extreme ultraviolet (EUV) source (as discussed above, the lithographic projection apparatus itself need not have a radiation source); illumination optics, which, for example, define a partial coherence (denoted as sigma) and may include optics 14A, 16Aa, and 16Ab for shaping the radiation from the source 12A; a patterning device 18A; and transmissive optics 16Ac, which project an image of the patterning device pattern onto a substrate plane 22A. An adjustable filter or aperture 20A at the pupil plane of the projection optics may define the range of beam angles incident on the substrate plane 22A, where the maximum possible angle defines the numerical aperture NA = n sin(Θ max ), where n is the refractive index of the medium between the substrate and the last element of the projection optics, and Θ max is the maximum angle of the beam emerging from the projection optics that can still be incident on the substrate plane 22A.

[0091] In a lithographic projection apparatus, a source provides illumination (i.e., radiation) to a patterning device, and projection optics direct and shape the illumination via the patterning device onto a substrate. The projection optics may include at least some of components 14A, 16Aa, 16Ab, and 16Ac. The aerial image (AI) is the radiation intensity distribution at the substrate level. A resist model can be used to calculate the resist image from the aerial image, and an example of such a situation can be found in U.S. Patent Application Publication No. US 2009-0157630, the entire disclosure of which is hereby incorporated by reference. The resist model is only related to the properties of the resist layer (e.g., the effects of chemical processes occurring during exposure, post-exposure bake (PEB), and development). The optical properties of the lithographic projection apparatus (e.g., the properties of the illumination, patterning device, and projection optics) prescribe or control the aerial image and can be defined in an optical model. Since the patterning device used in the lithographic projection apparatus can be changed, it is desirable to separate the optical properties of the patterning device from the optical properties of the remainder of the lithographic projection apparatus, which at least includes the source and the projection optics. Details of techniques and models for using those techniques and models to apply OPC, transform a design layout into various lithographic images (e.g., aerial images, resist images, etc.), and evaluate performance (e.g., in terms of process window) are described in U.S. Patent Application Publication Nos. US2008-0301620, US2007-0050749, US2007-0031745, US2008-0309897, US2010-0162197, and US2010-0180251, the entire disclosures of each of which are hereby incorporated by reference in their entireties.

[0092] Figure 2 Schematic overview depicting a lithographic cell LC. As Figure 2As shown, the lithographic apparatus LA can form part of a lithographic cell LC, which is sometimes also referred to as a litho cell or (lithographic) cluster and which often also includes equipment for performing pre- and post-exposure processes on a substrate W. Conventionally, such equipment includes a spin coater SC configured to deposit a resist layer, a developer DE for developing the exposed resist, a chill plate CH, and a bake plate BK, the chill plate and the bake plate being used, for example, to adjust the temperature of the substrate W, for example to adjust the solvent in the resist layer. A substrate transfer device or robot RO picks up the substrate W from the input / output ports I / O1, I / O2, moves the substrate W between different process equipment, and transfers the substrate W to the feed table LB of the lithographic apparatus LA. The equipment in the lithographic cell, which is commonly collectively referred to as a track or coat develop system, is typically under the control of a track or coat develop system control unit TCU, which itself can be controlled by a management control system SCS, which can also control the lithographic apparatus LA, for example via a lithography control unit LACU.

[0093] In order to correctly and consistently expose the substrates exposed by the lithographic apparatus LA, it is desirable to inspect the substrates to measure the properties of the patterned structures, such as overlay errors between subsequent layers, line thickness, critical dimension (CD), etc. For this purpose, an inspection tool (not shown) can be included in the lithographic cell LC. If an error is detected, then, for example, the exposure of subsequent substrates or other processing steps to be performed on the substrates can be adjusted, especially in cases where the inspection is performed before the other substrates in the same lot or batch are still to be exposed or processed.

[0094] An inspection device, which can also be referred to as a metrology device, is used to determine the properties of the substrates and how the properties of different substrates vary or how the properties associated with different layers of the same substrate vary between different layers. The inspection device can alternatively be configured to identify defects on the substrates and can, for example, be part of the lithographic cell LC, or can be integrated into the lithographic apparatus LA, or can even be a stand-alone device. The inspection device can measure the properties of a latent image (the image in the resist layer after exposure), or a semi-latent image (the image in the resist layer after a post-exposure bake step PEB), or the developed resist image (where the exposed or unexposed portions of the resist have been removed), or even the image after etching (after a pattern transfer step such as etching).

[0095] Figure 3Schematic representation depicting holistic lithography, showing the collaboration between three techniques for optimizing semiconductor processing. In general, the patterning process in a lithography apparatus LA is one of the most important steps in the processing, which requires high accuracy in the sizing and placement of structures on a substrate. To ensure this high accuracy, three systems (in this example) can be combined in a so-called "holistic" control environment, as Figure 3 schematically depicted in. One of these systems is the lithography apparatus LA, which is (virtually) connected to a metrology device (e.g., metrology tool) MT (the second system), and to a computer system CL (the third system). The "holistic" environment can be configured to optimize the collaboration between these three systems to enhance the overall process window and provide a tight control loop, thereby ensuring that the patterning performed by the lithography apparatus LA remains within the process window. The process window defines the range of process parameters (e.g., dose, focus, overlay), within which a particular processing step produces a defined result (e.g., a functional semiconductor device) - typically, the process parameters in a lithography process or patterning process can be allowed to vary within this range.

[0096] The computer system CL can use (a portion of) the design layout to be patterned to predict which resolution enhancement techniques to use, and perform computational lithography simulations and calculations to determine which mask layout and lithography apparatus settings achieve the maximum overall process window for the patterning process (depicted by the double arrows in the first scale SC1 in Figure 3 ). In general, the resolution enhancement techniques are arranged to match the patterning capabilities of the lithography apparatus LA. The computer system CL can also be used to (e.g., using input from the metrology tool MT) detect where the lithography apparatus LA is currently operating within the process window, to predict whether there may be defects attributable to, for example, sub-optimal processing (depicted by the arrow pointing to "0" in the second scale SC2 in Figure 3 ).

[0097] The metrology device (tool) MT can provide input to the computer system CL for accurate simulations and predictions, and can provide feedback to the lithography apparatus LA to identify possible drifts in, for example, the calibration state of the lithography apparatus LA (depicted by the multiple arrows in the third scale SC3 in Figure 3 ).

[0098] During the lithography process, frequent measurements of the resulting structures are desired, for example for process control and verification. Tools for making such measurements include metrology tools (equipment) MT. Different types of metrology tools MT for making such measurements are well known, including scanning electron microscopes or various forms of scatterometer metrology tools MT. A scatterometer is a versatile instrument that allows the measurement of parameters of the lithography process by having a sensor in the pupil or in a plane conjugate to the pupil of the objective of the scatterometer. The measurement is typically referred to as pupil-based measurement, or allows the measurement of parameters of the lithography process by having a sensor in the image plane or in a plane conjugate to the image plane. In this case, the measurement is typically referred to as image- or field-based measurement. Patent applications US20100328655, US2011102753A1, US20120044470A, US20110249244, US20110026032 or EP1,628,164A, which are incorporated herein by reference in their entirety, further describe such scatterometers and related measurement techniques. For example, the aforementioned scatterometers can use light from soft x-rays and visible light to the near-IR wavelength range to measure features of a substrate, such as gratings.

[0099] In some embodiments, the scatterometer MT is adapted to measure the overlap of two misaligned gratings or periodic structures (and / or other target features of the substrate) by measuring the reflection spectrum and / or detecting asymmetry in the detection configuration, the asymmetry being related to the degree of overlap. The two (usually superimposed) grating structures can be applied in two different layers (not necessarily consecutive layers) and can be formed at substantially the same location on the wafer. The scatterometer can have a symmetry detection configuration as described, for example, in patent application EP1,628,164A, such that any asymmetry is clearly distinguishable. This provides a way to measure misalignment in the grating. Additional examples of measuring overlap can be found in PCT patent application publication number WO2011 / 012624 or US patent application US20160161863, which are incorporated herein by reference in their entirety.

[0100] Other parameters of interest can be focal length and dose. The focal length and dose can be determined simultaneously by scatterometry (or alternatively by scanning electron microscopy) as described in US patent application US2011-0249244, which is incorporated herein by reference in its entirety. A single structure (e.g., a feature in the substrate) can be used that has a unique combination of critical dimension and sidewall angle measurements for each point in a focal length energy matrix (FEM, also known as a focal length exposure matrix). If these unique combinations of critical dimension and sidewall angle can be obtained, the focal length and dose values can be uniquely determined from these measurements.

[0101] It is often desirable to computationally determine how a patterning process will produce a desired pattern on a substrate. The computational determination may include, for example, simulation. The simulation may be provided for one or more parts of the process. For example, it is desirable to be able to simulate a lithography process that transfers a pattern of a patterning device onto a resist layer of a substrate and the pattern produced in the resist layer after development of the resist, to simulate metrology operations (such as the determination of overlay) and / or to perform other simulations. The aim of the simulation may be to accurately predict, for example, metrology metrics (such as overlay, critical dimension, reconstruction of the three-dimensional profile of a feature of the substrate, dose or focus of a lithography apparatus when printing a feature of the substrate using the lithography apparatus, etc.), process parameters (such as edge placement, aerial image intensity tilt, sub-resolution assist features (SRAF), etc.), and / or other information that can subsequently be used to determine whether an expected or target design has been achieved. The expected design is typically defined as a pre-optical proximity correction design layout, which may be provided in a standardized digital file format such as GDSII, OASIS or another file format.

[0102] The simulation can be used to determine one or more metrology metrics (such as overlay and / or other metrology measurements), to configure one or more features of the patterning device pattern (e.g., by simulating optical proximity correction), to configure one or more features of the illumination (e.g., by simulating changes in one or more characteristics of the spatial / angular intensity distribution of the illumination), to configure one or more features of the projection optics (such as numerical aperture, etc.) and / or for other purposes. Such determination and / or configuration may generally be referred to as, for example, mask optimization, source optimization and / or projection optimization. Such optimizations may be performed independently or in different combinations. One such example is source-mask optimization (SMO), which involves configuring one or more features of the patterning device pattern and one or more features of the illumination. The optimization may, for example, use the parameterized models described herein to predict values of various parameters (including images, etc.).

[0103] In some embodiments, the optimization process of a system can be represented as a cost function. The optimization process can include finding a set of parameters (design variables, process variables, etc.) of the system that minimizes the cost function. The cost function can have any suitable form depending on the optimization objective. For example, the cost function can be the weighted root mean square (RMS) of the deviations of certain characteristics (evaluation points) of the system from their expected values (e.g., ideal values). The cost function can also be the maximum value of these deviations (i.e., the worst deviation). The term "evaluation point" should be interpreted broadly to include any characteristic of the system or manufacturing method. Due to the applicability of the implementation of the system and / or method, the design and / or process variables of the system may be limited to a finite range and / or may be interdependent. In the case of a lithographic projection apparatus, the constraints are often associated with the physical properties and characteristics of the hardware (such as the tunable range and / or the design rules for the manufacturability of the patterning device). Evaluation points can include physical points on the resist image on the substrate, as well as non-physical characteristics such as (for example) dose and focus.

[0104] In some embodiments, the present system and method can include one or more processors configured to perform one or more of the operations described herein. The one or more processors can include one or more algorithms and / or other programs configured to simulate and / or otherwise predict an output based on the relationships between various inputs (e.g., one or more characteristics of an electric field image, one or more characteristics of a design layout, one or more characteristics of a patterning device, one or more characteristics of the illumination used in a lithography process, such as wavelength, etc.).

[0105] As an example, the algorithm can be a machine learning algorithm. In some embodiments, the machine learning algorithm can be and / or include mathematical equations, other algorithms, curves, charts, networks (such as neural networks), and / or other tools and machine learning components. For example, the machine learning algorithm can be and / or include one or more neural networks having an input layer, an output layer, and one or more intermediate or hidden layers. In some embodiments, the one or more neural networks can be and / or include deep neural networks (e.g., neural networks having one or more intermediate or hidden layers between the input layer and the output layer).

[0106] As an example, one or more neural networks can be based on a collection of large neural units (or artificial neurons). The one or more neural networks may not strictly mimic the way a biological brain works (e.g., via a large cluster of biological neurons connected by axons). Each neural unit of a neural network can be connected to many other neural units of the neural network. Such connections can strengthen or inhibit their influence on the activation state of the connected neural units. In some embodiments, each individual neural unit can have a summation function that combines the values of all its inputs. In some embodiments, each connection (or the neural unit itself) can have a threshold function such that a signal must exceed the threshold before it is allowed to propagate to other neural units. These neural network systems can be self - learning and trained rather than explicitly programmed and can perform significantly better in certain problem - solving domains compared to traditional computer programs. In some embodiments, one or more neural networks can include multiple layers (e.g., where the signal path crosses from a front - end layer to a back - end layer). In some embodiments, a neural network can utilize backpropagation techniques, where forward stimuli are used to reset the weights of "front - end" neural units. In some embodiments, the stimulation and inhibition of one or more neural networks may be more freely flowing, where the connections interact in a more chaotic and complex manner. In some embodiments, the intermediate layer of one or more neural networks includes one or more convolutional layers, one or more recurrent layers, and / or other layers.

[0107] A training data set can be used to train one or more machine - learning algorithms (i.e., determine their parameters). The training data can include a collection of training samples. Each sample can be a pair including an input object (usually an image, a measurement, a tensor or vector that can be referred to as a feature tensor or vector) and a desired output value (also called a governing signal). The training algorithm analyzes the training data and adjusts the behavior of the algorithm by adjusting the parameters of the algorithm based on the training data. For example, given a set of N training samples in the form of {(x1, y1), (x2, y2),..., (x N , y N )} such that x i is the feature tensor / vector of the i - th example and y i is its governing signal, the training algorithm seeks a result g: X → Y, where X is the input space and Y is the output space. The feature tensor / vector is an n - dimensional tensor / vector representing the numerical features of an object (e.g., a complex electric - field image). The tensor / vector space associated with these vectors is often referred to as the feature or latent space. After training, the algorithm can be used to make predictions using new samples.

[0108] Figure 4The figure illustrates an overview of the operation of the present method 400 for using reinforcement learning for in - process metric (e.g., yield) - aware process control. In semiconductor processing, certain parameters such as overlay (e.g., inter - layer alignment offset), critical dimension (CD), etc. are measured to ensure that the process proceeds according to (meets) the processing requirements. These measurements are used to monitor and control the processing. The results of these measurements for the immediately previous layer, along with additional context data (e.g., which machines and / or other equipment were used, which process parameters were used, etc.), are used in subsequent layers to initiate adjustments to the semiconductor processing for the subsequent layer (e.g., using an advanced process control (APC) system).

[0109] In a given wafer, some layers are more critical than others. Some layers have more stringent processing requirements than the other layers. For example, an implant layer is less critical compared to a via layer, which needs to be aligned more precisely relative to a metal layer. These critical layers require more intensive quality inspection measurements. As another example, a layer including optimal design metrology targets is important for identifying process variations and ultimately producing a working device. However, while the criticality of these and other layers in a given wafer is known, typical adjustments to the layers on the wafer are based only on information from the immediately previous layer of the wafer and do not consider information from additional previous layers (e.g., indicating the dimensions of those previous layers, the alignment of those previous layers, the overlay of those previous layers, whether the wafer or a portion of the wafer has failed to meet processing requirements, and / or other information).

[0110] Among other drawbacks, previous systems lacked yield (and / or other similar in - process metrics) awareness across the stack. Yield is a measure of functional integrated circuits (semiconductor devices). Often, previous systems only attempted to minimize the overlay between two adjacent layers (or meet some other processing requirement). Previous systems did not consider the potential cross - stack effects of overlays (and / or other parameters) on yield. Continuing with this example, it is possible that the processes for the first few layers of a semiconductor device produce regions of the wafer with zero yield, which would then be discarded. In theory, in the next subsequent layer of the wafer, adjustments should be applied such that the zero - yield regions of the wafer are not favored or even ignored. However, previous systems were not configured in this way.

[0111] As another example, prior systems required intensive time-consuming measurements. Intensive measurements were often performed on critical layers, and less intensive measurements were performed on non-critical layers. Measurements were not adjusted based on yield and / or other factors. Measurements were not adjusted based on information from previous layers in a stack. As another example, different lithography scanners and other different process machines and / or equipment including different metrology tools may introduce different fingerprints on a wafer. Certain combinations of machines and / or other process equipment may result in very poor yield performance regardless of applied corrections. Other combinations of machines and / or other process equipment may result in high yields. However, prior systems generally did not adjust for these combinations of previous machines and / or other equipment based on, for example, yield.

[0112] In contrast to prior systems, the present system and method are configured to use reinforcement learning for yield-aware (and / or other process metric) process control. Make data-driven adjustments throughout the stack. The adjustments are based on relevant historical data of wafer measurements and / or other information from more than just the immediately previous layer, the machines and / or process equipment used to fabricate and / or measure those layers, and / or other information. Additionally, the present system is configured to consider the impact of the adjustments on, for example, the overall yield (and / or other process metrics) of a semiconductor device. For example, using the present system and method, exposure correction (as one example of many possible adjustments) is not simply applied to minimize overlay error, CD error, etc. in a particular layer. Rather, exposure correction (for example) is applied by tuning the entire sequence of corrections and scanner combinations (and / or other adjustments) throughout the stack to optimize yield (and / or other process metrics) as the goal. The present system and method are also configured to reduce measurement cost by determining a sequence of corrections (adjustments) that require less intensive measurements without sacrificing yield. Additionally, the present system and method are configured to compare expected scanner-process machine and / or other equipment combinations and facilitate the identification of an optimal set of processing operations for a wafer (including the machines and / or other equipment for those operations). The present system and method utilize model-free reinforcement learning (MFRL) as described herein to achieve these and other advantages.

[0113] In some embodiments, as described herein, method 400 is performed in a semiconductor processing environment (e.g., the "environment" for MFRL as described below). At operation 402, a sequence of states of an object being processed (e.g., a semiconductor wafer and / or other object) is determined. At operation 404, a process metric associated with the object (e.g., yield and / or other process metrics and / or quality criteria) is determined. At operation 406, based on (1) at least one state within the sequence of states and (2) the process metric, an adjustment to the processing is initiated. The adjustment is configured to enhance the process metric for an individual state within the sequence of states such that the final processing requirements for the object are met (e.g., such that the processing results in a working semiconductor device). For example, the process metric may indicate the processing quality for an individual state within the sequence of states. The adjustment may enhance the process metric for an individual state within the sequence of states such that the final processing requirements meet a quality criterion (e.g., a specific yield percentage, etc.).

[0114] The operations of method 400 presented below are illustrative. In some embodiments, method 400 may be implemented with one or more additional operations not described and / or without one or more of the operations discussed. For example, method 400 may include a training operation as described below. Additionally, the order of the operations of method 400 illustrated and described below is not intended to be restrictive. Figure 4 in

[0115] In some embodiments, one or more portions of method 400 may be implemented (e.g., via simulation, etc.) in one or more processing devices (e.g., one or more processors). The one or more processing devices may include one or more devices that execute some or all of the operations of method 400 in response to instructions stored electronically on an electronic storage medium. The one or more processing devices may include one or more devices configured via hardware, firmware, and / or software that is specifically designed to perform, for example, one or more of the operations of method 400. In some embodiments, one or more processors form an "agent" (as described further below) of, for example, the MFRL architecture.

[0116] Operation 402 includes determining a sequence of states of an object being processed. In some embodiments, the object being processed is a semiconductor wafer or one or more portions of a semiconductor wafer and / or other object. For example, one or more portions of a semiconductor wafer may be one or more portions that meet the processing requirements up to the current point in the processing. This may include, for example, individual chips and / or other portions of the wafer.

[0117] In some embodiments, the sequence of states corresponds to a sequence of processing operations performed on the object. For example, a given state can correspond to a wafer before or after a particular lithography operation, etching operation, deposition operation, and / or other operation in a processing sequence. In some embodiments, the sequence of states includes one or more future states of the object. For example, a future state can include a wafer that has undergone one or more additional lithography operations, etching operations, deposition operations, and / or other operations up to and including a surface finishing operation for completing the fabrication of a semiconductor device.

[0118] The state is determined based on processing information and / or other information associated with the object. In some embodiments, the processing information includes values of measurements taken on the object as part of a processing operation, an indication of which processing operations were performed, an indication of the order or sequence of processing operations, an indication of which machines and / or other equipment were used in the processing operation, constants of such machines and / or other equipment, processing parameters of the processing operation, and / or other processing information. For example, the processing information can include an indication of which metrology equipment was used to measure overlay and the overlay value itself.

[0119] Operation 404 includes determining a process metric associated with the object based on at least one state within the sequence of states. In some embodiments, the process metric is determined based on two or more states within the sequence of states, a plurality of states within the sequence of states, or all of the states within the sequence of states. In some embodiments, the process metric is determined based on the sequence of states that includes the one or more future states.

[0120] The process metric includes an indication of whether the processing requirements for the object are met for individual states within the sequence of states. The process metric can indicate the processing quality for individual states within the sequence of states. For example, the process metric can be and / or relate to a particular quality standard, such as a particular yield percentage and / or other standard. In some embodiments, the process metric includes yield, cost of sensor measurements, production volume, an indication of a trade-off between yield optimization and measurement density, cost of overlay measurements, overlay, and / or other process metrics. In some embodiments, the process metric includes yield. In these embodiments, enhancing the process metric for individual states within the sequence of states such that the final processing requirements for the object are met (e.g., and / or such that the final processing requirements meet a quality standard) includes increasing the yield. The process metric and / or enhancement of the process metric can be, for example, a "reward" in MFRL (as described below).

[0121] Operation 406 includes initiating an adjustment to a processing procedure. The adjustment is initiated based on at least one of the states within the sequence of states, process metrics, and / or other information. In some embodiments, the adjustment to the processing procedure is initiated based on the sequence of states and process metrics, the sequence of states including one or more future states. The adjustment is configured to enhance the process metrics for individual states within the sequence of states such that the final processing requirements of the object are met (e.g., and / or such that the final processing requirements meet quality criteria). In some embodiments, initiating the adjustment includes optimizing the process metrics based on the sequence of states and determining the adjustment based on the optimized process metrics. The adjustment can be a change in processing procedure parameters, a correction to the condition of the processing procedure, a change to the processing operation itself, and / or other adjustments.

[0122] In some embodiments, the change in processing procedure parameters can be an increase or decrease in a process parameter, a different value of a process parameter, and / or some other change to a process parameter. In some embodiments, one or more process parameters include dose, focal length, master design, exposure level, one or more etch parameters, one or more parameters associated with the adjustment of the edges of a device structure (such as in edge placement control), one or more deposition parameters, one or more measurement parameters, and / or other parameters. For example, in some embodiments, the adjustment can include a change in process parameters associated with an illumination source, projection optics, pattern, substrate, and / or other aspects of a semiconductor and / or semiconductor processing procedure.

[0123] In some embodiments, the adjustment includes a correction. In some embodiments, the correction is an exposure correction and / or other correction associated with a semiconductor processing procedure. For example, other corrections can include a correction to a mask or a pattern on a mask, a correction to an alignment option, a correction to an overlay metrology option, a correction to an automatic process control (APC) option, an APC correction applied to the equipment by, for example, adjusting mask alignment, projection optics settings, illumination source settings, stage positioning settings, a correction to a manufacturing or fab equipment wiring and matching option, a correction to an equipment maintenance option, and / or other corrections.

[0124] In some embodiments, the adjustment includes where, when, and / or how to measure an indication of the object during one or more processing operations. In some embodiments, the adjustment includes a change in the processing operations performed on the object, a change in the order in which the processing operations are performed, a change in one or more machines and / or other pieces of equipment used to perform one or more of the processing operations, and / or other adjustments.

[0125] In some embodiments, initiating the adjustment includes prompting the user to make the adjustment. Prompting the user to make the adjustment can include providing the user with a message and / or other indication of the adjustment. The message and / or other indication can be provided on a user interface of a computing device associated with the user, a user interface of a computing device associated with the processing operation, and / or other interfaces. In some embodiments, prompting the user to make the adjustment includes facilitating typing and / or selection and / or other prompting of the adjustment by the user via the user interface.

[0126] As described above, determining the sequence of states (operation 402), determining the process metrics (operation 404), and initiating the adjustment (operation 406) are performed as at least part of a model-free reinforcement learning (MFRL) framework. The MFRL framework includes machine learning algorithms configured for sequential decision-making. The decisions (e.g., actions and / or other adjustments) made in a given situation (or state) are optimized to maximize the reward (e.g., enhancement of yield and / or other process metrics). The basic elements of the MFRL framework include a set of states (e.g., as described above), a set of actions (e.g., the adjustments as described above), an agent (e.g., one or more processors as described above), an environment (e.g., the semiconductor processing process as described above), and a reward (e.g., the enhanced yield and / or other process metric enhancements as described above).

[0127] A given state can include relevant historical data (processing information) and / or other information for a particular wafer. The historical data (processing information) can include measured overlaps in previous layers, context data used on previous and current layers (e.g., scanners, masks, process machines, and / or other equipment, etc.), and / or other information (including any additional processing information as described above). The actions (adjustments) include corrections and / or other adjustments that can be implemented on the processing machine and / or in the processing process, in associated electronic models, in product designs, in metrology target layouts, etc. Note that this can be machine-dependent. The agent is a decision-making architecture that determines the adjustments and actuates these adjustments during the processing. In the case of yield-aware process adjustment, the reward is the enhanced yield accumulated at the end of the processing.

[0128] Optionally, a negative reward can be accumulated for a given state for a set of adjustments that are expensive to apply (e.g., in terms of time, yield, or other process metrics), where higher-order electronic models require more measurements and thus incur the cost of implementing such expensive corrections. For example, this situation requires a trade-off between optimizing the yield (and / or other process metrics) and minimizing intensive measurements.

[0129] The environment (e.g., semiconductor processing) consists of all possible states and all transitions between the states. That is, whenever an action (e.g., adjustment) is taken for a certain state, the wafer moves to a new state, and rewards (and / or costs) are accumulated. The environment (individual states and transitions between states) can be learned from training data using machine learning (e.g., as described herein) and / or by other methods.

[0130] Figure 5 Illustrates how a decision-making agent interacts with other elements of a reinforcement learning framework. As Figure 5 shown, the agent 500 (e.g., one or more processors) determines a given action (e.g., adjustment) 502 for a given environment (e.g., semiconductor processing) 504 based on the current state 506 and the corresponding reward 508. This process can be iteratively repeated 510 as necessary. In some embodiments, such an arrangement includes a Markov decision process. Since the transitions between states are not known (e.g., predetermined) in the semiconductor processing context (e.g., a wafer can be processed using any of multiple different possible processing operations), the decision-making process by the agent 500 includes model-free reinforcement learning.

[0131] By way of non-limiting example, Figure 6 illustrates the possible states 600 (L1-1, L1-2,... LM-m) for different layers 602 (1, 2,..., M) of a wafer 604. The arrows 606 represent possible transitions between states that depend on the action (e.g., adjustment) taken (performed) by the agent (e.g., the agent 500 shown in n ) which is one or more processors as described above). The yield (e.g., reward) is determined for the final wafer 604 and / or for each state. It should be noted that the yield is used as one possible example of many process metrics. Depending on the wafer calibration model, the state can represent, for example, the overlap error between the current layer and the previous layer of the wafer. For example, other actions (e.g., causing transitions between states) can include performing different types of scanner or process tool adjustments. Figure 5

[0132] Algorithms for solving the model-free reinforcement learning problem are trial-and-error algorithms and / or other algorithms. In some embodiments, the MFRL framework includes one or more of the following: asynchronous advantage actor-critic algorithm, Q-learning with a normalized advantage function, trust region policy optimization algorithm, proximal policy optimization algorithm, twin-delayed deep deterministic policy gradient, flexible actor-critic algorithm, and / or other algorithms. In some embodiments, the process metrics as described above include rewards, and one or more processors include an agent, where the environment is a semiconductor processing process. ​

[0133] Return to Figure 4 In some embodiments, determining the sequence of the states (operation 402), determining process metrics (operation 404), and initiating an adjustment (operation 406) includes determining a policy function P(s), which defines or specifies process operation corrections for individual states, the machine and / or other equipment used to perform the process operations, one or more process parameters for the process operations, and / or other information. Performing these operations also includes determining a value function V(s), which defines or specifies the enhancement of the process metrics assuming that the policy function is followed until the completion of the process operation sequence. In some embodiments, the value function defines or specifies the expected process metrics for a given state (s).

[0134] The present system and method are configured to optimize process metrics such as yield. Accordingly, the adjustment for a single layer (e.g., correction, process parameter change, process operation sequence adjustment, etc.) is determined based on the impact the adjustment has on the process metrics (e.g., yield) at the end of the semiconductor process. In other words, the present system and method are not only configured to determine and / or perform adjustments to a single layer, but these adjustments follow a strategy for the entire wafer (e.g., a sequence of adjustments and / or other actions).

[0135] The present system and method are configured such that an individual adjustment incurs a reward (e.g., enhancement of a process metric such as yield) or a cost (e.g., a negative reward such as decreased yield, increased processing time, increased measurement density, etc.). This situation promotes consideration of the cost of an "expensive" adjustment (e.g., requiring more intensive measurements, additional processes, etc.). This introduces a trade-off between maximizing process metrics such as yield and minimizing the cost (e.g., measurement). The output policy (adjustment sequence) is configured to obtain an enhanced process metric (e.g., the highest possible yield) without breaching (e.g., exceeding) a target (e.g., measurement) budget (e.g., reducing the measurement cost without compromising the yield).

[0136] As a brief overview, the present system and method are configured to use the MFRL framework to determine the type function V(s) and the policy function P(s). P(s) specifies the adjustment for an individual state s. V(s) indicates the (e.g., average) reward (e.g., process metric enhancement) that can be obtained if the starting point is state s and the policy P(s) is followed until the completion of the semiconductor process. P(s) may represent an adjustment, including corrections and changes to the process parameters, machine, and / or other equipment used to manufacture the wafer, corrections and changes to the operations and / or the sequence of operations used to manufacture the wafer, and / or other adjustments as discussed above. And V(s) defines the individual (e.g., "optimal") correction to be applied in an individual layer.

[0137] In some embodiments, operation 406( Figure 4 ) includes comparing a first sequence of one or more processing operations having a first operation, process parameters, a machine, and / or other equipment, etc. (e.g., a first policy) with a second sequence of one or more processing operations having a second operation, process parameters, a machine, and / or other equipment, etc. (e.g., a second policy). The comparison is based on a policy function and a value function associated with the first sequence and the second sequence. Operation 406 includes, for example, selecting the first sequence or the second sequence based on which sequence results in optimized process metrics. As described above, the process metric can be the yield, such that operation 406 includes selecting the first sequence or the second sequence based on which sequence results in a better yield.

[0138] In some embodiments, determining a sequence of states, determining process metrics, and initiating adjustments are performed as part of a serving operational phase. For example, the serving phase can include using the policy function and the value function in actual processing and / or other uses. Before the serving phase, the policy function and the value function can be generated, and / or a machine learning algorithm that generates the value function and / or the policy function can be trained during a training operation phase. During the training phase, the system performs exploratory actions and thus uses stochastic rewards and observations (“explorations”) to learn the policy function and the value function. In some embodiments, the training operation phase is performed in a simulated semiconductor processing environment and / or using simulated semiconductor processing data. In some embodiments, the training operation phase is performed using actual measurement data. In some embodiments, the training operation phase is performed using both simulated and actual measurement data and / or other information. For example, in the serving phase, the learned policy function and value function are used to generate new actions.

[0139] For example, before using the serving initial policy function and / or value function as described above, training data can be used to generate and / or train the algorithm. The training data includes process and corresponding performance data and can be associated with one or more different processing processes. The process data and the corresponding performance data can include data for lithography and / or processing processes and / or process simulations, such as data associated with, regarding, and / or representing several processing operations described herein (e.g., see Figures 1 to 3) data and / or other data. Training the initial algorithm can include providing training data as input to the initial algorithm (e.g., as described above). The initial prediction algorithm can operate to learn to better predict performance data based on the corresponding process data. For example, learning to better predict performance can include iteratively updating one or more of the algorithm parameters (e.g., before or after servoing), and determining whether the update results in a better or worse prediction of known performance data.

[0140] By way of non-limiting example, Figure 7 Illustrates training phase 700 and servo phase 702. Both phases 700 and 702 show time series 704 and 706 with corresponding observations 708 (e.g., measurements on a given layer), actions 710 (e.g., one or more different corrections and / or other adjustments for individual layers indicated by arrows), rewards 712 (e.g., wafer yield enhancement), and different states 714 of wafer 716. Figure 7 Illustrates a sequence of adjustments (e.g., actions such as corrections) that can be taken by an agent in an MFRL framework for use in yield-aware overlay control. During training phase 700, the agent (e.g., the 500 shown in Figure 5 learns a policy function and / or a value function. During servo phase 702, the agent executes the optimal policy. Arrows represent actions taken during the semiconductor wafer processing. Dashed lines represent other possible actions that are not taken. The arrow for the (M - 1)th action indicates a probing action that can still be taken even if another action is considered a better action. Curved arrows represent updates to the policy and / or value function.

[0141] As a second non-limiting example, Figure 8 Illustrates an implementation of an MFRL framework for use in yield-aware overlay control. During training phase 700, training data 800 (e.g., as described above) is used to train agent 500 (e.g., one or more processors). Training data 800 includes process information indicating various adjustments 802 made to individual layers 804 (e.g., 804 - 1, 804 - 2,... 804 - M), and corresponding process metrics (e.g., yield) enhancements of wafer 806. During the training phase, agent 500 learns a policy function 810 and / or a value function 812. In some embodiments, agent 500 can make some probing adjustments to find improved policies for system dynamics and / or for other reasons.

[0142] During servo phase 702, agent 500 executes 814 an optimal policy (e.g., 810 and / or 812) for semiconductor processing procedure 815, and updates 811 training data 800 (e.g., for improving the policy) based on information generated during servo phase 702 (e.g., processing information such as overlay measurement 820 and corresponding yield enhancement 830).

[0143] It should be noted that the systems and methods described herein can also be used for other applications, such as intelligent sampling, alignment mark position optimization, and / or other applications. For intelligent sampling, the state can include all historical data for a particular wafer (e.g., processing information including overlay, focus, etc., measurements of previous layers, context information, and scanner sensor information, and / or other information). The action (e.g., adjustment) can be boolean, yes or no. The agent can be an intelligent optimal sampling system, such as SSO. The reward (process metric) for intelligent sampling can become a cost. For example, each time the system decides to take a sample, a cost may have been incurred, and at the end, a positive reward can be accumulated based on the yield (and / or other process metrics). This scenario can result in a sampling scheme that optimizes the yield (and / or other process metrics), but seeks a trade-off between the measurement cost and the final yield.

[0144] For alignment mark position optimization, the state can include historical data (e.g., processing information, including overlay measurements of previous layers, all context information, and scanner sensor information, and / or other processing information). For example, the action (e.g., adjustment) can define the layout of alignment marks on the wafer. The agent is again one or more processors. The reward (process metric) in this example is the overlay key performance indicator that is accumulated after the measurement layer.

[0145] Further embodiments of the present invention are disclosed in the following numbered list of aspects:

[0146] 1. A semiconductor processing method, the method comprising:

[0147] Determining, using one or more physical processors, a sequence of states of an object being processed, the states being determined based on processing information associated with the object;

[0148] Determining, using the one or more processors, a process metric associated with the object based on at least one state within the sequence of states, the process metric including an indication of whether a processing requirement for the object is satisfied for an individual state within the sequence of states; and

[0149] Using the one or more processors, initiate an adjustment to a processing procedure based on (1) at least one of the states within the sequence of the states and (2) the process metric, the adjustment being configured to enhance the process metric for the individual states within the sequence of the states such that a final processing requirement for the object is met.

[0150] 2. The method according to aspect 1, wherein the sequence of the states corresponds to a sequence of processing operations performed on the object, and wherein determining the sequence of the states, determining the process metric, and initiating the adjustment comprise:

[0151] Determine a policy function P(s), the policy function defining or specifying a processing operation correction for an individual state, equipment for performing the processing operation, and / or one or more process parameters for the processing operation; and / or

[0152] Determine a value function V(s), the value function defining or specifying the enhancement of the process metric assuming that the policy function is followed until the completion of the sequence of the processing operations.

[0153] 3. The method according to aspect 2, wherein the value function defines or specifies an expected process metric for a given state (s).

[0154] 4. The method according to any one of aspects 1 to 3, wherein the method is performed for a semiconductor processing environment, and the object being processed is a semiconductor wafer or one or more parts of the semiconductor wafer.

[0155] 5. The method according to any one of aspects 1 to 4, wherein the process metric comprises one or more of the following: yield, cost of sensor measurement, production volume, indication of a trade-off between yield optimization and measurement density, cost of overlap measurement, or overlap.

[0156] 6. The method according to any one of aspects 1 to 5, wherein the process metric comprises a reward, and the one or more processors comprise an agent.

[0157] 7. The method according to any one of aspects 1 to 6, wherein the process metric comprises yield, and enhancing the process metric for the individual states within the sequence of the states such that a final processing requirement for the object is met comprises increasing the yield.

[0158] 8. The method according to any one of aspects 1 to 7, wherein the process metric and / or the adjustment are determined based on at least two of the states within the sequence of the states.

[0159] 9. The method according to any one of aspects 1 to 8, wherein the process metric and / or the adjustment is determined based on a plurality of the states within the sequence of the states.

[0160] 10. The method according to any one of aspects 1 to 9, wherein starting the adjustment includes: (1) optimizing the process metric based on the sequence of the states, and determining the adjustment based on the optimized process metric; and / or (2) prompting a user to make the adjustment.

[0161] 11. The method according to any one of aspects 1 to 10, wherein the adjustment includes correction.

[0162] 12. The method according to aspect 11, wherein the correction is an exposure correction associated with the semiconductor processing.

[0163] 13. The method according to any one of aspects 1 to 12, wherein the adjustment includes where, when, and / or how to measure an indication of the object during one or more processing operations.

[0164] 14. The method according to any one of aspects 1 to 13, wherein the sequence of the states corresponds to a sequence of processing operations performed on the object, and the adjustment includes one or more of the following: a change in the processing operation performed, a change in the order of performing the processing operations, or a change in one or more pieces of equipment used to perform one or more of the processing operations.

[0165] 15. The method according to aspect 14, wherein the adjustment includes a change in one or more process parameters of the one or more processing operations.

[0166] 16. The method according to aspect 15, wherein the one or more process parameters include one or more of the following: dose, focal length, mask design, exposure level, one or more etching parameters, one or more deposition parameters, or one or more measurement parameters.

[0167] 17. The method according to any one of aspects 1 to 16, wherein the sequence of the states corresponds to a sequence of processing operations performed on the object, and wherein the processing information includes one or more of the following: a value of a measurement of the object performed as part of the processing operation, an indication of which processing operations are performed, an indication of the order of the sequence of the processing operations, an indication of which equipment and / or associated machine constants are used in the processing operation, or a processing parameter of the processing operation.

[0168] 18. The method according to any one of aspects 1 to 17, wherein determining the sequence of the states, determining the process metrics, and initiating the adjustment are performed as at least part of a model-free reinforcement learning (MFRL) framework.

[0169] 19. The method according to aspect 18, wherein the MFRL framework includes one or more of the following: Asynchronous Advantage Actor-Critic algorithm, Q-learning with a normalized advantage function, Trust Region Policy Optimization algorithm, Proximal Policy Optimization algorithm, Twin Delayed Deep Deterministic Policy Gradient, or Soft Actor-Critic algorithm.

[0170] 20. The method according to any one of aspects 1 to 19, wherein the sequence of the states includes one or more future states of the object, where

[0171] determining the process metrics is based on the sequence of the states including the one or more future states; and

[0172] initiating the adjustment to the processing is based on the sequence of the states including the one or more future states and the process metrics.

[0173] 21. The method according to any one of aspects 2 to 20, further comprising:

[0174] using the one or more processors to compare the first sequence and the second sequence based on a policy function and a value function associated with a first sequence of one or more processing operations having first process parameters and a second sequence of one or more processing operations having second process parameters.

[0175] 22. The method according to any one of aspects 2 to 21, further comprising:

[0176] performing determining the sequence of the states, determining the process metrics, and initiating the adjustment as part of a servo operation phase; and / or

[0177] training the policy function and the value function during a training operation phase before the servo phase.

[0178] 23. The method according to aspect 22, wherein the training operation phase is performed in a simulated semiconductor processing environment.

[0179] 24. A non-transitory computer-readable medium having instructions thereon, the instructions when executed by a computer implement the method according to any one of aspects 1 to 23.

[0180] 25. A non-transitory computer-readable medium having instructions thereon, the instructions when executed by a computer cause the computer to:

[0181] Determine a sequence of states of an object being processed, the states being determined based on processing information associated with the object;

[0182] Determine a process metric associated with the object based on at least one of the states within the sequence of states, the process metric including an indication of whether a processing requirement for the object is satisfied for an individual state within the sequence of states; and

[0183] Based on (1) at least one of the states within the sequence of states and (2) the process metric, initiate an adjustment to a processing process, the adjustment being configured to enhance the process metric for the individual state within the sequence of states such that a final processing requirement for the object is satisfied.

[0184] 26. The non-transitory computer-readable medium according to aspect 25, wherein the sequence of states corresponds to a sequence of processing operations performed on the object, and wherein determining the sequence of states, determining the process metric, and initiating the adjustment include:

[0185] Determine a policy function P(s), the policy function defining or specifying a processing operation correction for an individual state, equipment for performing the processing operation, and / or one or more process parameters for the processing operation; and / or

[0186] Determine a value function V(s), the value function defining or specifying the enhancement of the process metric assuming that the policy function is followed until completion of the sequence of processing operations.

[0187] 27. The non-transitory computer-readable medium according to aspect 26, wherein the value function defines or specifies an expected process metric for a given state (s).

[0188] 28. The non-transitory computer-readable medium according to any one of aspects 25 to 27, wherein the computer is associated with a semiconductor processing environment, and the object being processed is a semiconductor wafer or one or more portions of the semiconductor wafer.

[0189] 29. The non-transitory computer-readable medium according to any one of aspects 25 to 28, wherein the process metric includes one or more of the following: yield, cost of sensor measurements, production volume, indication of a trade-off between yield optimization and measurement density, cost of overlap measurements, or overlap.

[0190] 30. The non-transitory computer-readable medium according to any one of aspects 25 to 29, wherein the process metric includes a reward, and the computer includes an agent.

[0191] 31. The non-transitory computer-readable medium according to any one of aspects 25 to 30, wherein the process metric includes a yield, and enhancing the process metric for the individual states in the sequence of states such that a final processing requirement for the object is met includes increasing the yield.

[0192] 32. The non-transitory computer-readable medium according to any one of aspects 25 to 31, wherein the process metric and / or the adjustment is determined based on at least two of the states within the sequence of states.

[0193] 33. The non-transitory computer-readable medium according to any one of aspects 25 to 32, wherein the process metric and / or the adjustment is determined based on a plurality of the states within the sequence of states.

[0194] 34. The non-transitory computer-readable medium according to any one of aspects 25 to 33, wherein starting the adjustment includes: (1) optimizing the process metric based on the sequence of states and determining the adjustment based on the optimized process metric; and / or (2) prompting a user to make the adjustment.

[0195] 35. The non-transitory computer-readable medium according to any one of aspects 25 to 34, wherein the adjustment includes a correction.

[0196] 36. The non-transitory computer-readable medium according to aspect 35, wherein the correction is an exposure correction associated with a semiconductor processing procedure.

[0197] 37. The non-transitory computer-readable medium according to any one of aspects 25 to 36, wherein the adjustment includes where, when, and / or how to measure an indication of the object during one or more processing operations.

[0198] 38. The non-transitory computer-readable medium according to any one of aspects 25 to 37, wherein the sequence of states corresponds to a sequence of processing operations performed on the object, and the adjustment includes one or more of: a change in the processing operations performed, a change in the order in which the processing operations are performed, or a change in one or more pieces of equipment used to perform the processing operations.

[0199] 39. The non-transitory computer-readable medium according to aspect 38, wherein the adjustment includes changing one or more process parameters of the one or more processing operations.

[0200] 40. The non-transitory computer-readable medium according to aspect 39, wherein the one or more process parameters include one or more of the following: dose, focal length, mask design, exposure level, one or more etching parameters, one or more deposition parameters, or one or more measurement parameters.

[0201] 41. The non-transitory computer-readable medium according to any one of aspects 25 to 40, wherein the sequence of states corresponds to a sequence of processing operations performed on the object, and wherein the processing information includes one or more of the following: values of measurements of the object performed as part of the processing operations, an indication of which processing operations are performed, an indication of the order of the sequence of processing operations, an indication of which equipment and / or associated machine constants are used in the processing operations, or the processing parameters of the processing operations.

[0202] 42. The non-transitory computer-readable medium according to any one of aspects 25 to 41, wherein determining the sequence of states, determining the process metric, and initiating the adjustment are performed as at least part of a model-free reinforcement learning (MFRL) framework.

[0203] 43. The non-transitory computer-readable medium according to aspect 42, wherein the MFRL framework includes one or more of the following: asynchronous advantage actor-critic algorithm, W learning with a normalized advantage function, trust region policy optimization algorithm, proximal policy optimization algorithm, twin-delayed deep deterministic policy gradient, or flexible actor-critic algorithm.

[0204] 44. The non-transitory computer-readable medium according to any one of aspects 25 to 43, wherein the sequence of states includes one or more future states of the object, wherein

[0205] determining the process metric is based on the sequence of states including the one or more future states; and

[0206] initiating the adjustment of the processing process is based on the sequence of states including the one or more future states and the process metric.

[0207] 45. The non-transitory computer-readable medium according to any one of aspects 26 to 44 further includes: causing the computer to compare the first sequence with the second sequence based on a policy function and a value function associated with a first sequence of one or more processing operations having first process parameters and a second sequence of one or more processing operations having second process parameters.

[0208] 46. The non-transitory computer-readable medium according to any one of aspects 26 to 45 further includes: causing the computer to perform the sequence of determining the state, determining the process metric, and initiating the adjustment as part of a servo operation phase; and / or

[0209] training the policy function and the value function during a training operation phase prior to the servo phase.

[0210] 47. The non-transitory computer-readable medium according to aspect 46, wherein the training operation phase is performed in a simulated semiconductor processing environment.

[0211] 48. A lithographic apparatus, the apparatus comprising:

[0212] an illumination source and projection optics configured to image a pattern onto a substrate; and

[0213] one or more physical processors configured by machine-readable instructions to:

[0214] determine a sequence of states of an object being processed, the states being determined based on processing information associated with the object;

[0215] determine a process metric associated with the object based on at least one of the states within the sequence of states, the process metric including an indication of whether a processing requirement for the object is satisfied for an individual state within the sequence of states; and

[0216] initiate an adjustment to a processing process based on (1) at least one of the states within the sequence of states and (2) the process metric, the adjustment being configured to enhance the process metric for the individual state within the sequence of states such that a final processing requirement for the object is satisfied.

[0217] 49. The lithographic apparatus according to aspect 48, wherein the adjustment includes a change in a process parameter associated with the illumination source, the projection optics, the pattern, and / or the substrate.

[0218] 50. A lithographic apparatus according to aspect 48 or 49, wherein the processing process is a semiconductor processing process, and

[0219] wherein the object to be processed is a semiconductor wafer or one or more parts of the semiconductor wafer.

[0220] 51. A lithographic apparatus according to any one of aspects 48 to 50, wherein the process metric includes a yield, and meeting the processing requirements includes increasing the yield.

[0221] 52. A lithographic apparatus according to any one of aspects 48 to 51, wherein starting the adjustment includes optimizing the process metric based on the sequence of the states, and determining the adjustment based on the optimized process metric.

[0222] 53. A lithographic apparatus according to any one of aspects 48 to 52, wherein the adjustment is an exposure correction associated with the lithographic apparatus.

[0223] 54. A lithographic apparatus according to any one of aspects 48 to 53, wherein determining the sequence of the states, determining the process metric, and starting the adjustment are performed as at least a part of a model-free reinforcement learning (MFRL) framework.

[0224] 55. A lithographic apparatus according to any one of aspects 48 to 54, wherein the sequence of the states includes one or more future states of the object, wherein

[0225] determining the process metric is based on the sequence of the states including the one or more future states; and

[0226] starting the adjustment to the processing process is based on the sequence of the states including the one or more future states and the process metric.

[0227] 56. A lithographic apparatus according to any one of aspects 48 to 55, wherein the sequence of the states corresponds to a sequence of processing operations performed on the object, wherein meeting the processing requirements includes enhancing the process metric, and wherein starting the adjustment includes:

[0228] determining a policy function P(s), the policy function defining or specifying a processing operation correction for a separate state, equipment for performing the processing operation, and / or one or more process parameters for the processing operation; and / or

[0229] determining a value function V(s), the value function defining or specifying the enhancement of the process metric on the assumption that the policy function is followed until the completion of the sequence of the processing operations.

[0230] 57. A semiconductor processing method, the method comprising:

[0231] Determining, using one or more physical processors, a sequence of states of an object to be processed, the states being determined based on processing information associated with the object;

[0232] Determining, using the one or more processors, a process metric associated with the object based on at least one of the states within the sequence of states, the process metric indicating a processing quality for an individual state within the sequence of states; and

[0233] Initiating, using the one or more processors, an adjustment to a processing process based on (1) at least one of the states within the sequence of states and (2) the process metric, the adjustment being configured to enhance the process metric for the individual state within the sequence of states such that a final processing requirement meets a quality standard.

[0234] Figure 9 FIG. is a block diagram of a computer system 100 that may assist in implementing the methods, processes, or systems disclosed herein. Computer system 100 includes a bus 102 or other communication mechanism for communicating information, and a processor 104 (or processors 104 and 105) coupled to bus 102 for processing information. Computer system 100 also includes a main memory 106, such as a random access memory (RAM) or other dynamic storage, coupled to bus 102 for storing information and instructions to be executed by processor 104. Main memory 106 may also be used to store temporary variables or other intermediate information during execution of instructions to be executed by processor 104. Computer system 100 further includes a read only memory (ROM) 108 or other static storage device coupled to bus 102 for storing static information and instructions for processor 104. A storage device 110, such as a magnetic disk or optical disk, is provided and coupled to bus 102 for storing information and instructions.

[0235] The computer system 100 can be connected via a bus 102 to a display 112 for displaying information to a computer user, such as, for example, a cathode ray tube (CRT), a flat panel display, or a touch panel display. An input device 114 including alphanumeric keys and other keys is connected to the bus 102 for communicating information and command selections to the processor 104. Another type of user input device is a cursor control 116, such as a mouse, a trackball, or cursor direction keys, for communicating direction information and command selections to the processor 104 and for controlling cursor movement on the display 112. Such input devices typically have two degrees of freedom in two axes (a first axis (e.g., x) and a second axis (e.g., y)), which allows the device to specify a position in a plane. A touch panel (screen) display can also be used as an input device.

[0236] According to one embodiment, portions of one or more of the methods described herein can be performed by the computer system 100 in response to execution of one or more sequences of one or more instructions contained in the main memory 106 by the processor 104. These instructions can be read into the main memory 106 from another computer-readable medium, such as a storage device 110. Execution of the instruction sequences contained in the main memory 106 causes the processor 104 to perform the process steps described herein. One or more processors in a multiprocessing arrangement can also be used to execute the instruction sequences contained in the main memory 106. In alternative embodiments, hardwired circuitry can be used in place of or in combination with software instructions. Thus, the description herein is not limited to any specific combination of hardware circuitry and software.

[0237] As used herein, the term "computer-readable medium" refers to any medium that participates in providing instructions to the processor 104 for execution. Such a medium can take many forms, including (but not limited to) non-volatile media, volatile media, and transmission media. Non-volatile media includes, for example, optical or magnetic disks, such as the storage device 110. Volatile media includes volatile memory, such as the main memory 106. Transmission media includes coaxial cables, copper wire, and fiber optics, including the wires that comprise the bus 102. Transmission media can also take the form of acoustic or light waves, such as acoustic or light waves generated during radio frequency (RF) and infrared (IR) data communications. Common forms of computer-readable media include, for example, floppy disks, flexible disks, hard disks, magnetic tape, any other magnetic medium, CD-ROM, DVD, any other optical medium, punch cards, paper tape, any other physical medium with patterns of holes, RAM, PROM, and EPROM, FLASH-EPROM, any other memory chip or cartridge, a carrier wave as described below, or any other medium readable by a computer.

[0238] Computer-readable media of various forms may participate in carrying one or more sequences of one or more instructions to processor 104 for execution. For example, initially the instructions may be carried on a magnetic disk of a remote computer. The remote computer may load the instructions into its volatile memory and send the instructions using a modem via a telephone line. A modem local to computer system 100 may receive the data on the telephone line and convert the data into an infrared signal using an infrared transmitter. An infrared detector coupled to bus 102 may receive the data carried in the infrared signal and place the data on bus 102. Bus 102 carries the data to main memory 106, from which processor 104 retrieves and executes the instructions. The instructions received by main memory 106 may optionally be stored on storage device 110 before or after being executed by processor 104.

[0239] Computer system 100 may also include a communication interface 118 coupled to bus 102. Communication interface 118 provides a two-way data communication connection to network link 120, which is connected to a local area network 122. For example, communication interface 118 may be an integrated services digital network (ISDN) card or a modem to provide a data communication connection to a corresponding type of telephone line. As another example, communication interface 118 may be a local area network (LAN) card to provide a data communication connection to a compatible LAN. A wireless link may also be implemented. In any such implementation, communication interface 118 sends and receives electrical, electromagnetic, or optical signals that carry digital data streams representing various types of information.

[0240] Network link 120 typically provides data communication to other data devices via one or more networks. For example, network link 120 may be a connection provided by local area network 122 to a main computer 124 or a connection to data equipment operated by an Internet service provider (ISP) 126. ISP 126 in turn provides data communication services via the global packet data communication network (now commonly referred to as the "Internet" 128). Both local area network 122 and the Internet 128 use electrical, electromagnetic, or optical signals that carry digital data streams. Signals via various networks and signals on network link 120 and via communication interface 118 are exemplary forms of carrier waves that convey information and carry digital data to and from computer system 100.

[0241] Computer system 100 may send messages and receive data (including program code) via a network, network link 120, and communication interface 118. In an Internet example, server 130 may transmit requested code for an application via Internet 128, ISP 126, local area network 122, and communication interface 118. For example, one such downloaded application may provide all or part of the methods described herein. The received code may be executed by processor 104 when it is received and / or stored in storage device 110 or other non-volatile memory for later execution. In this manner, computer system 100 may obtain application code in the form of a carrier wave.

[0242] Figure 10 Schematically depicts an exemplary lithographic projection apparatus that may be utilized in conjunction with the techniques described herein. The apparatus includes:

[0243] - An illumination system IL for conditioning a radiation beam B. In such a particular case, the illumination system also includes a radiation source SO;

[0244] - A first stage (e.g., a patterning device stage) MT provided with a patterning device holder for holding a patterning device MA (e.g., a mask) and connected to a first positioner for accurately positioning the patterning device relative to an object PS;

[0245] - A second stage (substrate stage) WT provided with a substrate holder for holding a substrate W (e.g., a silicon wafer coated with resist) and connected to a second positioner for accurately positioning the substrate relative to an object PS; and

[0246] - A projection system (“lens”) PS (e.g., a refractive, reflective, or catadioptric optical system) for imaging an irradiated portion of the patterning device MA onto a target portion C (e.g., including one or more dies) of the substrate W.

[0247] As depicted herein, the apparatus is of the transmissive type (i.e., having a transmissive patterning device). However, in general, for example, it may also be of the reflective type (having a reflective patterning device). The apparatus may use different types of patterning devices relative to a classical mask; examples include a programmable mirror array or an LCD matrix.

[0248] A source SO (e.g., a mercury lamp, an excimer laser, an LPP (laser-produced plasma) EUV source) generates a radiation beam. For example, such a beam is fed directly or after having traversed an adjusting device such as a beam expander Ex into an illumination system (illuminator) IL. The illuminator IL may include an adjusting device AD for setting an outer radial range and / or an inner radial range of the intensity distribution in the beam (commonly referred to as σ-outer and σ-inner, respectively). Additionally, it will typically include various other components, such as an integrator IN and a condenser CO. In this way, the beam B incident on the patterning device MA has a desired uniformity and intensity distribution in its cross-section.

[0249] It should be noted that with respect to Figure 10 , the source SO may be located within the housing of the lithographic projection apparatus (which is often the case when the source SO is, for example, a mercury lamp), but it may also be remote from the lithographic projection apparatus, and the radiation beam generated by it (e.g., by means of a suitable directing mirror) is introduced into the apparatus; this latter scenario is often the case when the source SO is an excimer laser (e.g., emitting laser based on KrF, ArF, or F2).

[0250] The beam PB then intersects the patterning device MA held on the patterning device table MT. In the case where the beam has traversed the patterning device MA, the beam B passes through a lens PL which focuses the beam B onto a target portion C of the substrate W. By means of a second positioning device (and an interferometric device IF), the substrate table WT can be accurately moved, for example, in order to position different target portions C in the path of the beam PB. Similarly, the first positioning device can be used to accurately position the patterning device MA relative to the path of the beam B, for example, after mechanically retrieving the patterning device MA from a patterning device library or during scanning. Generally, the movement of the tables MT, WT will be achieved by means of long-stroke modules (coarse positioning) and short-stroke modules (fine positioning) not explicitly depicted in Figure 10 . However, in the case of a stepper (relative to a step-and-scan tool), the patterning device table MT may be connected only to a short-stroke actuator or may be fixed.

[0251] The tool depicted can be used in two different modes:

[0252] - In the step mode, the patterning device table MT remains substantially stationary, and an image of the entire patterning device is projected onto the target portion C in one go (i.e., a single "flash"). Subsequently, the substrate table WT is displaced in the x and / or y direction so that different target portions C can be irradiated by the beam PB;

[0253] - In the scanning mode, substantially the same situation applies, except that a given target portion C is not exposed during a single "flash". Specifically, the patterning device table MT can be moved at a speed v in a given direction (the so-called "scanning direction", e.g., the y-direction) such that the projection beam B scans across the image of the patterning device; simultaneously, the substrate table WT is moved simultaneously in the same or opposite direction at a speed V = Mv, where M is the magnification of the lens PL (usually, M = 1 / 4 or 1 / 5). In this way, a relatively large target portion C can be exposed without compromising the resolution.

[0254] Figure 11 Schematically depicts another exemplary lithographic projection apparatus 1000 that can be utilized in conjunction with the techniques described herein.

[0255] The lithographic projection apparatus 1000 includes:

[0256] - A source collector module SO;

[0257] - An illumination system (illuminator) IL configured to condition a radiation beam B (e.g., EUV radiation);

[0258] - A support structure (e.g., a patterning device table) MT configured to support a patterning device (e.g., a mask or a reticle) MA and connected to a first positioner PM configured to accurately position the patterning device;

[0259] - A substrate table (e.g., a wafer table) WT configured to hold a substrate (e.g., a wafer coated with resist) W and connected to a second positioner PW configured to accurately position the substrate; and

[0260] - A projection system (e.g., a reflective projection system) PS configured to project a pattern imparted to the radiation beam B by the patterning device MA onto a target portion C (e.g., including one or more dies) of the substrate W.

[0261] As Figure 11As depicted, device 1000 is of the reflective type (e.g., employing a reflective patterning device). It should be noted that since most materials are absorptive in the EUV wavelength range, the patterning device may have a multilayer reflector comprising, for example, multiple stacks of molybdenum and silicon. In one example, a multilayer reflector has 40 layer pairs of molybdenum and silicon, with each layer having a thickness of a quarter wavelength. X-ray lithography can be utilized to generate even smaller wavelengths. Since most materials are absorptive at EUV and x-ray wavelengths, a thin sheet of patterned absorptive material on the patterning device topography (e.g., a TaN absorber on top of the multilayer reflector) defines where features will be printed (positive resist) or not printed (negative resist).

[0262] Illuminator IL receives an extreme ultraviolet radiation beam from source collector module SO. Methods for generating EUV radiation include but are not limited to converting a material into a plasma state having at least one element (e.g., xenon, lithium, or tin) using one or more emission spectral lines in the EUV range. In one such method (often referred to as laser-produced plasma ("LPP")), a plasma can be generated by irradiating a fuel (e.g., a droplet, stream, or cluster of material having a spectral line-emitting element) with a laser beam. Source collector module SO can be a component of an EUV radiation system including a laser ( Figure 11 (not shown in the figure) for providing the laser beam to excite the fuel. The generated plasma emits output radiation (e.g., EUV radiation), and the output radiation is collected using a radiation collector disposed in the source collector module. For example, when a CO2 laser is used to provide the laser beam for fuel excitation, the laser and the source collector module can be separate entities.

[0263] In these cases, the laser is not considered part of or a component of the lithographic apparatus, and the radiation beam is transmitted from the laser to the source collector module by means of a beam delivery system including, for example, suitable steering mirrors and / or beam expanders. In other cases, for example, when the source is a discharge-produced plasma EUV generator (often referred to as a DPP source), the source can be an integral part of the source collector module. In an embodiment, a DUV laser source can be used.

[0264] Illuminator IL can include an adjuster for adjusting the angular intensity distribution of the radiation beam. Generally, at least the outer radial range and / or the inner radial range of the intensity distribution in the pupil plane of the illuminator can be adjusted (commonly referred to as σ - outer and σ - inner, respectively). Additionally, illuminator IL can include various other components, such as faceted field mirror devices and faceted pupil mirror devices. The illuminator can be used to adjust the radiation beam to have a desired uniformity and intensity distribution in its cross-section.

[0265] The radiation beam B is incident on a patterning device (e.g., a mask) MA which is held on a support structure (e.g., a patterning device table) MT, and the radiation beam B is patterned by the patterning device. After being reflected from the patterning device (e.g., a mask) MA, the radiation beam B passes through a projection system PS which focuses the beam onto a target portion C of a substrate W. By means of a second positioner PW and a position sensor PS2 (e.g., an interferometric device, a linear encoder or a capacitive sensor), the substrate table WT can be accurately moved, e.g., so as to position different target portions C in the path of the radiation beam B. Similarly, a first positioner PM and another position sensor PS1 can be used to accurately position the patterning device (e.g., a mask) MA relative to the path of the radiation beam B. Patterning device alignment marks M1, M2 and substrate alignment marks P1, P2 can be used to align the patterning device (e.g., a mask) MA and the substrate W.

[0266] The depicted apparatus 1000 can be used in at least one of the following modes:

[0267] In a step mode, the support structure (e.g., a patterning device table) MT and a suitable substrate table WT remain substantially stationary while the entire pattern imparted to the radiation beam is projected onto the target portion C at once (i.e., a single static exposure). Subsequently, the substrate table WT is displaced in the X and / or Y direction so that different target portions C can be exposed.

[0268] In a scan mode, the support structure (e.g., a patterning device table) MT and the substrate table WT are scanned simultaneously while the pattern imparted to the radiation beam is projected onto the target portion C (i.e., a single dynamic exposure). The rate and direction of the substrate table WT relative to the support structure (e.g., a patterning device table) MT can be determined by the magnification (reduction ratio) and the image inversion characteristics of the projection system PS.

[0269] In another mode, while the pattern to be imparted to the radiation beam is being projected onto the target portion C, the support structure (e.g., a patterning device table) MT holding the programmable patterning device is kept substantially stationary and the substrate table WT is moved or scanned. In this mode, a pulsed radiation source is typically used and the programmable patterning device is updated as required after each movement of the substrate table WT or between successive radiation pulses during the scan. This mode of operation can be readily applied to maskless lithography using a programmable patterning device such as a programmable mirror array of the type mentioned above.

[0270] Figure 12Device 1000 is shown in more detail and includes a source collector module SO, an illumination system IL, and a projection system PS. The source collector module SO is constructed and arranged such that a vacuum environment can be maintained within the enclosure structure 220 of the source collector module SO. An EUV radiation-emitting plasma 210 can be formed by a discharge-produced plasma radiation source. EUV radiation can be generated from a gas or vapor (e.g., Xe gas, Li vapor, or Sn vapor), in which a plasma 210 is generated to emit radiation in the EUV range of the electromagnetic spectrum. For example, the plasma 210 is generated by causing a discharge in at least a partially ionized plasma. To efficiently generate radiation, it may be necessary to have (e.g.) a partial pressure of 10 Pa of Xe, Li, Sn vapor, or any other suitable gas or vapor. In an embodiment, an excited tin (Sn) plasma is provided to generate EUV radiation.

[0271] The radiation emitted by the thermal plasma 210 is transferred from the source chamber 211 to the collector chamber 212 via an optional gas barrier or contaminant trap 230 (also referred to in some cases as a contaminant barrier or foil trap) positioned in or behind an opening in the source chamber 211. The contaminant trap 230 can include a channel structure. The contaminant trap 230 can also include a gas barrier, or a combination of a gas barrier and a channel structure. As is well known in the art, the contaminant trap or contaminant barrier 230 further indicated herein includes at least a channel structure.

[0272] The collector chamber 212 can include a radiation collector CO that can be a so-called grazing incidence collector. The radiation collector CO has an upstream radiation collector side 251 and a downstream radiation collector side 252. The radiation traversing the collector CO can be reflected from the grating spectral filter 240 and focused in a virtual source point IF along the optical axis indicated by the dotted line "O". The virtual source point IF is generally referred to as an intermediate focus, and the source collector module is arranged such that the intermediate focus IF is located at or near the opening 221 in the enclosure structure 220. The virtual source point IF is an image of the radiation-emitting plasma 210.

[0273] Subsequently, the radiation traverses the illumination system IL, which can include a faceted field mirror device 22 and a faceted pupil mirror device 24, which are arranged to provide a radiation beam 21 having a desired angular distribution and a radiation intensity having a desired uniformity at the patterning device MA. After being reflected at the patterning device MA held by the support structure MT, the radiation beam 21 forms a patterned beam 26, and the patterned beam 26 is imaged onto a substrate W held by a substrate stage WT via reflection elements 28, 30 by the projection system PS.

[0274] In the illumination optical device unit IL and the projection system PS, there may generally be more elements than those shown. Depending on the type of lithographic apparatus, optionally a grating spectral filter 240 may be present. Additionally, there may be more mirrors than those shown in the respective figures. For example, in the projection system PS, there may be 1 to 6 additional reflective elements more than the Figure 12 reflective elements shown.

[0275] Merely as an example of a collector (or collector mirror), as Figure 12 illustrated, the collector optics CO is depicted as a nested collector having grazing-incidence reflectors 253, 254, and 255. The grazing-incidence reflectors 253, 254, and 255 are arranged axially symmetrically about the optical axis O, and a collector optics CO of this type can be used in combination with a discharge-produced plasma source often referred to as a DPP source.

[0276] Alternatively, the source collector module SO may be part or component of an LPP radiation system as shown in Figure 13 . The laser LA is arranged to deposit laser energy into a fuel such as xenon (Xe), tin (Sn), or lithium (Li) to generate a highly ionized plasma 210 having an electron temperature of several 10 eV. The high-energy radiation generated during the de-excitation and recombination of these ions is emitted from the plasma, collected by a near-normal-incidence collector optics CO, and focused onto an opening 221 in the enclosure structure 220.

[0277] The concepts disclosed herein can be simulated or mathematically modeled for any general imaging system for imaging sub-wavelength features, and in particular can be used for emerging imaging technologies capable of generating increasingly shorter wavelengths. Emerging technologies already in use include extreme ultraviolet (EUV), DUV lithography capable of generating a 193 nm wavelength by using an ArF laser and even capable of generating a 157 nm wavelength by using a fluorine laser. Additionally, EUV lithography can generate wavelengths in the range of 20 nm to 5 nm by using a synchrotron or by impinging high-energy electrons onto a material (solid or plasma) to generate photons in this range. The concepts disclosed herein can also be simulated or mathematically modeled for other semiconductor processing steps.

[0278] Although the concepts disclosed herein can be used for imaging on substrates such as silicon wafers, it should be understood that the disclosed concepts can be used with any type of lithographic imaging system, e.g., a lithographic imaging system for imaging on substrates other than silicon wafers. Additionally, combinations and sub - combinations of the disclosed elements can include separate embodiments. For example, a model - free reinforcement learning system and yield as a process metric can be used together in a single embodiment, or the model - free reinforcement learning system can be used separately and / or with another process metric. These features can include separate embodiments, and / or these features can be used together in the same embodiment.

[0279] The foregoing description is intended to be illustrative, not limiting. Accordingly, those skilled in the art will appreciate that modifications can be made as described without departing from the scope of the claims set forth below.

Claims

1. A semiconductor processing method, the method comprising: Determining, by one or more physical processors, a sequence of states of an object undergoing a semiconductor manufacturing process, the states being determined based on process information associated with the object, wherein the sequence of states includes one or more future states of the object; Determining, by the one or more processors, a process metric associated with the object based on at least one state within the sequence of states and the one or more future states, the process metric including an indication of whether processing requirements for the object are met for individual states within the sequence of states; And Initiating, by the one or more processors, an adjustment to the semiconductor manufacturing process based on (1) at least one state of the state and the one or more future states and (2) the process metric, the adjustment being configured to enhance the process metric for the individual states within the sequence of states such that final processing requirements for the object are met.

2. The method according to claim 1, wherein, The sequence of states corresponds to a sequence of processing operations performed on the object, and wherein determining the sequence of states, determining the process metric, and initiating the adjustment includes: Determining a policy function P(s), the policy function defining a processing operation correction for an individual state, equipment for performing the processing operation, and / or one or more process parameters for the processing operation; and / or Determining a value function V(s), the value function defining the enhancement of the process metric assuming the policy function is followed until completion of the sequence of processing operations.

3. The method according to claim 2, wherein, The value function defines an expected process metric for a given state (s).

4. The method according to claim 1, wherein The method is performed for a semiconductor processing environment, and the object being processed is a semiconductor wafer or one or more portions of the semiconductor wafer.

5. The method according to claim 1, wherein The process metric includes a reward, and the one or more processors include an agent.

6. The method according to claim 1, wherein The process metric includes a yield, and enhancing the process metric for the individual states within the sequence of states such that final processing requirements for the object are met includes increasing the yield.

7. The method according to claim 1, wherein, Initiating the adjustment includes: (1) optimizing the process metric based on the sequence of states and determining the adjustment based on the optimized process metric; and / or (2) prompting a user to make the adjustment.

8. The method according to claim 1, wherein The sequence of states corresponds to a sequence of processing operations performed on the object, and the adjustment includes one or more of the following: a change in the processing operation performed, a change in the order in which the processing operation is performed, or a change in one or more pieces of equipment used to perform the processing operation.

9. The method according to claim 1, wherein The sequence of the states corresponds to a sequence of processing operations performed on the object, and wherein the processing information includes one or more of the following: values of measurements of the object performed as part of the processing operations, an indication of which processing operations are performed, an indication of the order of the sequence of the processing operations, an indication of which equipment and / or associated machine constants are used in the processing operations, or processing parameters of the processing operations.

10. The method according to claim 1, wherein, Determining the sequence of the states, determining the process metrics, and initiating the adjustment are performed as at least part of a model-free reinforcement learning (MFRL) framework.

11. The method according to claim 10, wherein, The MFRL framework includes one or more of the following: Asynchronous Advantage Actor-Critic algorithm, Q-learning with a normalized advantage function, Trust Region Policy Optimization algorithm, Proximal Policy Optimization algorithm, Twin Delayed Deep Deterministic Policy Gradient, or Soft Actor-Critic algorithm.

12. The method according to claim 2 further comprises: Using the one or more processors, compare the first sequence with the second sequence based on a policy function and a value function associated with a first sequence of one or more processing operations having a first process parameter and a second sequence of one or more processing operations having a second process parameter.

13. The method according to claim 2 further comprises: Determining the sequence of the states, determining the process metrics, and initiating the adjustment are performed as part of a servo operation phase; and / or prior to the servo phase, the policy function and the value function are trained during a training operation phase.

14. A computer program product comprising instructions that are configured to perform the following operations when executed on a computer system: Using one or more physical processors, determine a sequence of states of an object undergoing a semiconductor manufacturing process, the states being determined based on processing information associated with the object, wherein the sequence of states includes one or more future states of the object; Using the one or more processors, determine a process metric associated with the object based on at least one state within the sequence of states and the one or more future states, the process metric including an indication of whether a processing requirement for the object is met for an individual state within the sequence of states; And Using the one or more processors, initiate an adjustment to a processing process based on (1) at least one state of the state and the one or more future states, and (2) the process metric, the adjustment being configured to enhance the process metric for the individual state within the sequence of states such that a final processing requirement for the object is met.

15. The computer program product according to claim 14, wherein, The sequence of the states corresponds to a sequence of processing operations performed on the object, and wherein the instructions are further configured to cause determining the sequence of the states, determining the process metrics, and initiating the adjustment to include: Determine a policy function P(s), the policy function defining a processing operation correction for an individual state, equipment for performing the processing operation, and / or one or more process parameters for the processing operation; and / or Determine a value function V(s), which defines the enhancement of the process metric assuming that the policy function is followed until the sequence of processing operations is completed.

Images