Control apparatus, lithographic apparatus, and article manufacturing method

Abstract

The present disclosure relates to a control apparatus, a lithographic apparatus and an article manufacturing method. A control apparatus to control an object to be controlled. The apparatus comprises a generator configured to generate a probability distribution for determining a manipulated variable; and a determiner configured to determine the manipulated variable based on the probability distribution generated by the generator. In an operational phase, the determiner determines the manipulated variable in accordance with an expected value of the probability distribution.

Description

TECHNICAL FIELD

[0001] The present application relates to a control apparatus, a lithographic apparatus, and an article manufacturing method. BACKGROUND

[0002] When learning a policy for maximizing total reward by reinforcement learning, one of continuous space and discrete space can be selected as an action space depending on constraints of an algorithm and properties of an environment. When a discrete action space is selected, an ε-greedy algorithm (non-patent literature 1, patent literature 1), a Softmax method (non-patent literature 1), or the like is generally used as an action policy during search. As an action policy during operation, a greedy algorithm is generally used.

[0003] Performance of a controller that outputs a probability distribution for determining a manipulated variable can be improved by learning using a method in which the manipulated variable is determined by sampling according to a random number. However, in actual operation, if the control variable is determined using a random number for sampling as in learning, random behavior can affect quality assurance. Therefore, generally, a manipulated variable that maximizes a probability value is continuously selected. On the other hand, when a manipulated variable having the largest probability is continuously selected, the control performance can be deteriorated compared to a case in which the manipulated variable is determined by sampling using a random number.

[0004] LIST OF CITATIONS

[0005] Patent Literature 1: Japanese Patent Publication No. 2020-98538

[0006] Non-patent Literature 1: Sutton, R. S., Barto, A. G.: “Reinforcement Learning: An Introduction”, MIT Press, Cambridge, MA (1998) SUMMARY

[0007] The present application provides a technique that is advantageous in suppressing deterioration of control performance during operation compared to control performance during learning.

[0008] A first aspect of the present application provides a control apparatus for controlling an object to be controlled, the apparatus comprising: a generator configured to generate a probability distribution for determining a manipulated variable; and a determiner configured to determine the manipulated variable based on the probability distribution generated by the generator, wherein, in an operation phase, the determiner determines the manipulated variable in accordance with an expected value of the probability distribution.

[0009] A second aspect of the present application provides a lithographic apparatus for transferring a pattern of a master to a substrate, the apparatus comprising: a movable part; and a control device as defined in the first aspect, the control device being configured to control the movable part.

[0010] A third aspect of the present application provides an article manufacturing method comprising: transferring a pattern of a master to a substrate using a lithographic apparatus as defined in the second aspect; and obtaining an article by processing the substrate to which the pattern has been transferred.

[0011] Other features of the present application will become apparent from the following description of example embodiments thereof, taken in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0012] Figure 1 is a diagram illustrating a configuration of a system according to an embodiment;

[0013] Figure 2 is a diagram illustrating a configuration example of a stage control device according to an embodiment; Figure 1 is a diagram illustrating a configuration example of an object to be controlled in a case where the system illustrated in

[0014] Figure 3 is a diagram illustrating a more specific configuration example of the stage control device illustrated in Figure 2

[0015] Figure 4 is a flowchart illustrating a method of determining a parameter value of a neural network by reinforcement learning;

[0016] Figure 5 is a diagram illustrating a configuration example of a neural network;

[0017] Figure 6 is a flowchart illustrating an operation of a neural network compensator;

[0018] Figure 7 is a graph illustrating a probability distribution (probability mass function);

[0019] Figure 8 is a diagram illustrating a sampling method using an inverse transform method;

[0020] Figure 9 is a graph illustrating a response of a stage;

[0021] Figure 10 is a diagram illustrating another configuration example of a neural network;

[0022] Figure 11 is a diagram illustrating another specific configuration example of a stage control device;

[0023] ​Figure 12 is a diagram showing a configuration example of an exposure apparatus as an example of a lithography apparatus; and

[0024] Figure 13 is a flowchart showing an operation example of the exposure apparatus shown in Figure 12 DETAILED DESCRIPTION

[0025] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments are not intended to limit the scope of the claimed invention. In the embodiments, a plurality of features are described, but the invention is not limited to the invention requiring all of these features, and a plurality of such features can be appropriately combined. In addition, in the drawings, the same reference numerals are assigned to the same or similar configurations, and redundant descriptions thereof are omitted.

[0026] Figure 1 A configuration of a system according to an embodiment is exemplified. The system can include a controlled object 1, a control server 2 that controls the controlled object 1, and a learning server 3 that learns by acquiring a control result from the controlled object 1 via the control server 2. The learning server 3 can transmit parameter information of a neural network to the neural network formed inside the controlled object 1 via the control server 2. Then, the control server 2 can transmit a control instruction to the controlled object 1 and acquire a control result from the controlled object 1. The control result acquired by the control server 2 from the controlled object 1 can be transmitted from the control server 2 to the learning server 3. According to the control result, the learning server 3 can calculate a reward indicating the quality of the parameter value of the neural network, and update the parameter value of the neural network based on the reward.

[0027] Since the calculation cost related to updating the parameter value of the neural network is high, it is advantageous to independently configure the control server 2 and the learning server 3. In a configuration in which the control server 2 and the learning server 3 are independent, when there are a plurality of controlled objects, it is possible to operate by preparing a plurality of learning servers 3 each having a high calculation cost and one control server 2 having a low calculation cost.

[0028] Figure 2 A configuration example of the controlled object 1 is shown in the case where the system shown in Figure 1

[0029] ​​Figure 3 A more specific configuration example of the stage control device is shown. Figure 2 A more specific configuration example of the stage control device is shown. The control board (controller) 7 can include, for example, a subtracter 76, a compensator 71, a neural network compensator 72, and an adder 75. The control board 7 can receive a manipulation instruction supplied from the control server 2, position information of the stage 5 supplied from the sensor 6, and stage information supplied from the control server 2. The position information of the stage 5 is an example of the state information indicating the state of the stage 5. The subtracter 76 can calculate a difference, i.e., a deviation, between the manipulation instruction supplied from the control server 2 and the position information supplied from the sensor 6, and supply the deviation to the compensator 71 and the neural network compensator 72. The compensator 71 generates a first manipulation variable based on the deviation supplied from the subtracter 76, and supplies the first manipulation variable to the adder 75.

[0030] The neural network compensator 72 generates a second manipulation variable based on the difference supplied from the subtracter 76, and supplies the second manipulation variable to the adder 75. The neural network compensator 72 can include a neural network 73 and a manipulation variable determiner 74 (determiner) that determines the second manipulation variable. The neural network 73 can output a probability distribution for determining the second manipulation variable based on the deviation supplied from the subtracter 76. The neural network 73 can be understood as a component that outputs a function defining a probability distribution for determining the second manipulation variable based on the deviation supplied from the subtracter 76. The neural network 73 can be understood as a probability distribution generator (generator) that generates a probability distribution for determining the second manipulation variable.

[0031] The manipulation variable determiner 74 determines the second manipulation variable based on the probability distribution or the function defining the probability distribution supplied from the neural network 73 and the stage information supplied from the control server 2. Possible values of the stage information can include a value indicating a learning stage in which parameter values of the neural network are learned, and a value indicating an operation stage in which control is performed using parameters of the neural network that have completed learning. A method of determining the manipulation variable by the manipulation variable determiner 74 will be described later. The compensator 71 and the neural network compensator 72 can be understood as a first compensator and a second compensator, respectively.

[0032] The adder 75 adds the first manipulation variable supplied from the compensator 71 and the second manipulation variable supplied from the neural network compensator 72, thereby generating a manipulation variable (combined manipulation variable). The adder 75 supplies the manipulation variable as a current instruction to the driver 8. As described above, the driver 8 includes a current driver and an actuator. The current driver can supply a current corresponding to the current instruction to the actuator, and the actuator can drive the stage 5. Note that the deviation supplied to the neural network compensator 72 is not necessarily a deviation of the position information. For example, a deviation of a velocity, an acceleration, or a jerk can be used.

[0033] The neural network parameter values of the neural network 73 (hereinafter simply referred to as parameter values) are required to be determined in advance by some learning method. An example of the learning method is reinforcement learning. Figure 4 A method (learning sequence) of determining the parameter values of the neural network 73 by reinforcement learning is exemplified. First, in step S400, the learning server 3 initializes the parameter values of the neural network 73. Then, in step S401, the learning server 3 changes the parameter values of the neural network 73. In step S402, the stage 7 manipulates the stage 5 serving as the controlled object in accordance with a predetermined manipulation instruction data (e.g., time series data of manipulation instructions).

[0034] In step S403, the learning server 3 acquires a control result of the stage 5 serving as the controlled object, such as deviation data (e.g., time series data of deviations). Here, the stage 7 can provide the control result to the learning server 3 via the control server 2. Then, the learning server 3 calculates a reward based on the deviation data of the controlled object. For example, the smaller the deviation, the higher the reward. Then, the learning server 3 determines whether or not the learning is completed. If it is determined that the learning is not completed, the process returns to step S401. If it is determined that the learning is completed, the process proceeds to step S406. In an example, the learning server 3 can determine that the learning is not completed if the number of learning times is equal to or less than a predetermined number, and can determine that the learning is completed if the number of learning times exceeds the predetermined number. In step S401, the learning server 3 can change the parameter values of the neural network 73 so as to increase the reward. In step S406, the learning server 3 saves the parameter values with which the maximum reward is obtained as a learning result. In the learning phase, the learning server 3 functions as a setter that sets the parameter values defining the operation of the neural network 73 (probability distribution generator) based on the control result of the controlled object controlled in accordance with the second manipulation variable determined by the manipulation variable determiner 74.

[0035] Figure 5 A configuration example of the neural network 73 is shown. The neural network 73 can include an input layer 731, one or more intermediate layers 732, an output layer 733, a function 734, and an output layer 735. The input layer 731 can input the deviation for the past N a control periods including the current control period as input data 736. In response to the input, the output data 738 of the output layer 733 can be determined via the one or more intermediate layers 732. The output layer 738 can have N b numerical values (probabilities). The function 734 is, for example, a Softmax function. The function 734 can generate a probability mass function as the output data 739 of the output layer 735, which is obtained by normalizing the N bEach of the numerical values is converted into a normalized probability. The function 734 functions as a converter that converts the output of the neural network 73 into a probability mass function.

[0036] In the learning phase, learning is performed using a reinforcement learning method such as proximal policy optimization (hereinafter referred to as PPO) including a policy network, and the manipulated variable can be determined by generating a sample from the probability mass function of the output data 739. In order to sample from the probability distribution represented by the probability mass function, for example, a pseudo-random number generation algorithm such as an inverse transform method or an MCMC method can be used. In this way, learning can be performed while performing a search action.

[0037] After the learning phase is completed, in an operation phase in which the parameter values at which the learning phase is completed or the parameter values at which the maximum reward is obtained are used, the manipulated variable having the highest probability of the output data 739 after conversion is generally selected. However, in a system that exhibits a transient response such as a low-pass filter in stage control or the like, the cumulative value of the manipulated variable affects the stage response. Therefore, the reward obtained by continuing to select the manipulated variable having the maximum probability can decrease compared to the reward obtained when sampling from the probability mass function in the learning phase.

[0038] In order to prevent this, in the present embodiment, when using the expected value that is the sum of the products of each manipulated variable candidate and its probability as the output of the neural network compensator 72 (i.e., the second manipulated variable), an effect similar to that in the learning phase can be obtained in the operation phase.

[0039] Figure 6 The operation of the neural network compensator 72 is exemplified. First, in step S601, the neural network 73 outputs a probability distribution using the manipulated variable candidate as a random variable to the output layer 735, in other words, the probability distribution is used to determine the second manipulated variable. The probability distribution can be, for example, a probability mass function, but can also be a probability density function that will be described later. In step S602, the manipulated variable determiner 74 receives the phase information included in the control instruction supplied from the control server 2 and checks the current phase. If the received phase information indicates the learning phase, the manipulated variable determiner 74 advances the process to step S603. If the received phase information indicates the operation phase, the manipulated variable determiner 74 advances the process to step S605.

[0040] In step S603, i.e., during the learning phase, the manipulation variable determiner 74 randomly determines the value of a random variable as the second manipulation variable based on the probability distribution (temporarily set probability distribution) of the output layer 735 of the neural network 73. In step S605, i.e., during the operation phase, the manipulation variable determiner 74 determines the second manipulation variable based on the expected value of the probability distribution of the output layer 735 of the neural network 73. In step S604, if it is during the learning phase, the manipulation variable determiner 74 outputs the second manipulation variable determined in step S603; if it is during the operation phase, the manipulation variable determiner 74 outputs the second manipulation variable determined in step S605.

[0041] Here, in execution Figure 4 The process shown, namely the method for determining the parameter values ​​of neural network 73 (learning sequence), during which... Figure 6 Steps S601, (S602), S603 and S604 in the process shown are performed in step S402.

[0042] The method for determining the manipulated variable during the operation phase (step S605) will now be described exemplarily. Here, N is defined. b Candidate manipulated variables a i (i=0 to N b ). Assigned to each candidate manipulate variable a i The probability p i Output data 739 appears as output data of output layer 735.

[0043] Figure 7 Example of a candidate manipulated variable a i With probability p i The relationship between these factors is that of probability distribution (probability mass function). The expected value E determined in step S605 is the expected value of the probability distribution output to the output layer 735 of the neural network 73. The expected value E is a i With p i The sum of the products, expressed as:

[0044]

[0045] The method for determining the manipulated variable during the learning phase (step S603) will be described below as an example. Here, as an example, reference will be made to... Figure 8 Describe the inverse transform method. Consider a probability mass function, where a[i] represents the probability of selecting the i-th candidate manipulator. The cumulative distribution function b[i] is defined as:

[0046]

[0047] By using a continuous uniform random number r in the interval [0,1] and selecting the smallest i that satisfies r ≤ b[i], a sample from the probability distribution represented by the probability mass function can be obtained. That is, based on the probability distribution, the value of the random variable can be randomly determined as the second manipulated variable.

[0048] As learning methods used in the learning phase, in addition to reinforcement learning methods such as PPO which includes a policy network, reinforcement learning methods such as Deep Q-Networks (DQNs) which do not include a policy network can also be used. In this case, the past N, including the current control period, is considered. a The deviation of the control command for each control cycle is input as input data 736 to input layer 731. N can be obtained via one or more intermediate layers 732. b The scores of the candidate manipulated variables are used as output data 738 of output layer 733. By using a specific function 734, such as the Softmax function, to convert the scores of the candidate manipulated variables into probabilities of the candidate manipulated variables, output data 739 of output layer 735 can be generated.

[0049] Figure 9 The response of stage 9 is illustrated. Solid lines represent the deviation of stage 9 during the learning phase. Dashed lines represent the deviation of stage 9 during the operation phase when the candidate with the highest probability of manipulation is output as the second manipulation variable. Short dashes represent the deviation of stage 9 when, according to this embodiment, the expected value of the probability distribution output to the output layer 735 of the neural network 73 is output as the second manipulation variable. (From...) Figure 9 It can be seen that if the candidate with the highest probability of manipulation is output as the second manipulation variable during the operation phase, the waveform deteriorates compared to the waveform during the learning phase. On the other hand, as from... Figure 9 As can be seen, if the expected value is output as the second manipulated variable during the operation phase, a waveform similar to that in the learning phase can be obtained.

[0050] As described above, in systems that display transient responses (such as low-pass filters in stage control), by using the expected value as the output in the operation phase of a neural network that produces discrete outputs, a bias suppression effect similar to that in the learning phase can be obtained.

[0051] The neural network 73 described above is merely an example; it can be used as follows: Figure 10 The neural network 303 shown is used instead. The neural network 303 may include an input layer 761, one or more intermediate layers 762, an output layer 763, a function 764, and an output layer 765. The input layer 761 can receive inputs for past N cycles, including the current control cycle. aThe deviation of each control cycle is used as input data 766. As output data 769 of output layer 765 via one or more intermediate layers 762, output layer 763, and activation function 764, the coefficients α and β of a probability density function β distribution can be determined. When determining the second manipulated variable, the β distribution represented by coefficients α and β is scaled to the range [Fmin, Fmax] of the second manipulated variable.

[0052] During the learning phase, reinforcement learning methods such as PPO (Policy Processing Network) are used, and the second manipulated variable can be determined by generating samples based on the probability density function. To sample from the probability distribution represented by the probability density function, an appropriate pseudo-random number generation algorithm, such as inverse transformation or acceptance-rejection, can be used depending on the type of probability density function. This allows learning to occur simultaneously with the search action. On the other hand, in the operation phase, where the parameter values ​​from the completed learning phase or the parameter values ​​from which the maximum reward was obtained are used, the aforementioned scaling is applied to the candidate manipulated variable with the highest probability in the β distribution represented by coefficients α and β, which is output data 769, and the obtained value can be used as the output. However, as mentioned above, in systems exhibiting transient responses (e.g., low-pass filters in stage control), the cumulative value of the manipulated variable affects the stage response. Therefore, the reward obtained by continuing to select the manipulated variable with the highest probability may be reduced compared to the reward obtained when sampling from the probability density function during the learning phase. To prevent this, the second manipulated variable is determined based on the expected value E of the β distribution represented by the following formula:

[0053]

[0054] For example, by scaling the expected value E as described above, the second manipulated variable can be determined. This achieves a similar effect to that in the learning phase. The manipulated variable determiner 74 operates as described above. Reinforcement learning methods that do not include a policy network can be used as the learning method employed in the learning phase.

[0055] As mentioned above, even when using a neural network with continuous output values ​​in a system that displays transient response (such as a low-pass filter in stage control), a bias suppression effect similar to that in the learning phase can be obtained by using the expected value as the output in the operation phase.

[0056] Figure 11Another specific configuration example of the stage control device is shown. In the example above, the difference (deviation) between the operation command and the position information is supplied to either the neural network compensator 72 or the neural network 73. However, the quality of the neural network parameter values ​​can be determined from the reward calculated based on the deviation data of the object to be controlled. Therefore, the difference (deviation) between the operation command and the position information does not necessarily have to be input to the neural network compensator 72, but one or both of the operation command and the position information obtained from the output of sensor 6 can be input. Note that, again in this case, the position information does not necessarily have to be input to the neural network compensator 72. For example, velocity, acceleration, or jerk can be input. Also in this configuration, during the operation phase, a second manipulation variable can be determined based on the expected value of the probability distribution output from the neural network 73. In this way, even when the difference (deviation) between the operation command and the position information is input to the neural network compensator 72, a similar deviation suppression effect as in the learning phase can be obtained by using the expected value of the probability distribution as the second manipulation variable during the operation phase.

[0057] In the above description, the manipulation variable to be supplied to the driver 8 is generated by adding the first manipulation variable output from the compensator 71 and the second manipulation variable output from the neural network compensator 72. However, the compensator 71 is not always necessary. For example, the second manipulation variable output from the neural network compensator 72 can be supplied to the driver 8 unchanged.

[0058] Figure 12 An example of a scanning exposure apparatus 800 in which the above system is applied as an example of a photolithography apparatus is shown. The scanning exposure apparatus 800 is a step-scanning exposure apparatus that scans and exposes a substrate 14 using slit-shaped light formed by slit-forming. The scanning exposure apparatus 800 may include: an illumination optics system 23, a component stage 12, a projection optics system 13, a substrate stage 15, a component stage position measuring device 17, a substrate stage position measuring device 18, a substrate mark measuring device 21, a substrate conveyor 22, a controller 24, and a temperature controller 25.

[0059] The controller 24 can control the illumination optics system 23, the original stage 12, the projection optics system 13, the substrate stage 15, the original stage position measuring device 17, the substrate stage position measuring device 18, the substrate marking measuring device 21, and the substrate conveyor 22. The controller 24 can control the process of transferring the pattern formed in the original 11 to the substrate 14 (the process of scanning and exposing the substrate 14). The controller 24 is formed by, for example, a PLD (programmable logic device) such as an FPGA (field-programmable gate array), an ASIC (application-specific integrated circuit), a general-purpose computer with a program installed, or a combination of all or part of these components. The controller 24 also includes a driver for controlling the actuators.

[0060] The illumination optical system 23 illuminates the original object 11. The illumination optical system 23 can use a light-shielding member such as a shielding blade to shape the light emitted from the light source (not shown) into a long strip or arc-shaped slit in the X direction, and use this slit-shaped light to illuminate a portion of the original object 11. The original object 11 and the substrate 14 are held by the original object stage 12 and the substrate stage 15, respectively, and are arranged in an optically conjugate position (object plane and image plane of the projection optical system 13) via the projection optical system 13.

[0061] The projection optical system 13 has a predetermined projection magnification (e.g., 1 / 2 or 1 / 4) and projects the pattern of the original 11 onto the substrate 14 using a slit-shaped light beam. The area on the substrate 14 where the pattern of the original 11 is projected (the area illuminated by the slit-shaped light) is called the illumination area. The original stage 12 and the substrate stage 15 are configured to be movable in a direction orthogonal to the optical axis (Z direction) of the projection optical system 13 (Y direction). The original stage 12 and the substrate stage 15 are respectively driven by a driver (not shown) at a speed ratio corresponding to the projection magnification of the projection optical system 13, and are synchronously scanned and driven relative to each other. Therefore, the substrate 14 is scanned in the Y direction relative to the illumination area, and the pattern formed in the original 11 is transferred to the shot region on the substrate 14. The exposure process of a substrate 14 is completed by sequentially performing the scanning exposure as described above on each of the plurality of shot regions of the substrate 14 while moving the substrate stage 15.

[0062] The component stage position measuring device 17 includes, for example, a laser interferometer, and measures the position of the component stage 12. For example, the laser interferometer emits a laser beam toward a reflector (not shown) disposed on the component stage 12, and detects the displacement of the component stage 12 (displacement from a reference position) based on the interference between the laser beam reflected from the reflector and the laser beam reflected from a reference surface. The component stage position measuring device 17 can obtain the current position of the component stage 12 based on this displacement. Here, the component stage position measuring device 17 measures the position of the component stage 12 using an interferometer with a laser beam, but the invention is not limited thereto. For example, an encoder can measure the position of the component stage 12.

[0063] The substrate stage position measuring device 18 includes, for example, a laser interferometer, and measures the position of the substrate stage 15. For example, the laser interferometer emits a laser beam toward a reflector (not shown) disposed on the substrate stage 15, and detects the displacement of the substrate stage 15 (displacement from a reference position) based on the interference between the laser beam reflected on the reflector and the laser beam reflected on a reference surface. The substrate stage position measuring device 18 can obtain the current position of the substrate stage 15 based on this displacement. Here, the substrate stage position measuring device 18 measures the position of the substrate stage 15 using an interferometer with a laser beam, but the invention is not limited thereto. For example, an encoder can measure the position of the substrate stage 15.

[0064] The substrate marking measurement device 21 includes, for example, an image sensor and can detect the position of markings disposed on the substrate. Here, the substrate marking measurement device 21 of this embodiment detects the markings using an image sensor, but the invention is not limited thereto. For example, a transmission sensor can detect the markings. The substrate conveyor 22 supplies substrates to and collects substrates from the substrate stage 15. The temperature controller 25 maintains a constant temperature and humidity within the exposure apparatus.

[0065] Figure 13 It shows Figure 12 An example of the operation of the exposure apparatus shown is presented. In step S901, the substrate conveyor 22 supplies the substrate 14 onto the substrate stage 15. In step S902, the substrate stage 15 is driven such that the marks on the substrate 14 specified in the exposure recipe enter the measurement field of view of the substrate mark measuring device 21, and the substrate 14 is aligned. In step S903, scanning exposure of the substrate 14 is performed for each injection region of the substrate 14. The exposure sequence and exposure angle follow the specifications of the exposure recipe. In step S904, the substrate conveyor 22 collects the substrate 14 from the substrate stage.

[0066] The following describes an example of applying the above system to the control of the substrate stage (movable part) 15. Figure 2 The sensor 6 shown corresponds to the substrate stage position measuring device 18, the control board 7 corresponds to the controller 24, the driver 8 corresponds to the substrate stage driver (not shown), and the stage 5 corresponds to the substrate stage 15. When the above system is applied to control the substrate stage 15, the settling time, which is the time from when the substrate stage 15 is driven until the deviation converges, can be shortened, thereby improving the accuracy and throughput of the exposure apparatus. Similarly, in the system used to control the substrate stage 15, by determining the manipulated variable based on the expected value of the probability distribution used to determine the manipulated variable during the operation phase, a deviation suppression effect similar to that in the learning phase can be obtained.

[0067] The following describes an example of applying the above system to the control of the original stage (movable part) 12. Figure 2 The control panel 7 shown corresponds to the controller 24, the driver 8 corresponds to the original stage driver (not shown), the sensor 6 corresponds to the original stage position measuring device 17, and the stage 5 corresponds to the original stage 12. Similarly, in the system used to control the original stage 12, by determining the manipulated variable based on the expected value of the probability distribution used to determine the manipulated variable during the operation phase, a similar bias suppression effect as in the learning phase can be obtained.

[0068] The following describes an example of applying the above system to the control of the substrate conveyor (movable part) 22. Figure 2 The control board 7 shown corresponds to the controller 24, the driver 8 corresponds to the substrate conveyor driver (e.g., an AC servo motor) (not shown), the sensor 6 corresponds to the rotary encoder (not shown), and the stage 5 corresponds to the substrate conveyor 22. When the above system is applied to the control of the substrate conveyor 22, deviations during the driving of the substrate conveyor 22 can be suppressed, thereby improving the reproducibility of the supply position when supplying the substrate 14 to the substrate stage 15. Furthermore, by suppressing deviations while increasing acceleration and speed, the throughput can also be improved. Similarly, in the system used to control the substrate conveyor 22, by determining the manipulated variable based on the expected value of the probability distribution used to determine the manipulated variable during the operation phase, a similar deviation suppression effect as in the learning phase can be obtained.

[0069] So far, applications of driving devices for each of the substrate stage, component stage, and substrate transport in a scanning exposure apparatus have been described; however, the present invention can be applied to another driving device in a scanning exposure apparatus. The present invention can also be applied to exposure apparatuses that perform exposure while the component and substrate are stopped, or to another lithography apparatus, such as an imprinting apparatus. Furthermore, the present invention can be applied to another control device for controlling an object to be controlled.

[0070] Next, a method for manufacturing articles (semiconductor IC components, liquid crystal display components, MEMS, etc.) using the aforementioned photolithography apparatus will be described. This article manufacturing method may include a transfer step of transferring a pattern of a component onto a substrate using the photolithography apparatus, and a processing step of obtaining the article by processing the substrate with the transferred pattern. When the photolithography apparatus is an exposure apparatus, the article manufacturing method may include a transfer step of transferring a pattern of a component onto a substrate (wafer, glass substrate, etc.) to which a photosensitizer has been applied, and a processing step of obtaining the article by processing the substrate with the transferred pattern. This processing step may include a step of developing the substrate (photosensitive agent). This processing step may also include other known steps, such as steps for etching, resist removal, cutting, bonding, and encapsulation. According to this article manufacturing method, articles of higher quality than conventional articles can be manufactured.

[0071] Please note that a series of embodiments have been described using stage control equipment and exposure apparatus, but control equipment with other configurations can be used.

[0072] Other embodiments

[0073] One or more embodiments of the present invention can also be implemented by a computer that reads and executes computer-executable instructions (e.g., one or more programs) recorded on a storage medium (which may also be more fully referred to as a "non-transient computer-readable storage medium") to perform the functions of one or more embodiments described above and / or includes one or more circuits (e.g., application-specific integrated circuits (ASICs)) for performing the functions of one or more embodiments described above, and by a method performed by a computer of a system or device by, for example, reading and executing computer-executable instructions from a storage medium to perform the functions of one or more embodiments described above and / or controlling one or more circuits to perform the functions of one or more embodiments described above. The computer may include one or more processors (e.g., a central processing unit (CPU), a microprocessor unit (MPU)) and may include a network of individual computers or individual processors to read and execute computer-executable instructions. The computer-executable instructions may be provided to the computer, for example, from a network or a storage medium. The storage medium may include, for example, a hard disk, random access memory (RAM), read-only memory (ROM), a storage device for a distributed computing system, an optical disc (such as a CD, DVD, or Blu-ray disc), or a digital versatile disc (BD). TM One or more of the following: flash memory devices, memory cards, etc.

[0074] The embodiments of the present invention can also be implemented by providing software (programs) that perform the functions of the above embodiments to a system or device via a network or various storage media, and the computer or central processing unit (CPU) or microprocessor unit (MPU) of the system or device reads out and executes the program.

[0075] Although the invention has been described with reference to exemplary embodiments, it should be understood that the invention is not limited to the disclosed exemplary embodiments. The scope of the appended claims should be given the broadest interpretation to cover all such modifications and equivalent structures and functions.

Claims

1. A control device for controlling an object to be controlled, the device comprising: The generator is configured to receive the difference between control commands and state information indicating the state of the object, and to generate a probability distribution for determining the manipulated variable based on the difference. as well as The determiner is configured to determine the manipulated variable based on the probability distribution generated by the generator. In the operation phase, the determiner determines the manipulated variable based on the expected value of the probability distribution.

2. The device according to claim 1, wherein During the learning phase, the determiner will use the values ​​of random variables, which are randomly determined based on a temporarily set probability distribution, as the manipulated variables.

3. The device according to claim 2, further comprising: The setter is configured to set parameter values ​​for the generator's operations during the learning phase based on the control results of the object controlled according to the manipulation variables determined by the determiner.

4. The device according to any one of claims 1 to 3, wherein, The probability distribution is a probability mass function.

5. The device according to claim 4, wherein The generator consists of a neural network that generates scores for multiple candidates of manipulated variables.

6. The device according to claim 5, wherein The generator also includes a converter configured to transform the output of the neural network into a probability mass function.

7. The device according to claim 6, wherein The converter transforms the output of the neural network based on the Softmax function.

8. The device according to claim 1, wherein A probability distribution is a probability density function.

9. The device according to claim 1, wherein Status information is the location of an object.

10. The device according to claim 1, wherein The state information is one of the object's velocity, acceleration, and jerk.

11. The device according to claim 1, further comprising: The first compensator is configured to generate a first manipulated variable based on the difference between the control command and the state information; as well as The adder is configured to generate a combined manipulated variable obtained by adding a first manipulated variable to a manipulated variable determined by the determiner. The combined manipulation variable is supplied to the driver configured as the driving object.

12. A photolithography apparatus for transferring a pattern of an original onto a substrate, the apparatus comprising: Movable parts; as well as The control device as described in claim 1 is configured to control a movable part.

13. The apparatus according to claim 12, wherein, The movable part is one of the following: substrate stage, component stage, and substrate conveyor.

14. A method for manufacturing an article, comprising: Using the photolithography apparatus as described in claim 13, the pattern of the original is transferred onto the substrate; as well as Items are obtained by processing a substrate that has already been transferred with a pattern.

15. A control device for controlling an object to be controlled, the device comprising: The generator is configured to receive control commands and state information indicating the state of the object, and to generate a probability distribution for determining the manipulated variables based on the control commands and state information. as well as The determiner is configured to determine the manipulated variable based on the probability distribution generated by the generator. In the operation phase, the determiner determines the manipulated variable based on the expected value of the probability distribution.

16. A photolithography apparatus for transferring a pattern of an original onto a substrate, the apparatus comprising: Movable parts; as well as The control device as described in claim 15 is configured to control a movable part.

17. The apparatus according to claim 16, wherein, The movable part is one of the following: substrate stage, component stage, and substrate conveyor.

18. A method for manufacturing an article, comprising: Using the photolithography apparatus as described in claim 17, the pattern of the original is transferred onto the substrate; as well as Items are obtained by processing a substrate that has already been transferred with a pattern.

Images