Control method, control apparatus, learning method, information processing apparatus, exposure apparatus, and article manufacturing method

Abstract

To provide a technique advantageous for suppressing control deviation.SOLUTION: A first step of performing machine learning of a parameter used for outputting a control signal for controlling the driving device, and a second step of controlling the driving device according to the control signal output using the parameter, in which in the first step, the machine learning is performed by evaluating a control result of each of a plurality of sections having different driving states of the driving device using different evaluation reference values set according to each of the plurality of sections.SELECTED DRAWING: Figure 8

Description

[Technical Field]

[0001] The present invention relates to a control method, a control device, a learning method, an information processing device, an exposure apparatus, and a method for manufacturing an article. [Background technology]

[0002] The control devices that control the physical quantities of the controlled object include classical controllers such as PID controllers, controllers based on modern control theory, and controllers that use neural networks. In addition, some control devices use both controllers that do not include neural networks and controllers that include neural networks.

[0003] Patent Document 1 discloses a technology related to a control device that uses both a PID controller and a controller using a neural network. This technology aims to improve control accuracy during positioning operations by switching between multiple neural networks depending on the driving state of the controlled object. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 7466403 Summary of the Invention [Problem to be solved by the invention]

[0005] However, conventional network-based controllers have multiple neural networks, which requires a large amount of memory and requires training of multiple neural networks.

[0006] The present invention has been made in view of the problems of the conventional technology, and has an exemplary object to provide a technology that is advantageous in suppressing control deviation. [Means for solving the problem]

[0007] In order to achieve the above-mentioned object, a control method as one aspect of the present invention is a control method for controlling a drive device, comprising a first step of performing machine learning of parameters used to output a control signal for controlling the drive device, and a second step of controlling the drive device in accordance with a control signal output using the parameters, characterized in that in the first step, the machine learning is performed by evaluating the control results of each of a plurality of sections in which the drive device has different driving states using different evaluation reference values ​​set according to each of the plurality of sections.

[0008] Further objects and other aspects of the present invention will become apparent from the following description of the embodiments with reference to the accompanying drawings. [Effects of the Invention]

[0009] According to the present invention, for example, it is possible to provide a technique that is advantageous in suppressing control deviation. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram showing the configuration of a control system to which the present invention can be applied. [Figure 2] FIG. 10 is a diagram illustrating a hardware configuration of a control device when the control system is applied to a positioning device. [Figure 3] FIG. 2 is a control block diagram for controlling the positioning device. [Figure 4] 1 is a flowchart for explaining learning of a neural network by machine learning. [Figure 5] FIG. 10 is a diagram illustrating an example of evaluation information indicating a correlation between a control deviation of the positioning device and an evaluation value. [Figure 6] 10A and 10B are diagrams for explaining an example of a method for defining a driving state of a positioning device. [Figure 7] FIG. 10 is a diagram illustrating the relationship between the driving state of the positioning device and the control deviation. [Figure 8]10A and 10B are diagrams for explaining how the evaluation reference value is changed depending on the driving state of the positioning device. [Figure 9] 10A and 10B are diagrams for explaining how the evaluation reference value is changed depending on the driving state of the positioning device and the elapsed time. [Figure 10] FIG. 2 is a control block diagram for controlling the positioning device. [Figure 11] FIG. 1 is a schematic diagram showing the configuration of an exposure apparatus. [Figure 12] 10 is a flowchart for explaining an exposure process in the exposure apparatus. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the invention according to the claims. Although multiple features are described in the embodiments, not all of these multiple features are necessarily essential to the invention, and multiple features may be combined arbitrarily. Furthermore, in the accompanying drawings, the same reference numerals are used to designate the same or similar components, and redundant explanations will be omitted.

[0012] 1 is a diagram showing the configuration of a control system CS to which the present invention can be applied. The control system CS is a system for controlling a control target, and includes a control device 1, a control server 2, and a learning server 3 (information processing device).

[0013] The learning server 3 (learning unit) uses machine learning to learn and determine (update) the control parameters of the controller configured inside the control device 1 via the control server 2. The control server 2 issues control commands to the control device 1 to control the controlled object, and obtains the control results resulting from controlling the controlled object from the control device 1. The control results obtained by the control server 2 are provided by the control server 2 to the learning server 3. The learning server 3 calculates a reward according to the control result provided by the control server 2, and updates the control parameters of the controller that learns through machine learning based on this reward.

[0014] In the control system CS, the computational cost related to the control parameters of the controllers in the learning server 3 is high, so in this embodiment, the control server 2 and the learning server 3 are configured independently. This makes it possible, when the control system CS has multiple control devices 1, to have multiple learning servers 3 with high computational costs and one control server 2 with low computational costs, thereby reducing the number of servers that make up the control system CS.

[0015] 2 is a diagram showing the hardware configuration of a control device 1 when the control system CS is applied to a positioning device. The control device 1 is a device that controls a positioning device 4 (drive device) that is the object to be controlled, and is composed of a sensor 5, a control board 6, and a driver 7.

[0016] The control board 6 provides the driver 7 with a control signal for controlling the positioning device 4, specifically, a current command as a control command (operation command) at predetermined time intervals. The driver 7 outputs (supplies) a current according to the current command from the control board 6 to an actuator (not shown) configured inside the positioning device 4. The sensor 5 acquires position information of the positioning device 4 as a result (positioning result) of the positioning device 4 being driven by the current supplied from the driver 7. The position information of the positioning device 4 acquired by the sensor 5 is provided from the sensor 5 to the control board 6.

[0017] FIG. 3 is a control block diagram for controlling the positioning device 4. As described above, the control board 6 acquires position information of the positioning device 4 from the sensor 5. The control board 6 calculates the difference (control deviation) between the control command for the positioning device 4 and the position information of the positioning device 4, and inputs the difference to a controller (neural network controller) 9 using a neural network. The control deviation may include past control deviations (history) acquired at predetermined time intervals. For example, if the neural network has 20 inputs, the inputs can include the control deviation at the time of neural network calculation (learning) (the current control deviation) and the control deviations up to 19 times before (the past 19 control deviations). The neural network controller 9 performs control so as to reduce the current control deviation based on the trend of the past control deviations. Therefore, the positioning device 4 is controlled in accordance with the control signal output from the neural network controller 9 (neural network parameters).

[0018] In this embodiment, the controller in the control board 6 is a controller using a neural network, but this is not limited to this and a controller capable of updating parameters through machine learning may also be used. The following describes an example in which a neural network is used. Also, a PID controller (feedback control) may be configured, and the output (control signal) of the PID controller and the output (output signal) of the neural network controller 9 may be added together to output to the driver 7.

[0019] A current command given from the control board 6 to the driver 7 is converted into a current by the driver 7 and supplied to the positioning device 4. The current supplied from the driver 7 drives an actuator configured inside the positioning device 4. The sensor 5 acquires position information of the positioning device 4 as a positioning result and provides it to the control board 6.

[0020] Here, the neural network must be trained in advance by some method in the neural network controller 9. As a neural network training method, for example, learning of the neural network (control parameters) by machine learning can be used.

[0021] Learning of a neural network by machine learning will be described with reference to Figure 4. In S401, the learning server 3 changes the parameters of the neural network in the neural network controller 9 (control board 6). If this is the first time that the neural network to be learned has been learned, the parameters of the neural network are initialized to their initial values.

[0022] In S402, the control board 6 (control device 1) drives (operates) the positioning device 4 (control target) by providing a current command, which is a control command, to the driver 7. In S403, the control board 6 (control device 1) acquires the control deviation of the positioning device 4. Specifically, the control board 6 acquires position information of the positioning device 4, which is the result of driving the positioning device 4 (positioning result), via the sensor 5, and acquires the control deviation by calculating the difference between the control command (current command) for the positioning device 4 and the position information of the positioning device 4. In this way, the control board 6 and the sensor 5 function as an acquisition unit that acquires the control deviation of the positioning device 4 as the control result of the positioning device 4. The control board 6 provides the control deviation of the positioning device 4 acquired from the sensor 5 to the learning server 3 via the control server 2.

[0023] In S404, the learning server 3 calculates a reward based on the control deviation of the positioning device 4 acquired in S403 and a reference evaluation value. Specifically, the control deviation of the positioning device 4 acquired in S403 is compared with evaluation information indicating the correlation between the control deviation of the positioning device 4 and an evaluation value, as shown in FIG. 5, to obtain an evaluation value corresponding to the control deviation of the positioning device 4 acquired in S403. The evaluation information is information for converting the control deviation of the positioning device 4 into an evaluation value. The evaluation information is set so that an evaluation value is obtained when the control deviation of the positioning device 4 is smaller than the evaluation reference value, and so that a higher evaluation value is obtained as the control deviation of the positioning device 4 becomes smaller. The control deviation of the positioning device 4 can be acquired at regular intervals while the positioning device 4 is being driven, so that a reward can be obtained using multiple control deviations. For example, evaluation values ​​corresponding to multiple control deviations can be obtained, and the sum of these evaluation values ​​can be used as the reward.

[0024] Here, as will be described later, the neural network controller 9 can change its output by learning the neural network while changing the evaluation reference value depending on the driving state of the positioning device 4. Details of changing the evaluation reference value depending on the driving state of the positioning device 4 will be described in the first embodiment.

[0025] Furthermore, the drive state of the positioning device 4 can be defined by, for example, the jerk, acceleration, or speed of the positioning device 4. Fig. 6 is a diagram for explaining an example of a method for defining the drive state of the positioning device 4. In Fig. 6, the horizontal axis represents time, and the vertical axis represents speed (upper graph) or acceleration (lower graph). In this embodiment, as shown in Fig. 6, a case will be described in which the drive state of the positioning device 4 is defined in interval 1 (601) to interval 8 (608).

[0026] Referring to FIG. 6, Section 1 (601) is a section in which the acceleration of the positioning device 4 gradually increases until it reaches a constant positive acceleration (constant acceleration), and the driving state of the positioning device 4 can be defined as a jerk driving state in which the acceleration increases in a positive direction. Section 2 (602) is a section in which the positioning device 4 maintains a constant positive acceleration, and the driving state of the positioning device 4 can be defined as a positive constant acceleration driving state. Section 3 (603) is a section in which the acceleration of the positioning device 4 gradually decreases until it reaches a constant velocity (constant speed), and the driving state of the positioning device 4 can be defined as a jerk driving state in which the acceleration decreases. Section 4 (604) is a section in which the positioning device 4 maintains a constant speed, and the driving state of the positioning device 4 can be defined as a constant velocity driving state in which the speed is maintained. Section 5 (605) is a section in which the acceleration of the positioning device 4 gradually increases until it reaches a constant negative acceleration (constant acceleration), and the driving state of the positioning device 4 can be defined as a jerk driving state in which the acceleration increases in a negative direction. Section 6 (606) is a section in which the positioning device 4 maintains a constant negative acceleration, and the driving state of the positioning device 4 can be defined as a driving state of constant negative acceleration. Section 7 (607) is a section in which the acceleration is reduced until the positioning device 4 stops, and the driving state of the positioning device 4 can be defined as a jerk driving state in which the acceleration is reduced. Section 8 (608) is a section in which the positioning device 4 stops, and the driving state of the positioning device 4 can be defined as a stopped driving state.

[0027] FIG. 7 is a diagram showing the relationship between the driving state of the positioning device 4 and the control deviation. In FIG. 7, the horizontal axis represents time, and the vertical axis represents speed (upper graph) or control deviation (lower graph). Note that FIG. 7 shows the control deviation of the positioning device 4 in the absence of the neural network controller 9. Also, section 1 (601) to section 8 (608) are the same as section 1 (601) to section 8 (608) shown in FIG. 6.

[0028] Referring to FIG. 7, the control deviation of the positioning device 4 increases in sections where the positioning device 4 is in a jerk drive state or a constant acceleration drive state, and decreases in sections where the positioning device 4 is in a constant velocity drive state. As described above, the sections where the positioning device 4 is in a jerk drive state correspond to section 1 (601), section 3 (603), section 5 (605), and section 7 (607). The sections where the positioning device 4 is in a constant acceleration drive state correspond to section 2 (602) and section 6 (606). The sections where the positioning device 4 is in a constant velocity drive state correspond to section 4 (604) and section 8 (608), which are sections where acceleration is zero. Thus, the control deviation of the positioning device 4 depends on acceleration and is related to the application of force to the positioning device 4. There are various error factors in the actuator that generates force, and these error factors cause the control deviation of the positioning device 4. Therefore, the control deviation of the positioning device 4 becomes small in the section where the acceleration is zero, and becomes large in other sections (sections where the acceleration is not zero).

[0029] Returning to FIG. 4, in S405, the learning server 3 determines whether the number of times the neural network has been trained is equal to or less than a specified number. If the number of times the neural network has been trained is equal to or less than the specified number, the process proceeds to S401, where the learning server 3 changes the neural network parameters in the neural network controller 9. In this case, the learning server 3 changes the neural network parameters according to a predetermined algorithm so as to increase the reward calculated in S404. On the other hand, if the number of times the neural network has been trained is not equal to or less than the specified number, i.e., if the number of times exceeds the specified number, the process proceeds to S406. In this way, the loop from S401 to S405 continues until the number of times the neural network has been trained reaches the specified number.

[0030] In S406, the learning server 3 updates the neural network parameters to the parameters that have obtained the maximum reward among the rewards calculated in S404. In other words, the learning server 3 updates the neural network parameters so that they are improved through machine learning. The neural network parameters updated in this way are saved in the learning server 3, the control server 2, or the control device 1 (control board 6).

[0031] First Embodiment Referring to Fig. 8, it will be described how the evaluation reference value is changed depending on the driving state of the positioning device 4 when training the neural network. In Fig. 8, the horizontal axis represents time, and the vertical axis represents the control deviation of the positioning device 4. As described above, the evaluation reference value is used to calculate a reward for the control deviation of the positioning device 4 (S404).

[0032] The first drive state 701, second drive state 702, and third drive state 703 of the positioning device 4 can each be associated with one of Section 1 (601) to Section 8 (608) shown in Figure 6. For example, consider a case where the drive states of the positioning device 4 are associated with Section 1 (601) to Section 8 (608) so as to reduce the control deviation in Section 4 (604), where the positioning device 4 maintains a constant speed. In this case, the first drive state 701 is associated with Section 2 (602) as the first section, the second drive state 702 is associated with Section 3 (603) as the second section, and the third drive state 703 is associated with Section 4 (604) as the third section.

[0033] In this embodiment, as shown in Fig. 8, the evaluation reference value is set to decrease in the order of the first driving state 701, the second driving state 702, and the third driving state 703, and neural network learning is performed. This reduces the control deviation in accordance with the driving state of the positioning device 4, and in the third driving state 703 (section 4 (604)), a neural network controller 9 is obtained that can suppress the control deviation to a value that satisfies the positioning accuracy (target value) required of the positioning device 4. Furthermore, by varying the evaluation reference value depending on the driving state of the positioning device 4 (for each section), a neural network controller 9 that outputs control in accordance with the driving state of the positioning device 4 is obtained. Therefore, a single neural network controller 9 can suppress the control deviation of the positioning device 4 in accordance with the driving state of the positioning device 4.

[0034] Specific settings of the evaluation reference value will be described. In this embodiment, the evaluation reference value used in calculating the reward for the control deviation of the positioning device 4 (S404) is set to gradually decrease in the order of the first driving state 701, the second driving state 702, and the third driving state 703. Specifically, in the first driving state 701, an evaluation reference value 711 (first evaluation reference value) is set to evaluate the control deviation of the positioning device 4 in section 2 (602). In the second driving state 702, an evaluation reference value 712 (second reference evaluation value) smaller than the evaluation reference value 711 is set to evaluate the control deviation of the positioning device 4 in section 3 (603). In the third driving state 703, an evaluation reference value 713 (third evaluation reference value) smaller than the evaluation reference value 712 is set to evaluate the control deviation of the positioning device 4 in section 4 (604). In other words, the evaluation reference values ​​711 and 712 corresponding to the first driving state 701 and the second driving state 702, respectively, which are sections where acceleration is high, are set to be larger than the evaluation reference value 713 corresponding to the third driving state 703, which is a section where acceleration is zero.

[0035] Without the neural network controller 9, the change in acceleration (change in actuator output) of the positioning device 4 decreases, and the control deviation also decreases in the order of the first drive state 701, the second drive state 702, and the third drive state 703. The neural network controller 9 receives the control deviation (history) of the positioning device 4 as input. The neural network controller 9 determines the output so as to reduce the current control deviation of the positioning device 4 based on the trend of the past control deviation of the positioning device 4. Here, consider a case where the neural network is trained by setting the evaluation reference values ​​711 and 712 corresponding to the first drive state 701 and the second drive state 702, respectively, to small values. In this case, the neural network controller 9 outputs control that strongly suppresses the control deviation in the first drive state 701 and the second drive state 702, but outputs excessive control in the third drive state 703, increasing the control deviation. Therefore, as in this embodiment, it is advantageous to perform neural network training by setting the evaluation reference values ​​711 and 712 corresponding to the first driving state 701 and the second driving state 702, respectively, to be greater than the evaluation reference value 713 corresponding to the third driving state 703. The evaluation reference value 713 corresponding to the third driving state 703 should preferably be set to a level equivalent to the positioning accuracy required of the positioning device 4.

[0036] In this way, in this embodiment, machine learning of the neural network is performed by evaluating the control deviation for each of a plurality of sections in which the driving state of the positioning device 4 is different, using a different evaluation reference value set according to each of the plurality of sections. This makes it possible to realize, with a single controller, the neural network controller 9 that can reduce the control deviation according to the driving state of the positioning device 4 and suppress the control deviation to a level that satisfies the positioning accuracy required of the positioning device 4.

[0037] In this embodiment, the driving states of the positioning device 4 have been described as three different driving states (sections), but the number of driving states is not limited as long as there are two or more driving states.

[0038] The driving state of the positioning device 4 can also be defined by the time that has elapsed since the positioning device 4 started to drive, and for example, can also be defined by adding a time offset to the above-mentioned interval.

[0039] For example, consider a case where 10 msec of settling time is required for the control deviation to converge in section 4 (604), where the positioning device 4 maintains a constant speed. In this case, section 4 (604) can be divided into section 4A, which lasts until 10 msec after reaching section 4, and section 4B, which lasts until section 5 (605) after section 4A. Section 4A is a section where less positioning accuracy is required than section 4B.

[0040] Changing the evaluation reference value depending on the drive state of the positioning device and elapsed time will be described with reference to Fig. 9. In Fig. 9, the horizontal axis represents time, and the vertical axis represents the control deviation of the positioning device 4. For example, with reference to Fig. 9, a first drive state 1201, a second drive state 1202, a third drive state 1203, and a fourth drive state 1204 of the positioning device 4 are associated with section 2 (602), section 3 (603), section 4A (fourth section), and section 4B (fifth section), respectively.

[0041] As shown in Figure 9, the neural network is trained by setting the evaluation reference value to decrease in the order of first drive state 1201, second drive state 1202, third drive state 1203, and fourth drive state 1204. This reduces the control deviation in accordance with the drive state of the positioning device 4, and in the fourth drive state 1204 (section 4B), a neural network controller 9 is obtained that can suppress the control deviation to a value that achieves the positioning accuracy (target value) required of the positioning device 4. Furthermore, by varying the evaluation reference value (for each section) depending on the drive state of the positioning device 4, a neural network controller 9 that outputs control in accordance with the drive state of the positioning device 4 is obtained. Therefore, a single neural network controller 9 can suppress the control deviation of the positioning device 4 in accordance with the drive state of the positioning device 4.

[0042] Specific settings of the evaluation reference values ​​will be described. In Fig. 9, the evaluation reference values ​​used in calculating the reward for the control deviation of the positioning device 4 (S404) are set to become smaller in stages in the order of a first driving state 1201, a second driving state 1202, a third driving state 1203, and a fourth driving state 1204. Specifically, an evaluation reference value 1211 is set in the first driving state 1201, and an evaluation reference value 1212 smaller than the evaluation reference value 1211 is set in the second driving state 1202. Furthermore, an evaluation reference value 1213 (fourth evaluation value) smaller than the evaluation reference value 1212 is set in the third driving state 1203, and an evaluation reference value 1214 (fifth evaluation value) smaller than the evaluation reference value 1213 is set in the fourth driving state 1204. In other words, the evaluation reference values ​​1211 and 1212 corresponding to the first driving state 1201 and the second driving state 1202, respectively, are set to be greater than the evaluation reference values ​​1213 and 1214 corresponding to the third driving state 1203 and the fourth driving state 1204, respectively.

[0043] In this way, the driving state of the positioning device 4 can be defined not only by the jerk, acceleration, or velocity, but also by the time that has elapsed since the positioning device 4 started to drive.

[0044] Furthermore, the correspondence between the drive states of the positioning device 4 and the sections is not limited to the above. For example, the drive states of the positioning device 4 may be associated with the sections so as to suppress the control deviation in section 8 (608) where the positioning device 4 is stopped. Specifically, in Fig. 8, the first drive state 701 is associated with section 6 (606) as the first section, the second drive state 702 is associated with section 7 (607) as the second section, and the third drive state 703 is associated with section 8 (608) as the third section.

[0045] 8, the neural network is trained by setting the evaluation reference value to decrease in the order of the first driving state 701, the second driving state 702, and the third driving state 703. This reduces the control deviation in accordance with the driving state of the positioning device 4, and in the third driving state 703 (section 8 (608)), a neural network controller 9 is obtained that can suppress the control deviation to a value that satisfies the positioning accuracy (target value) required of the positioning device 4. Furthermore, by varying the evaluation reference value (for each section) depending on the driving state of the positioning device 4, a neural network controller 9 that outputs control in accordance with the driving state of the positioning device 4 is obtained. Therefore, a single neural network controller 9 can suppress the control deviation of the positioning device 4 in accordance with the driving state of the positioning device 4.

[0046] Furthermore, when calculating a reward for the control deviation of the positioning device 4, weighting may be performed according to the drive state of the positioning device 4 (i.e., for each section). For example, the reward may be calculated as (α × sum of evaluation values ​​of the first drive state 701) + (β × sum of evaluation values ​​of the second drive state 702) + (γ × sum of evaluation values ​​of the third drive state 703) using weighting coefficients α, β, and γ. In this case, by increasing any of the weighting coefficients α, β, and γ corresponding to a drive state (section) in which it is desired to suppress the control deviation of the positioning device 4, machine learning will proceed so as to suppress the control deviation in that drive state.

[0047] Second Embodiment In the first embodiment, a case has been described in which the input to the neural network controller 9 is the control deviation of the positioning device 4. In the second embodiment, a case will be described in which the input to the neural network controller 9 is the control deviation of the positioning device 4 and the driving state of the positioning device 4.

[0048] FIG. 10 is a control block diagram for controlling the positioning device 4. The control board 6 acquires position information of the positioning device 4 from the sensor 5. The control board 6 calculates the difference (control deviation) between the control command for the positioning device 4 and the position information of the positioning device 4, and inputs it to a controller 9 using a neural network (neural network controller). The control deviation may include past control deviations (history) acquired at predetermined time intervals. In this embodiment, in addition to the control deviation of the positioning device 4, the driving state of the positioning device 4 is input to the neural network controller 9.

[0049] In this embodiment, the controller in the control board 6 is a controller using a neural network, but this is not limited to this and a controller capable of updating parameters through machine learning may also be used. The following describes an example in which a neural network is used. Alternatively, a PID controller may be configured, and the output of the PID controller and the output of the neural network controller 9 may be added together to output to the driver 7.

[0050] In this embodiment, the driving state of the positioning device 4, which is input to the neural network controller 9, is, as described above, a state defined by the jerk, acceleration, or velocity of the positioning device 4. In this way, by adding the driving state of the positioning device 4 to the input of the neural network controller 9, the output of the neural network controller 9 can be changed depending on the driving state of the positioning device 4.

[0051] A specific example will be described below in which the driving state of the positioning device 4 added to the input of the neural network controller 9 is the acceleration of the positioning device 4. In this example, in FIG. 8, the first driving state 701 corresponds to section 6 (606), the second driving state 702 corresponds to section 7 (607), and the third driving state 703 corresponds to section 8 (608). Also, the maximum acceleration of the positioning device 4 is assumed to be 1.0 G. If the neural network has 20 inputs, 19 of them are input to the control deviation (history) of the positioning device 4, and the remaining one is input to the acceleration of the positioning device 4.

[0052] When the positioning device 4 is in the first drive state 701, the acceleration of the positioning device 4 is input to the neural network as an input value of −1.0. When the positioning device 4 is in the second drive state 702, the acceleration of the positioning device 4 is input to the neural network as an input value between −1.0 and 0.0. When the positioning device 4 is in the third drive state 703, the acceleration of the positioning device 4 is input to the neural network as an input value of 0.0.

[0053] By inputting the acceleration of the positioning device 4 into the neural network as the driving state of the positioning device 4, it is possible to identify the driving state (section) of the positioning device 4 from the change in the acceleration, which can then be reflected in the output of the neural network.

[0054] The drive state of the positioning device 4 to be added to the input of the neural network controller 9 can also be expressed by a one-hot vector. Specifically, the drive state of the positioning device 4 is expressed using "0" and "1." Here, in FIG. 8, the first drive state 701 is associated with section 6 (606), the second drive state 702 is associated with section 7 (607), and the third drive state 703 is associated with section 8 (608). If the neural network has 20 inputs, 17 of them are input as the control deviation (history) of the positioning device 4, and the remaining 3 are input as one-hot vectors, which are information representing the drive state of the positioning device 4.

[0055] When the positioning device 4 is in the first drive state 701, (1, 0, 0) representing the first drive state 701 is used as the input to the neural network. When the positioning device 4 is in the second drive state 702, (0, 1, 0) representing the second drive state 702 is used as the input to the neural network. When the positioning device 4 is in the third drive state 703, (0, 0, 1) representing the third drive state 703 is used as the input to the neural network.

[0056] In this way, by expressing the driving state of the positioning device 4 as a one-hot vector, it is possible to input to the neural network only information indicating the driving state of the positioning device 4. As a result, it becomes possible to reflect this in the output of the neural network, and by adjusting the output of the neural network controller 9, it is possible to gradually reduce the control deviation of the positioning device 4.

[0057] <Third embodiment> An exposure apparatus 900 according to one aspect of the present invention will be described with reference to FIG. 11. FIG. 11 is a schematic diagram showing the configuration of the exposure apparatus 900. The exposure apparatus 900 is a lithography apparatus used in, for example, a lithography process, which is a manufacturing process for devices such as semiconductor elements, and forms a pattern on a substrate. The exposure apparatus 900 employs a step-and-scan method in which the substrate is scanned and exposed with light shaped by a slit. Therefore, the exposure apparatus 900 is embodied as a scanning exposure apparatus (scanner) that transfers the pattern of the original onto the substrate by exposing the substrate while scanning the original and the substrate relative to each other. The control using the neural network (neural network controller 9) described in the first and second embodiments can be applied to control each part of the exposure apparatus 900.

[0058] As shown in FIG. 11, the exposure apparatus 900 has an illumination optical system 23, an original stage 12, a projection optical system 13, a substrate stage 15, an original position measurement unit 17, a substrate position measurement unit 18, a mark measurement unit 21, a substrate transport unit 22, and a control unit 24.

[0059] In this embodiment, directions are shown in an XYZ coordinate system in which the direction along the optical axis of the projection optical system 13 is defined as the Z axis, and two directions parallel to a plane perpendicular to the Z axis and perpendicular to each other are defined as the X axis and the Y axis. Furthermore, directions parallel to the X axis, Y axis, and Z axis in the XYZ coordinate system are defined as the X direction, Y direction, and Z direction, respectively.

[0060] The illumination optical system 23 illuminates the original 11 with light from a light source (not shown). For example, the illumination optical system 23 shapes the light from the light source into light having a strip-like or arc-like shape that is long in the X direction using a light-blocking member such as a masking blade, and illuminates a part of the original 11 with this light.

[0061] The original stage 12 holds the original 11. The substrate stage 15 holds the substrate 14. The original 11 held by the original stage 12 and the substrate 14 held by the substrate stage 15 are arranged at optically approximately conjugate positions (the object plane and image plane of the projection optical system 13) via the projection optical system 13.

[0062] The projection optical system 13 has a predetermined projection magnification, such as 1 / 2 or 1 / 4, and projects the pattern of the original 11 onto the substrate 14. The area of ​​the substrate 14 onto which the pattern of the original 11 is projected (the area illuminated by light from the original 11) is called the illumination area.

[0063] The substrate stage 15 is configured to be movable in the X and Y directions, which are directions perpendicular to the optical axis of the projection optical system 13. The substrate stage 15 includes independent actuators for each direction so that it can move in each of the X and Y directions. The original stage 12 is configured to be movable in the Y direction.

[0064] The original stage 12 and the substrate stage 15 move relatively in synchronization with each other at a speed ratio according to the projection magnification of the projection optical system 13 under the control of the control unit 24. As a result, the substrate 14 is scanned in the Y direction relative to the irradiation area, and the pattern of the original 11 is transferred to the shot area on the substrate. This scanning exposure is performed sequentially for each of the multiple shot areas on the substrate while moving the substrate stage 15 in steps, thereby completing the exposure process for one substrate 14.

[0065] The original position measuring unit 17 includes, for example, a laser interferometer and measures the position of the original stage 12. The original position measuring unit 17 irradiates a reflecting plate provided on the original stage 12 with laser light and detects the displacement of the original stage 12 (displacement from a reference position) from the interference between the laser light reflected by the reflecting plate and the laser light reflected by a reference surface. Then, the original position measuring unit 17 obtains the current position of the original stage 12 based on the displacement of the original stage 12. Note that the original position measuring unit 17 is not limited to a laser interferometer, and the position of the original 11 may be measured by, for example, an encoder.

[0066] The substrate position measuring unit 18 includes, for example, a laser interferometer, and measures the position of the substrate stage 15. The substrate position measuring unit 18 irradiates a reflecting plate provided on the substrate stage 15 with laser light, and detects the displacement of the substrate stage 15 (displacement from a reference position) from the interference between the laser light reflected by the reflecting plate and the laser light reflected by a reference surface. The substrate position measuring unit 18 then obtains the current position of the substrate stage 15 based on the displacement of the substrate stage 15. Note that the substrate position measuring unit 18 is not limited to a laser interferometer, and the position of the substrate stage 15 may be measured by, for example, an encoder.

[0067] The mark measurement unit 21 includes, for example, an imaging element, and measures the position of a mark provided on the substrate. In this embodiment, the mark measurement unit 21 detects the mark using an imaging element, but is not limited to this, and the mark may be detected using, for example, a transmission sensor.

[0068] The substrate transport unit 22 has a function of transporting the substrate 14. In this embodiment, the substrate transport unit 22 supplies the substrate 14 to the substrate stage 15 and retrieves the substrate 14 from the substrate stage 15.

[0069] The control unit 24 is configured by an information processing device (computer) including, for example, a CPU, a memory, etc., and controls each part of the exposure apparatus 900 in accordance with a program stored in the storage unit to operate the exposure apparatus 900. The control unit 24 controls the exposure process for transferring the pattern of the original 11 onto the substrate 14 (exposure process for scanning and exposing the substrate 14).

[0070] Specifically, the control unit 24 is configured by a PLD (abbreviation for Programmable Logic Device) such as an FPGA (abbreviation for Field Programmable Gate Array), an ASIC (abbreviation for Application Specific Integrated Circuit), a general-purpose computer with an embedded program, or a combination of all or part of these. The control unit 24 also includes a driver that controls the actuator.

[0071] The operation of exposure apparatus 900 will be described with reference to Figure 12. Figure 12 is a flowchart for explaining the exposure process in exposure apparatus 900. As described above, this exposure process is performed by control unit 24 comprehensively controlling each unit of exposure apparatus 900.

[0072] In S1001, the substrate 14 is loaded into the exposure apparatus 900. Specifically, the substrate transport unit 22 supplies the substrate 14 to the substrate stage 15. In S1002, alignment measurement is performed. Specifically, the substrate stage 15 is driven so that a mark (a mark defined in the exposure recipe) provided on the substrate 14 is positioned within the measurement field of the mark measurement unit 21, and the mark measurement unit 21 measures the position of the mark.

[0073] In S1003, the substrate 14 is subjected to scanning exposure. Specifically, while the original stage 12 and the substrate stage 15 are moved synchronously, the pattern of the original 11 is projected onto the substrate 14 via the projection optical system 13, and multiple shot areas on the substrate are sequentially exposed. At this time, the substrate 14 is scanned and exposed in accordance with the exposure sequence and exposure angle of view defined in the exposure recipe. In S1004, the substrate 14 is unloaded from the exposure apparatus 900. Specifically, the substrate transport unit 22 retrieves the substrate 14 from the substrate stage 15.

[0074] In exposure apparatus 900, control using a neural network, i.e., neural network controller 9, can be applied to substrate stage 15, for example. In this case, control board 6 corresponds to control unit 24, driver 7 corresponds to control unit 24, sensor 5 corresponds to substrate position measurement unit 18, and positioning device 4 corresponds to substrate stage 15. The actuator is a linear motor (not shown) configured in substrate stage 15. By applying neural network controller 9 to substrate stage 15, it is possible to shorten the time until the control deviation after driving substrate stage 15 converges and / or the settling time until a constant velocity is reached. The exposure accuracy (exposure performance) and throughput of exposure apparatus 900 can be improved.

[0075] When the neural network controller 9 is applied to the substrate stage 15, the driving state of the substrate stage 15 is defined by the jerk, acceleration, or velocity of the substrate stage 15 (determined from the interval).

[0076] In alignment measurement (S1002), it is necessary to reduce the control deviation immediately after the substrate stage 15 stops. This is because the mark measurement unit 21 measures the marks provided on the substrate 14 in section 8 (608) when the substrate stage 15 is in a stopped driving state. Therefore, it is preferable for the neural network controller 9 to start control from section 6 (606) when the substrate stage 15 maintains a constant negative acceleration, which is a driving state of a constant negative acceleration (a decelerating state). The vibration component excited in section 5 (605), where the acceleration of the substrate stage 15 is gradually increased until it reaches a constant negative acceleration, is smaller than the vibration component excited in section 6 (606), so the neural network controller 9 starts control from section 6 (606).

[0077] 8, the first driving state 701 is associated with section 6 (606), the second driving state 702 is associated with section 7 (607), and the third driving state 703 is associated with section 8 (608). The evaluation reference value is set to decrease in the order of the first driving state 701, the second driving state 702, and the third driving state 703, and neural network learning is performed. This makes it possible to realize, with a single controller, a neural network controller 9 that can reduce the control deviation in accordance with the driving state of the substrate stage 15 and suppress the control deviation to a level that satisfies the required accuracy in the third driving state 703 (section 8 (608)).

[0078] In alignment measurement (S1002), the substrate stage 15 is driven in the horizontal direction. Therefore, it is preferable to apply the neural network controller 9 to each of the multiple drive axes (control axes) of the substrate stage 15, specifically, the drive axes in the X and Y directions. Furthermore, if the required accuracy differs between the X and Y directions, different evaluation reference values ​​may be set for each of the X and Y directions.

[0079] In scanning exposure (S1003), it is necessary to reduce the control deviation in a constant speed driving state where the substrate stage 15 maintains a constant speed. This is because the substrate 14 is scanned and exposed in section 4 (604) where the substrate stage 15 is driven at a constant speed.

[0080] In scanning exposure (S1003), the neural network controller 9 is applied to the Y-direction drive axis of the substrate stage 15. The neural network controller 9 preferably starts control from section 2 (602) where a constant positive acceleration is maintained, which is a drive state (accelerating state) of constant positive acceleration. The vibration component excited in section 1 (601), where the acceleration of the substrate stage 15 is gradually increased until it reaches a constant positive acceleration, is smaller than the vibration component excited in section 2 (602), so the neural network controller 9 starts control from section 2 (602).

[0081] 8, the first driving state 701 is associated with section 2 (602), the second driving state 702 is associated with section 3 (603), and the third driving state 703 is associated with section 4 (604). The evaluation reference value is set to decrease in the order of the first driving state 701, the second driving state 702, and the third driving state 703, and neural network learning is performed. This makes it possible to realize, with a single controller, a neural network controller 9 that can reduce the control deviation in accordance with the driving state of the substrate stage 15 and suppress the control deviation to a level that satisfies the required accuracy in the third driving state 703 (section 4 (604)).

[0082] Regarding the drive axis of the substrate stage 15 in the X direction, it is necessary to stop driving the substrate stage 15 before scanning and exposing the substrate 14. Therefore, it is necessary to associate the drive states of the substrate stage 15 separately from the drive axis of the substrate stage 15 in the Y direction. Here, as in the alignment measurement (S1002), the first drive state 701 is associated with section 6 (606), the second drive state 702 is associated with section 7 (607), and the third drive state 703 is associated with section 8 (608). Then, the evaluation reference value is set to decrease in the order of the first drive state 701, the second drive state 702, and the third drive state 703, and the neural network is trained. This makes it possible to realize a neural network controller 9 that can reduce the control deviation according to the drive state of the substrate stage 15 and suppress the control deviation to a value that satisfies the required accuracy in the third drive state 703 (section 8 (608)) with a single controller.

[0083] In this way, it is possible to set the driving state of the substrate stage 15 for each driving axis in the X and Y directions of the substrate stage 15. Also, different evaluation reference values ​​can be set for each of the X and Y directions.

[0084] In loading the substrate 14 (S1001), it is necessary to reduce the control deviation immediately after stopping the substrate stage 15. Therefore, by applying the neural network controller 9 in the same way as in alignment measurement (S1002), the control deviation of the substrate stage 15 can be suppressed.

[0085] In unloading the substrate 14 (S1004), it is necessary to reduce the control deviation immediately after stopping the substrate stage 15. Therefore, by applying the neural network controller 9 in the same way as in alignment measurement (S1002), the control deviation of the substrate stage 15 can be suppressed.

[0086] In this way, by gradually decreasing the evaluation reference value for the driving state (section) in which the control deviation of the substrate stage 15 should be reduced and training the neural network, it is possible to obtain a neural network controller 9 that achieves the required accuracy.

[0087] In the exposure apparatus 900, the neural network controller 9 can also be applied to, for example, the original stage 12. In this case, the control board 6 corresponds to the control unit 24, the driver 7 corresponds to the control unit 24, the sensor 5 corresponds to the original position measurement unit 17, and the positioning device 4 corresponds to the original stage 12. The actuator is a linear motor (not shown) configured in the original stage 12.

[0088] When the neural network controller 9 is applied to the original stage 12, the driving state of the original stage 12 can be associated with the substrate stage 15 in the same manner as the substrate stage 15. However, the loading and unloading of the substrate 14 are equivalent to the loading and unloading of the original 11, respectively.

[0089] With regard to the master stage 12 as well, by gradually decreasing the evaluation reference value for the driving state (section) in which the control deviation of the master stage 12 should be reduced and training the neural network, a neural network controller 9 that achieves the required accuracy can be obtained.

[0090] In exposure apparatus 900, neural network controller 9 can also be applied to, for example, substrate transport unit 22. In this case, control board 6 corresponds to control unit 24, driver 7 corresponds to control unit 24, sensor 5 corresponds to a rotary encoder (not shown), and positioning device 4 corresponds to substrate transport unit 22. The actuator is an AC servo motor (not shown) configured in substrate transport unit 22.

[0091] By applying the neural network controller 9 to the substrate transport unit 22, it is possible to suppress control deviations during operation of the substrate transport unit 22 and improve the repeatability of the supply position when supplying the substrate 14 to the substrate stage 15. Furthermore, since it is possible to suppress control deviations while increasing the acceleration and speed of the substrate transport unit 22, it is possible to improve throughput.

[0092] When the neural network controller 9 is applied to the substrate transport section 22, the driving state of the substrate transport section 22 is defined by the jerk, acceleration or velocity of the substrate transport section 22 (determined from the section).

[0093] When the neural network controller 9 is applied to the substrate transport unit 22, the drive state of the substrate transport unit 22 can be associated with the substrate stage 15 in the same way. For the substrate transport unit 22 as well, the neural network can be trained by gradually decreasing the evaluation reference value for the drive state (section) in which the control deviation of the substrate transport unit 22 should be reduced, thereby obtaining a neural network controller 9 that achieves the required accuracy.

[0094] In the exposure apparatus 900, the neural network controller 9 can also be applied to, for example, the projection optical system 13. The projection optical system 13 includes, for example, a plurality of optical elements 131 such as lenses, and a drive unit 132 (drive device) that drives the optical elements 131. In this case, the control board 6 corresponds to the control unit 24, the driver 7 corresponds to the control unit 24, the sensor 5 corresponds to a capacitance sensor (not shown), and the positioning device 4 corresponds to the drive unit 132. The actuator is a piezo motor (not shown) configured in the drive unit 132.

[0095] The optical element 131 included in the projection optical system 13 is driven during scanning exposure of the substrate 14, so the positioning accuracy of the optical element 131 affects the exposure accuracy. By applying the neural network controller 9 to the projection optical system 13 (driver 132), it is possible to suppress control deviations during driving of the optical element 131 included in the projection optical system 13.

[0096] When the neural network controller 9 is applied to the projection optical system 13, the driving state of the driving unit 132 that drives the optical element 131 included in the projection optical system 13 is defined by the jerk, acceleration or velocity of the driving unit 132 (determined from the interval).

[0097] When the neural network controller 9 is applied to the projection optical system 13, it is possible to associate the drive state of the drive unit 132 that drives the optical element 131 included in the projection optical system 13, as with the substrate stage 15. Then, by gradually decreasing the evaluation reference value for the drive state (section) in which the control deviation of the drive unit 132 should be reduced and performing neural network learning, it is possible to obtain a neural network controller 9 that achieves the required positioning accuracy.

[0098] <Fourth embodiment> The method for manufacturing an article according to an embodiment of the present invention is suitable for manufacturing articles such as semiconductor devices, flat panel displays, liquid crystal display devices, and MEMS. This manufacturing method includes the steps of exposing a substrate coated with a photosensitive agent using the exposure apparatus 900 described above and developing the exposed photosensitive agent. The developed photosensitive agent pattern is then used as a mask to perform etching and ion implantation processes on the substrate, forming a circuit pattern on the substrate. These exposure, development, etching, and other processes are repeated to form a circuit pattern consisting of multiple layers on the substrate. In subsequent processes, the substrate on which the circuit pattern has been formed is diced, followed by chip mounting, bonding, and inspection. This manufacturing method may also include other well-known processes (e.g., oxidation, film formation, vapor deposition, doping, planarization, resist stripping, etc.). The method for manufacturing an article according to this embodiment is advantageous over conventional methods in at least one of the performance, quality, productivity, and production cost of the article.

[0099] The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0100] The disclosure of this specification includes the following control method, control device, learning method, information processing device, exposure apparatus, and method for manufacturing an article.

[0101] (Item 1) A control method for controlling a drive device, comprising: a first step of performing machine learning of parameters used to output a control signal for controlling the driving device; a second step of controlling the driving device in accordance with a control signal output using the parameter; and In the first step, the machine learning is performed by evaluating the control results for each of a plurality of sections in which the driving state of the driving device is different, using different evaluation reference values ​​set corresponding to each of the plurality of sections. A control method comprising:

[0102] (Item 2) 2. The control method according to item 1, wherein the driving state of the driving device is defined by a jerk, acceleration, or speed of the driving device.

[0103] (Item 3) 3. The control method according to item 1 or 2, wherein the driving state of the driving device is defined by the elapsed time since the driving device starts to drive.

[0104] (Item 4) The control method according to any one of items 1 to 3, wherein the plurality of sections include a section in which the acceleration of the drive device increases or decreases, a section in which the acceleration of the drive device maintains a constant acceleration, a section in which the speed of the drive device maintains a constant speed, and a section in which the drive device is stopped.

[0105] (Item 5) the plurality of sections include a first section in which the acceleration of the drive device maintains a constant acceleration, a second section after the first section in which the acceleration of the drive device decreases, and a third section after the second section in which the speed of the drive device maintains a constant speed; In the first step, the machine learning is performed by evaluating the control result of the first section using a first evaluation reference value, evaluating the control result of the second section using a second evaluation reference value that is smaller than the first evaluation reference value, and evaluating the control result of the third section using a third evaluation reference value that is smaller than the second evaluation reference value. 2. The control method according to item 1,

[0106] (Item 6) the plurality of sections include a first section in which the acceleration of the drive device is maintained at a constant rate, a second section after the first section in which the acceleration of the drive device decreases, and a third section after the second section in which the drive device is stopped; In the first step, the machine learning is performed by evaluating the control result of the first section using a first evaluation reference value, evaluating the control result of the second section using a second evaluation reference value that is smaller than the first evaluation reference value, and evaluating the control result of the third section using a third evaluation reference value that is smaller than the second evaluation reference value. 2. The control method according to item 1,

[0107] (Item 7) The third section includes a fourth section and a fifth section following the fourth section, the third evaluation reference value includes a fourth evaluation reference value and a fifth evaluation reference value that is smaller than the fourth evaluation reference value; In the first step, the machine learning is performed by evaluating the control result of the fourth section using the fourth evaluation reference value and evaluating the control result of the fifth section using the fifth evaluation reference value. 7. The control method according to item 5 or 6,

[0108] (Item 8) 8. The control method according to any one of items 1 to 7, wherein the parameters include parameters of a neural network.

[0109] (Item 9) 9. The control method according to item 8, wherein the neural network receives a control deviation of the drive device as an input.

[0110] (Item 10) 9. The control method according to item 8, wherein the neural network receives the control deviation of the drive device and the drive state of the drive device as inputs.

[0111] (Item 11) Item 11. The control method according to item 10, wherein the driving state of the driving device input to the neural network is expressed by a one-hot vector.

[0112] (Item 12) the control result includes a control deviation of the drive device; 12. The control method according to any one of items 1 to 11, wherein the evaluation reference value includes a target value of a control deviation of the drive device.

[0113] (Item 13) Item 13. The control method according to item 12, wherein in the first step, an evaluation value for the control deviation of the drive device is calculated based on a difference between the control deviation and the target value.

[0114] (Item 14) Item 14. The control method according to item 13, wherein in the first step, the parameters are updated by the machine learning so that the evaluation value becomes higher.

[0115] (Item 15) the drive unit includes a plurality of drive shafts; In the first step, machine learning of parameters used to output control signals for controlling each of the plurality of drive axes is performed. 15. The control method according to any one of items 1 to 14,

[0116] (Item 16) Item 16. The control method according to item 15, wherein the drive state of the drive device is set for each of the plurality of drive shafts.

[0117] (Item 17) 17. The control method according to item 15 or 16, wherein the evaluation reference value is set for each of the plurality of drive axes.

[0118] (Item 18) 18. The control method according to any one of items 1 to 17, wherein in the second step, the drive device is controlled in accordance with the control signal and an output signal of a feedback control having a control deviation of the drive device as an input.

[0119] (Item 19) A control device that controls a drive device, a controller including parameters used to output a control signal for controlling the drive device; a driver that controls the driving device in accordance with a control signal output using the parameters; and the parameters are parameters that have been machine-learned by evaluating control results for each of a plurality of sections in which the driving state of the driving device is different, using different evaluation reference values ​​that are set for each of the plurality of sections; A control device characterized by:

[0120] (Item 20) A learning method for learning parameters used to output a control signal for controlling a drive device, comprising: a first step of acquiring control results for each of a plurality of sections in which the driving state of the driving device is different; a second step of performing machine learning of the parameters based on the control results of each of the plurality of sections acquired in the first step; and In the second step, the machine learning is performed by evaluating the control results for each of the plurality of sections using different evaluation reference values ​​set according to each of the plurality of sections. A learning method characterized by:

[0121] (Item 21) An information processing device that learns parameters used to output a control signal for controlling a drive device, an acquisition unit that acquires control results for each of a plurality of sections in which the driving state of the driving device is different; a learning unit that performs machine learning of the parameters based on the control results for each of the plurality of sections acquired by the acquisition unit; and the learning unit performs the machine learning by evaluating the control results for each of the plurality of sections using different evaluation reference values ​​set in accordance with each of the plurality of sections. 1. An information processing device comprising:

[0122] (Item 22) A holding device for holding an object, a drive device that drives the object; a control unit that controls the drive device using the control method described in item 1; A holding device comprising:

[0123] (Item 23) An exposure apparatus that exposes a substrate through an original, a substrate stage including a drive device that drives the substrate; a control unit that controls the drive device using the control method described in item 1; An exposure apparatus comprising:

[0124] (Item 24) An exposure apparatus that exposes a substrate through an original, an original stage including a drive device for driving the original; a control unit that controls the drive device using the control method according to claim 1; An exposure apparatus comprising:

[0125] (Item 25) An exposure apparatus that exposes a substrate through an original, a substrate transport unit including a drive device that drives the substrate and transports the substrate to a substrate stage; a control unit that controls the drive device using the control method described in item 1; An exposure apparatus comprising:

[0126] (Item 26) An exposure apparatus that exposes a substrate through an original, a projection optical system that projects the pattern of the original onto the substrate; a driving device that drives an optical element included in the projection optical system; a control unit that controls the drive device using the control method described in item 1; An exposure apparatus comprising:

[0127] (Item 27) exposing a substrate using the exposure apparatus according to any one of items 23 to 26; developing the exposed substrate; manufacturing an article from the developed substrate; A method for manufacturing an article, comprising:

[0128] The invention is not limited to the above-described embodiments, and various changes and modifications can be made without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0129] 1: Control device 2: Control server 3: Learning server 4: Positioning device 5: Sensor 6: Control board 7: Driver 9: Neural network controller

Claims

1. A control method for controlling a drive device, comprising: a first step of performing machine learning of parameters used to output a control signal for controlling the driving device; a second step of controlling the driving device in accordance with a control signal output using the parameter; and In the first step, the machine learning is performed by evaluating control results for each of a plurality of sections in which the driving state of the driving device is different, using different evaluation reference values ​​set in accordance with each of the plurality of sections. A control method comprising:

2. 2. The control method according to claim 1, wherein the driving state of the driving device is defined by a jerk, an acceleration, or a velocity of the driving device.

3. 2. The control method according to claim 1, wherein the driving state of the driving device is defined by an elapsed time since the driving device starts to drive.

4. 2. The control method according to claim 1, wherein the plurality of sections include a section in which the acceleration of the drive device increases or decreases, a section in which the acceleration of the drive device maintains a constant acceleration, a section in which the speed of the drive device maintains a constant speed, and a section in which the drive device is stopped.

5. the plurality of sections include a first section in which the acceleration of the drive device maintains a constant acceleration, a second section after the first section in which the acceleration of the drive device decreases, and a third section after the second section in which the speed of the drive device maintains a constant speed, In the first step, the machine learning is performed by evaluating the control result of the first section using a first evaluation reference value, evaluating the control result of the second section using a second evaluation reference value that is smaller than the first evaluation reference value, and evaluating the control result of the third section using a third evaluation reference value that is smaller than the second evaluation reference value.

2. The control method according to claim 1.

6. the plurality of sections include a first section in which the acceleration of the drive device is maintained at a constant rate, a second section after the first section in which the acceleration of the drive device decreases, and a third section after the second section in which the drive device is stopped; In the first step, the machine learning is performed by evaluating the control result of the first section using a first evaluation reference value, evaluating the control result of the second section using a second evaluation reference value that is smaller than the first evaluation reference value, and evaluating the control result of the third section using a third evaluation reference value that is smaller than the second evaluation reference value.

2. The control method according to claim 1.

7. The third section includes a fourth section and a fifth section subsequent to the fourth section, the third evaluation reference value includes a fourth evaluation reference value and a fifth evaluation reference value that is smaller than the fourth evaluation reference value; In the first step, the machine learning is performed by evaluating the control result of the fourth section using the fourth evaluation reference value and evaluating the control result of the fifth section using the fifth evaluation reference value.

7. The control method according to claim 5 or 6.

8. 2. The method of claim 1, wherein the parameters include parameters of a neural network.

9. 9. The control method according to claim 8, wherein the neural network receives a control deviation of the drive device as an input.

10. 9. The control method according to claim 8, wherein the neural network receives the control deviation of the drive device and the drive state of the drive device as inputs.

11. 11. The control method according to claim 10, wherein the driving state of the driving device input to the neural network is expressed by a one-hot vector.

12. the control result includes a control deviation of the drive device; 2. The control method according to claim 1, wherein the evaluation reference value includes a target value of a control deviation of the drive device.

13. 13. The control method according to claim 12, wherein in the first step, an evaluation value for the control deviation of the drive device is calculated based on a difference between the control deviation and the target value.

14. 14. The control method according to claim 13, wherein in the first step, the parameters are updated by the machine learning so that the evaluation value becomes higher.

15. the drive unit includes a plurality of drive shafts; In the first step, machine learning of parameters used to output control signals for controlling each of the plurality of drive axes is performed.

2. The control method according to claim 1.

16. 16. The control method according to claim 15, wherein the drive state of the drive device is set for each of the plurality of drive shafts.

17. 16. The control method according to claim 15, wherein the evaluation reference value is set for each of the plurality of drive axes.

18. 2. The control method according to claim 1, wherein the second step includes controlling the drive device in accordance with the control signal and an output signal of a feedback control that inputs a control deviation of the drive device.

19. A control device that controls a drive device, a controller including parameters used to output a control signal for controlling the drive device; a driver that controls the driving device in accordance with a control signal output using the parameters; and the parameters are parameters that have been machine-learned by evaluating control results for each of a plurality of sections in which the driving state of the driving device is different, using different evaluation reference values ​​that are set for each of the plurality of sections; A control device characterized by:

20. A learning method for learning parameters used to output a control signal for controlling a drive device, comprising: a first step of acquiring control results for each of a plurality of sections in which the driving state of the driving device is different; a second step of performing machine learning of the parameters based on the control results for each of the plurality of sections acquired in the first step; and In the second step, the machine learning is performed by evaluating the control results for each of the plurality of sections using different evaluation reference values ​​set according to each of the plurality of sections. A learning method characterized by:

21. An information processing device that learns parameters used to output a control signal for controlling a drive device, an acquisition unit that acquires control results for each of a plurality of sections in which the driving state of the driving device is different; a learning unit that performs machine learning of the parameters based on the control results for each of the plurality of sections acquired by the acquisition unit; and the learning unit performs the machine learning by evaluating the control results for each of the plurality of sections using different evaluation reference values ​​set in accordance with each of the plurality of sections.

1. An information processing device comprising:

22. A holding device for holding an object, a drive device that drives the object; a control unit that controls the drive device using the control method according to claim 1; A holding device comprising:

23. An exposure apparatus that exposes a substrate through an original, a substrate stage including a drive device that drives the substrate; a control unit that controls the drive device using the control method according to claim 1; An exposure apparatus comprising:

24. An exposure apparatus that exposes a substrate through an original, an original stage including a drive device for driving the original; a control unit that controls the drive device using the control method according to claim 1; An exposure apparatus comprising:

25. An exposure apparatus that exposes a substrate through an original, a substrate transport unit including a drive device that drives the substrate and transports the substrate to a substrate stage; a control unit that controls the drive device using the control method according to claim 1; An exposure apparatus comprising:

26. An exposure apparatus that exposes a substrate through an original, a projection optical system that projects the pattern of the original onto the substrate; a driving device that drives an optical element included in the projection optical system; a control unit that controls the drive device using the control method according to claim 1; An exposure apparatus comprising:

27. exposing a substrate using an exposure apparatus according to any one of claims 23 to 26; developing the exposed substrate; manufacturing an article from the developed substrate; A method for manufacturing an article, comprising:

Images