Control method, control apparatus, lithography apparatus, and article manufacturing method

Abstract

The purpose of the present invention is to effectively perform learning for determining a parameter value of a neural network, and to improve control accuracy in control of a control object using the neural network. Control method for controlling an object to be controlled by a first signal (S1) generated by a first compensator (401) on the basis of a control deviation of the object to be controlled and a second signal (S2) generated by a second compensator (402) by a neural network on the basis of the control deviation, including a first control step for controlling the object to be controlled on the basis of a target value of a first sequence, while virtually disturbing (N) a second control step for controlling the control object on the basis of a target value of a second sequence while giving the control object; and a determination step for determining a parameter value of the neural network (402) by learning on the basis of a second index obtained by the second control step.

Description

Technical Field

[0001] The present invention relates to a control method, a control device, a lithography apparatus, and an article manufacturing method. Background Art

[0002] In recent years, the requirement for improving control accuracy has been becoming strict, and sometimes the required accuracy cannot be achieved only by conventional feedback control. Therefore, a structure is adopted in which, in addition to a conventional controller (compensator), a controller including a neural network is arranged in parallel (Patent Document 1). The neural network adjusts parameters by machine learning, but at this time, it is required to obtain a robust behavior criterion for environmental changes. For such a problem, a method of adding interference to a control object or the environment has been proposed (Patent Document 2).

[0003] Prior Art Documents

[0004] Patent Documents

[0005] Patent Document 1: Japanese Unexamined Patent Application Publication No. 2022-46317

[0006] Patent Document 2: Japanese Unexamined Patent Application Publication No. 2002-189502 Summary of the Invention

[0007] Problems to be Solved by the Invention

[0008] In the case of performing learning for determining parameter values of a neural network while giving virtual interference to a control object, it is desired to give virtual interference approximate to the interference generated during the execution of a production sequence. By giving such virtual interference, learning for determining parameter values of the neural network can be effectively performed, and control accuracy is improved in the control of a control object using the neural network.

[0009] Therefore, an object of the present invention is to provide a technique advantageous for improving control accuracy in the control of a control object using a neural network.

[0010] Technical Means for Solving the Problems

[0011] In order to achieve the above object, as an aspect of the present invention, a control method controls a controlled object by a control signal generated based on a first signal and a second signal. The first signal is generated by a first compensator based on a control deviation of the controlled object, and the second signal is generated by a second compensator based on the control deviation and through a neural network. The control method is characterized by including: a first control process for controlling the controlled object based on a target value specified by a first sequence; a second control process for controlling the controlled object based on a target value specified by a second sequence while applying a virtual disturbance set based on a first index obtained through the first control process to the controlled object; and a determination process for learning based on a second index obtained through the second control process to determine parameter values of the neural network.

[0012] Advantages of the Invention

[0013] According to the present invention, for example, a technique that is advantageous for improving control accuracy in the control of a controlled object using a neural network can be provided.

[0014] Other features and advantages of the present invention will become apparent from the following description with reference to the drawings. In addition, in the drawings, the same or similar structures are assigned the same reference numerals. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The drawings are included in the specification and form a part thereof, showing embodiments of the present invention and used to explain the principles of the present invention together with its description.

[0016] Figure 1 FIG. is a diagram showing a structural example of the system according to the first embodiment.

[0017] Figure 2 FIG. is a diagram showing a structural example of the system according to the first embodiment.

[0018] Figure 3 FIG. is a diagram showing a structural example of the sequence unit in the system according to the first embodiment.

[0019] Figure 4 FIG. is a diagram showing a structural example of the controller in the system according to the first embodiment.

[0020] Figure 5 FIG. is a diagram showing a structural example of the disturbance generator in the system according to the first embodiment.

[0021] Figure 6A FIG. is a flowchart showing an operation example of the system according to the first embodiment.

[0022] Figure 6B FIG. is a flowchart showing an operation example of the system according to the first embodiment.

[0023] Figure 7 This is a diagram showing a structural example of the stage control device according to the second embodiment.

[0024] Figure 8 This is a diagram showing a structural example of the stage control device according to the second embodiment.

[0025] Figure 9 This is a diagram showing a structural example of the exposure apparatus according to the third embodiment.

[0026] Figure 10 This is a diagram showing a structural example of the exposure apparatus according to the third embodiment.

[0027] (Reference Signs)

[0028] 100: Control device; 101: Sequence unit; 102: Controller; 103: Controlled object; 201: Learning unit; 202: Interference generator; 401: First compensator; 402: Second compensator. Detailed Embodiments

[0029] Hereinafter, embodiments will be described in detail with reference to the drawings. In addition, the following embodiments do not limit the invention described in the claims. A plurality of features are described in the embodiments, but not all of these plurality of features are necessarily required for the invention. Further, the plurality of features can be arbitrarily combined. Further, in the drawings, the same or similar structures are given the same reference numerals, and redundant descriptions are omitted.

[0030] <First Embodiment>

[0031] The first embodiment of the present invention will be described. In Figure 1 , a structural example of the system SS according to the first embodiment is shown. The system SS can be applied, for example, to a manufacturing apparatus for manufacturing an article. The manufacturing apparatus can include, for example, a processing apparatus that processes a material or a component of an article or a part of an article. The processing apparatus can be, for example, a lithography apparatus that transfers a pattern to a material or a component, a film forming apparatus that forms a film on a material or a component, an apparatus that etches a material or a component, and a heating apparatus that heats a material or a component.

[0032] System SS may include, for example, a sequence unit 101, a control device 100, and a controlled object 103. The control device 100 may include a controller 102. The control device 100 or the controller 102 may generate an operation amount (operation amount signal) MV as a control signal for controlling the controlled object 103. When the system SS is applied to a production system, a production sequence (first sequence) may be provided to the sequence unit 101. The production sequence may define the order for production (operation), specifically, the time series data of the target value (target control amount) R for controlling the controlled object 103 during production. The sequence unit 101 may generate the target value R for controlling the controlled object 103 based on the production sequence and provide the target value R to the control device 100 or the controller 102.

[0033] The control device 100 or the controller 102 may perform feedback control on the controlled object 103. Specifically, the control device 100 or the controller 102 may control the controlled object 103 in such a way that the control amount CV follows the target value R based on the difference, i.e., the control deviation, between the target value R provided from the sequence unit 101 and the control amount CV provided from the controlled object 103. The controlled object 103 may have a sensor for detecting the control amount CV, and the control amount CV detected by the sensor may be provided to the controller 102. The target value R, the operation amount MV, and the control amount CV may be time series data whose values change over time.

[0034] As Figure 2 shown, in the system SS, a learning unit 201 and a disturbance generator 202 may be assembled. The learning unit 201 and the disturbance generator 202 may be configured as part of the control device 100 or as an external device of the control device 100. When the learning unit 201 and the disturbance generator 202 are configured as an external device of the control device 100, the learning unit 201 and the disturbance generator 202 may be separated from the control device 100 after learning. The learning unit 201 may be configured to transmit a learning sequence (second sequence) to the sequence unit 101. In the learning sequence, the target value (target control amount) R for controlling the controlled object 103 during learning is defined as time series data. The learning sequence may be set (configured) such that the time for controlling the controlled object 103 through the learning sequence is shorter than the time for controlling the controlled object 103 through the production sequence. The sequence unit 101 may generate the target value R according to the learning sequence and provide it to the controller 102. The disturbance generator 202 may be configured to transmit a disturbance signal N to the controller 102. The disturbance signal N may also be generated based on the action history HP provided from the controller 102 when the production sequence is executed.

[0035] The controller 102 can generate an operation amount MV based on the difference, i.e., the control deviation E, between the target value R generated and provided by the sequence unit 101 according to the production sequence or the learning sequence and the control amount CV provided by the controlled object 103. Here, the controller 102 has a neural network, and this neural network can be used to generate the operation amount MV. The operation amount MV generated by the controller 102 is provided to the controlled object 103, and the controlled object 103 can act according to this operation amount MV. The control amount CV as the result of this action can be provided to the controller 102. The controller 102 can provide an action history HL indicating the history of the action of the controller 102 based on the target value R to the learning unit 201. The learning unit 201 can determine the parameter values of the neural network of the controller 102 based on this action history HL and set these parameter values into the neural network. These parameter values can be determined, for example, by machine learning such as reinforcement learning.

[0036] In Figure 3 a structure example of the sequence unit 101 is shown. The sequence unit 101 can include a first target value generator 301 that generates a first target value R1 based on the production sequence and a second target value generator 302 that generates a second target value R2 based on the learning sequence. The first target value R1 can be the target value (target control amount) of the controlled object 103 during the execution of the production sequence. The second target value R2 can be the target value (target control amount) of the controlled object 103 during the execution of the learning sequence, taking a value different from the first target value R1. The learning sequence can also be, for example, a sequence that specifies the target value changing with time for keeping the controlled object 103 in a stopped state. In this case, the second target value R2 can be a stop instruction indicating the target value for stopping the controlled object 103, but is not limited to these. In addition, the sequence unit 101 can include a target value switch 303 that generates the target value R based on the first target value R1 and the second target value R2. The target value R is obtained by selecting one of the first target value R1 and the second target value R2. The target value switch 303 can selectively switch between the first target value R1 and the second target value R2 according to which one of the production sequence or the learning sequence is being executed, and thus output the target value R.

[0037] In Figure 4shows a structural example of the controller 102. The controller 102 may include a subtractor 405, a first compensator 401, a second compensator 402, and an arithmetic unit 406. The subtractor 405 generates a control deviation E, which is the difference between the target value R and the control amount CV. The first compensator 401 is, for example, a PID compensator and generates a first signal S1 based on the control deviation E. The second compensator 402 generates a second signal S2 based on the control deviation E through a neural network. The arithmetic unit 406 generates a control signal S based on the first signal S1 and the second signal S2. The control signal S is the sum of the first signal S1 and the second signal S2, and the arithmetic unit 406 may be constituted by an adder. From another perspective, the control signal S is a signal obtained by correcting the first signal S1 based on the second signal S2.

[0038] The controller 102 may further include an arithmetic unit 407 that generates an operation amount MV based on the control signal S and the disturbance signal N. The disturbance signal N is a signal for applying a virtual disturbance (vibration) to the controlled object 103 during the execution of the learning sequence and is generated by the disturbance generator 202 based on the operation history HP obtained from the controller 102 during the execution of the production sequence. The disturbance signal N may also be a signal for applying a virtual disturbance obtained by imitating the disturbance generated during the execution of the production sequence to the controlled object 103. The disturbance signal N may be composed of a single frequency or a combination of multiple frequencies. For example, the disturbance signal N may be a sine wave, random noise, impulse interference, but is not limited to these. In addition, the operation history HP may be the history (time series data) of the first index obtained from the controller 102 during the execution of the production sequence. The first index may be, for example, the control deviation E and / or the control signal S, but is not limited to these. The operation amount MV is the sum of the control signal S and the disturbance signal N, and the arithmetic unit 407 may be constituted by an adder.

[0039] The controller 102 may further include a first recording unit 404. The learning unit 201 may be configured to perform learning for determining the parameter values of the neural network of the second compensator 402. To perform the learning implemented by the learning unit 201, the first recording unit 404 may record (store) the operation history HL required for the learning implemented by the learning unit 201 and provide the recorded operation history HL to the learning unit 201. The operation history HL may be the history (time series data) of the second index obtained from the controller 102 during the execution of the learning sequence. The second index may be, for example, the input data to the second compensator 402, i.e., the control deviation E, and / or the output data of the second compensator 402, i.e., the second signal S2, or may alternatively include or additionally include other data and signals.

[0040] In Figure 5FIG. 0 shows a structural example of the interference generator 202. The interference generator 202 may include a reference interference generation unit 502, a second recording unit 501, and a third recording unit 504. The reference interference generation unit 502 generates a reference interference signal NO that serves as a reference for the interference signal. The second recording unit 501 records (stores) the operation history HP obtained from the controller 102 during the execution of the production sequence. As described above, the operation history HP may be the history of the first index (e.g., control deviation E and / or control signal S) obtained from the controller 102 during the execution of the production sequence. The third recording unit 504 records (stores) the operation history HA obtained from the controller 102 during the adjustment of the virtual interference (during the learning process) for the adjustment and setting of the virtual interference. The operation history HA may be the history (time series data) of the first index (e.g., control deviation E and / or control signal S) obtained from the controller 102 during the adjustment of the virtual interference.

[0041] In addition, the interference generator 202 may include an interference adjustment unit 503 that adjusts the interference signal N based on the reference interference signal NO, the operation history HP during the execution of the production sequence, and the operation history HA during the adjustment of the virtual interference. The interference signal N may be obtained by correcting the amplitude of the reference interference signal NO based on the operation history HP during the execution of the production sequence and the operation history HA during the adjustment of the virtual interference. That is, the interference adjustment unit 503 multiplies the gain by the reference interference signal NO based on the operation history HP during the execution of the production sequence and the operation history HA during the adjustment of the virtual interference to correct the amplitude of the reference interference signal NO, and provides the resulting signal as the interference signal N to the arithmetic unit 407.

[0042] In Figures 6A - 6B FIG. 8 shows an operation example of the system SS when the system SS of the first embodiment is applied to a production apparatus. Figure 6A FIG. 10 shows a first control process (S601 to S602) for controlling the controlled object 103 according to the production sequence and a setting process (S603 to S612) for adjusting and setting the virtual interference. In addition, Figure 6B FIG. 12 shows a second control process (S613 to S617) for controlling the controlled object 103 according to the learning sequence while applying the virtual interference and a determination process (S618) for determining the parameter values of the neural network of the second compensator 402 through learning.

[0043] In Figure 6AIn step S601, the control device 100 executes a production sequence. Specifically, the sequence unit 101 can generate the first target value R1 specified by the given production sequence as the target value R, and provide the target value R to the control device 100 (controller 102). Then, the control device 100 (controller 102) can control the controlled object 103 based on the target value R provided from the sequence unit 101. Next, in step S602, the control device 100 (the second recording unit 501) can record the history of the first index (for example, the control deviation E) obtained in the production sequence of step S601 as the operation history HP. In addition, step S602 can also be performed in parallel with step S601.

[0044] In step S603, the control device 100 (the disturbance adjustment unit 503 included in the disturbance generator 202) can adjust (correct) the amplitude of the reference disturbance signal NO by multiplying an arbitrary gain to the reference disturbance signal NO, and generate the disturbance signal N obtained as a result. Next, in step S604, the control device 100 (controller 102) can start applying vibration to the controlled object 103 based on the disturbance signal N (applying virtual disturbance). Specifically, start providing the disturbance signal N from the disturbance generator 202 to the arithmetic unit 407. Thereby, start applying the virtual disturbance corresponding to the disturbance signal N to the controlled object 103.

[0045] In step S605, the control device (controller 102) executes a learning sequence. Specifically, the sequence unit 101 can generate the second target value R2 specified by the learning sequence given from the learning unit 201 as the target value R, and provide the target value R to the control device 100 (controller 102). Then, the control device 100 (controller 102) can control the controlled object 103 based on the target value R provided from the sequence unit 101. In step S605, while applying virtual disturbance to the controlled object 103, control the controlled object 103 based on the learning sequence. Here, the virtual disturbance can be continuously applied to the controlled object 103 or intermittently applied. In addition, as described above, the learning sequence can be set such that the time for controlling the controlled object 103 through the learning sequence is shorter than the time for controlling the controlled object 103 through the production sequence.

[0046] In process S606, the control device 100 (controller 102) can stop applying vibration to the controlled object 103 based on the interference signal N. Specifically, it stops supplying the interference signal N from the interference generator 202 to the arithmetic unit 407. Thereby, applying the virtual interference corresponding to the interference signal N to the controlled object 103 is stopped. Next, in process S607, the control device 100 (the third recording unit 504) can record the history of the first index (for example, the control deviation E) obtained in process S605 as the operation history HA. In addition, process S607 can also be performed in parallel with process S605.

[0047] In process S608, the control device 100 can determine whether the difference between the first index of the operation history HP recorded in process S602 and the first index of the operation history HA recorded in process S607 is lower than the threshold value. For example, when using the control deviation E as the first index, the control device 100 can determine whether the difference between the amplitude of the control deviation E in the operation history HP recorded in process S602 and the amplitude of the control deviation E in the operation history HA recorded in process S607 is lower than the threshold value. As the amplitude of the control deviation E, the maximum value, average value, or median value of the amplitude can be used. When this difference is lower than the threshold value, it is determined to use the currently used interference signal N (virtual interference) in the second control process (S613 to S617), and the process proceeds to process S613. That is, the control device 100 sets the currently used interference signal N as the interference signal N to be used in the second control process. On the other hand, when this difference is equal to or greater than the threshold value, the process proceeds to process S609.

[0048] In process S609, the control device 100 determines whether the first index (for example, the amplitude of the control deviation E) of the operation history HA is greater than that of the operation history HP. When the first index of the operation history HA is greater than that of the operation history HP, the process proceeds to process S610, and when it is not greater, the process proceeds to process S611. In process S610, the control device 100 (interference adjustment unit 503) can decrease the value of the gain multiplied by the reference interference signal NO by one level. On the other hand, in process S611, the control device 100 (interference adjustment unit 503) can increase the value of the gain multiplied by the reference interference signal NO by one level. The amount of decreasing the gain by one level in process S610 and the amount of increasing the gain by one level in process S611 can be arbitrary, but it is preferably set in a manner that can reduce the difference in the first index (for example, the amplitude of the control deviation E).

[0049] In step S612, the control device 100 (interference adjustment unit 503) can adjust the amplitude of the reference interference signal NO by multiplying the gain adjusted in step S610 or step S611 by the reference interference signal NO, and generate an interference signal N as a result. When step S612 ends, it can proceed to step S604 and execute steps S604 to S608 again.

[0050] Proceed to Figure 6B , in step S613, the control device 100 (interference adjustment unit 503) can start applying vibration to the controlled object 103 based on the interference signal N set through steps S603 to S612. Specifically, it starts providing the interference signal N from the interference generator 202 to the arithmetic unit 407. Thereby, starting to apply virtual interference corresponding to the interference signal N to the controlled object 103.

[0051] In step S614, the control device 100 (controller 102) executes a learning sequence. Specifically, similar to the aforementioned step S605, the sequence unit 101 can generate the second target value R2 specified by the learning sequence given from the learning unit 201 as the target value R, and provide the target value R to the control device 100 (controller 102). Then, the control device 100 (controller 102) can control the controlled object 103 based on the target value R provided from the sequence unit 101. In step S614, while applying virtual interference to the controlled object 103, the control of the controlled object 103 based on the learning sequence is performed. Here, the virtual interference can be continuously applied to the controlled object 103 or intermittently applied.

[0052] In step S615, the control device 100 (controller 102) can stop applying vibration to the controlled object 103 based on the interference signal N. Specifically, it stops providing the interference signal N from the interference generator 202 to the arithmetic unit 407. Thereby, stopping the application of virtual interference corresponding to the interference signal N to the controlled object 103. Next, in step S616, the control device 100 (first recording unit 404) can record the history of the second index (for example, control deviation E) obtained in step S614 as the operation history HL. In addition, step S616 can also be performed in parallel with step S614.

[0053] In process S617, the control device 100 can determine whether the second index of the operation history HL recorded in process S616 is lower than a specified value. For example, when using the control deviation E as the second index, the control device 100 can determine whether the control deviation E recorded in process S616 is lower than the specified value. As the second index, either the maximum value, average value, or median value of the amplitude of the control deviation E can be used, or the second signal S2 output from the second compensator 402 can be used. When the second index recorded in process S616 is lower than the specified value, it ends. In this case, the control device 100 determines the parameter value of the neural network of the currently used second compensator 402 as the parameter value to be used in the future. On the other hand, when the second index recorded in process S616 is equal to or greater than the specified value, it proceeds to process S618.

[0054] In process S618, the control device 100 (learning unit 201) determines (updates) the parameter value of the neural network of the second compensator 402 by performing machine learning based on the operation history HL recorded in process S616. When process S618 ends, it can proceed to process S613, set the parameter value determined in process S618 into the neural network of the second compensator 402, and execute processes S613 to S617 again.

[0055] As described above, in the first embodiment, while applying virtual interference to the control object 103 through the interference signal N, the control object 103 is controlled according to the learning sequence. Then, learning is performed based on the second index (such as the control deviation E) obtained through this control, so as to determine the parameter value of the neural network of the second compensator 402 in a manner that reduces the control deviation of the control object 103. According to the first embodiment, by applying virtual interference approximate to the interference generated during the execution of the production sequence, learning for determining the parameter value of the neural network is effectively performed. Moreover, the control accuracy is improved in the control of the control object 103 using the neural network. In addition, according to the first embodiment, in the learning sequence, virtual interference is applied to the control object 103. Therefore, even if the time for controlling the control object 103 is shorter than the production sequence, the time (effective time) for applying interference to the control object 103 can be longer than the production sequence. Therefore, the time required for learning to determine the parameter value of the neural network of the second compensator 402 can be shortened, and the productivity decline of the manufacturing device of the application system SS can be suppressed.

[0056] For example, in the past, learning to determine the parameter values of a neural network was performed by controlling the control object 103 in the same sequence as the production sequence. However, in the conventional method, during the learning using the same sequence as the production sequence, the control accuracy of the control object 103 was uncertain. Therefore, it was necessary to stop the production implemented by the device. That is, it might lead to a decrease in the productivity of the device. On the other hand, according to the first embodiment, in order to simulate the interference environment generated during the execution of the production sequence, while applying virtual interference to the control object 103, learning was performed using a learning sequence with a short control time for the control object. Thus, the learning for determining the parameter values of the neural network was effectively performed, and in the control of the control object 103 using the neural network, the control accuracy was improved. In addition, the time required for learning could be shortened. Therefore, as a result, it was possible to suppress the decrease in the productivity of the manufacturing device of the application system SS.

[0057] Here, the control device 100 of the first embodiment may be constituted by a computer (information processing device) having a processor such as a CPU (Central Processing Unit) and a storage unit such as a memory. The control device 100 may also be constituted by, for example, a PLD (abbreviation of Programmable Logic Device) such as an FPGA (abbreviation of Field Programmable Gate Array), or an ASIC (abbreviation of Application Specific Integrated Circuit), or a general-purpose computer with a program installed, or a combination of all or part of them. In addition, the sequence unit 101 may be assembled into the computer constituting the control device 100, or may be constituted by a computer different from the control device 100.

[0058] <Second Embodiment>

[0059] The second embodiment of the present invention will be described. The second embodiment basically follows the first embodiment and may be implemented in accordance with the first embodiment except for the matters mentioned below.

[0060] In Figure 7In this case, an example is shown in which the system SS or the control device 100 of the first embodiment is applied to the stage control device 700. The stage control device 700 is configured to control a stage 704. The stage control device 700 may include, for example, a control board 701, a current driver 702, a motor 703, a stage 704, and a sensor 705. The control board 701 corresponds to the control device 100 or the controller 102 in the system SS of the first embodiment. The current driver 702 may also be assembled into the control board 701. Although not shown in Figure 7 this case, the stage control device 700 may include the sequence unit 101, the learning unit 201, and the interference generator 202 that have been described in the first embodiment.

[0061] A position target value PR, which is a target value of the position of the stage 704, can be supplied from the sequence unit 101 to the control board 701. The control board 701 can generate a current command as a control signal or an operation amount (operation amount command) based on the position target value PR supplied from the sequence unit 101 and the position information (current position) of the stage 704 supplied from the sensor 705, and supply it to the current driver 702. In addition, the control board 701 can supply the action history HL to the sequence unit 101.

[0062] The current driver 702 can supply a current in accordance with the current command to the motor 703. The motor 703 can be an actuator that converts the current supplied from the current driver 702 into thrust and drives the stage 704 with this thrust. The stage 704 can hold an object such as a plate (substrate) or a mask (reticle), for example. The sensor 705 can detect the current position of the stage 704 and supply the resulting position information to the control board 701.

[0063] In Figure 8 this case, a structural example of the control board 701 is shown as a block diagram. The control board 701 may include a first compensator 401 that generates a first signal S1 based on a position control deviation E of the stage 704 as a control object, and a second compensator 402 that generates a second signal S2 based on the position control deviation E through a neural network. In addition, the control board 701 may include an arithmetic unit 406 that generates a control signal S based on the first signal S1 and the second signal S2, and an arithmetic unit 407 that generates a current command as an operation amount signal based on the control signal S and an interference signal N. The control board 701 may further include a subtractor 405 that generates a position control deviation E, which is the difference between the position target value PR and the position information.

[0064] The mounting stage control device 700 according to the second embodiment can include the learning unit 201 and the disturbance generator 202 described in the first embodiment. The learning unit 201 can be configured to perform learning for determining the parameter values of the neural network for the second compensator 402. To perform the learning implemented by the learning unit 201, the first recording unit 404 can record the action history HL required for the learning implemented by the learning unit 201, and provide the recorded action history HL to the learning unit 201. The action history HL can be a history (time series data) of the second index obtained during the execution of the learning sequence. The second index can be, for example, the position control deviation E which is the input data to the second compensator 402 and / or the second signal S2 which is the output data of the second compensator 402, but alternatively, other data and signals can be included or additionally included. In addition, the disturbance generator 202 can generate a disturbance signal N based on the action history HP during the execution of the production sequence, and provide it to the control board 701. The action history HP can be a history (time series data) of the first index obtained during the execution of the production sequence. The first index can be, for example, the position control deviation E, the control signal S, but is not limited to these. Here, the learning unit 201 and the disturbance generator 202 can also be configured as structural elements of the control board 701.

[0065] Next, Figures 6A - 6B , the operation of the mounting stage control device 700 when the mounting stage control device 700 according to the second embodiment is applied to a production device will be exemplarily described. In step S601, the mounting stage control device 700 executes a production sequence. Specifically, the sequence unit 101 can generate a position target value PR based on the given production sequence, and provide the position target value PR to the mounting stage control device 700 (control board 701). Then, the mounting stage control device 700 (control board 701) can control the position of the mounting stage 704 based on the position target value PR provided from the sequence unit 101. In step S602, the mounting stage control device 700 (second recording unit 501) can record the history of the first index (for example, control deviation E) obtained in the production sequence in step S601 as the action history HP. Next, in steps S603 to S612, the mounting stage control device 700 (disturbance adjustment unit 503) can generate a disturbance signal N for applying a virtual disturbance to the mounting stage 704 by adjusting the amplitude of the reference disturbance signal NO based on the action history HP recorded in the second recording unit 501.

[0066] In process S613, the stage control device 700 (control board 701) may start applying vibration to the stage 704 based on the interference signal N. In process S614, the stage control device 700 (control board 701) executes a learning sequence. Specifically, the sequence unit 101 may generate a position target value PR based on the learning sequence and provide it to the stage control device 700 (control board 701). Then, the stage control device 700 (control board 701) may control the stage 704 based on the target value R provided from the sequence unit 101. In process S614, while giving virtual interference to the stage 704, the control of the stage 704 based on the learning sequence is performed. Here, the virtual interference may be continuously given to the stage 704 or intermittently given.

[0067] In process S615, the stage control device 700 (control board 701) may stop applying vibration to the stage 704 based on the interference signal N. In process S616, the stage control device 700 (first recording unit 404) may record the history of the second index (for example, the position control deviation E) obtained in process S614 as the operation history HL. In process S617, the stage control device 700 may determine whether the second index of the operation history HL recorded in process S616 is lower than a specified value. For example, when using the position control deviation E as the second index, the stage control device 700 may determine whether the position control deviation E recorded in process S616 is lower than the specified value. It ends when the second index recorded in process S616 is lower than the specified value. In this case, the stage control device 700 determines the parameter value of the neural network of the currently used second compensator 402 as the parameter value to be used in the future. On the other hand, when the position control deviation E recorded in process S616 is equal to or greater than the specified value, it proceeds to process S618. In process S618, the stage control device 700 (learning unit 201) determines (updates) the parameter value of the neural network of the second compensator 402 by performing machine learning based on the operation history HL recorded in process S616.

[0068] As described above, in the second embodiment, while applying virtual interference to the stage 704 through the interference signal N, the stage 704 is controlled according to the learning sequence. Then, learning is performed based on the second index (for example, the position control deviation E) obtained through this control, and the parameter values of the neural network of the second compensator 402 are determined in such a way as to reduce the control deviation of the stage 704. According to the second embodiment, as in the first embodiment, learning for determining the parameter values of the neural network is effectively performed, and the control accuracy is improved in the control of the stage 704 using the neural network. In addition, according to the second embodiment, as in the first embodiment, the time required for learning the parameter values of the neural network of the second compensator 402 can be shortened, and a decrease in the productivity of the manufacturing apparatus to which the stage control device 700 is applied can be suppressed.

[0069] <Third Embodiment>

[0070] The third embodiment of the present invention will be described. The third embodiment basically follows the first and second embodiments, and except for the matters mentioned below, the first and second embodiments can be followed.

[0071] In Figure 9 FIG. schematically shows a structural example of the exposure apparatus EXP of the third embodiment. The exposure apparatus EXP can be configured as a scanning exposure apparatus. The exposure apparatus EXP can include, for example, an illumination light source 900, an illumination optical system 901, a mask stage 903, a projection optical system 904, and a wafer stage 906. The illumination light source 900 can include a mercury lamp, an excimer laser light source, or an EUV light source, but is not limited to these. The exposure light 910 from the illumination light source 900 is shaped by the illumination optical system 901 into the shape of the irradiation area of the projection optical system 904 with uniform illuminance. In one example, the exposure light 910 can be shaped into a rectangle that is long in the X direction, which is an axis perpendicular to the plane formed by the Y axis and the Z axis. Depending on the type of the projection optical system 904, the exposure light 910 can be shaped into an arc shape. The shaped exposure light 910 is irradiated onto the pattern of the mask (reticle) 902, and the exposure light 910 that has passed through the pattern of the mask 902 forms an image of the pattern of the mask 902 on the surface of the wafer (substrate) 905 via the projection optical system 904.

[0072] The mask 902 is held by the mask stage 903 by vacuum suction or the like, and a pattern to be transferred to the wafer 905 is formed on the mask 902. In addition, the wafer 905 is held by the chuck 907 of the wafer stage 906 by vacuum suction or the like. The positions of the mask stage 903 and the wafer stage 906 can be controlled by a multi-axis position control device including position sensors 930a to 930b such as laser interferometers or laser scales, drive mechanisms 931a to 931b such as linear motors, and a controller 932.

[0073] The position of the on-board stage 906 (board 905) is controlled by a position sensor 930a, a drive mechanism 931a, and a controller 932. The position measurement value of the on-board stage 906 output from the position sensor 930a can be provided to the controller 932. The controller 932 generates a control signal (operation amount signal) based on the difference between the position target value and the position measurement value, i.e., the position control deviation, and provides it to the drive mechanism 931a, thereby driving the on-board stage 906. Similarly, the position of the mask stage 903 (mask 902) is controlled by a position sensor 930b, a drive mechanism 931b, and a controller 932. The position measurement value of the mask stage 903 output from the position sensor 930b can be provided to the controller 932. The controller 932 generates a control signal (operation amount signal) based on the difference between the position target value and the position measurement value, i.e., the position control deviation, and provides it to the drive mechanism 931b, thereby driving the mask stage 903. By synchronously driving the mask stage 903 and the on-board stage 906 in the Y direction while scanning and exposing the board 905, the pattern of the mask 902 is transferred to the board 905 (the photosensitive material thereon).

[0074] Here, the control device 100 of the first embodiment or the stage control device 700 of the second embodiment is used to control the position of the mask stage 903 (mask 902) and / or the position of the on-board stage 906 (board 905). That is, the controller 932 that controls the mask stage 903 (mask 902) and / or the on-board stage 906 (board 905) as the control object can be constituted by the control device 100 of the first embodiment or the stage control device 700 of the second embodiment. In addition, the exposure device EXP or the controller 932 can also be configured to include the sequence unit 101, the learning unit 201, and / or the interference generator 202 described in the first embodiment.

[0075] In Figure 10 FIG., a structural example of the controller 932 is shown. The controller 932 can switch between single-axis control and synchronous control for multiple control objects and multiple controllers respectively. Here, a master-slave type synchronous control is shown in which the on-board stage 906 is set as the master axis, the mask stage 903 is set as the slave axis, and the slave axis follows the master axis.

[0076] The controller 932 may include a first compensator 1001 for board position control that generates a first signal S1 based on the position control deviation E1 of the on-board stage 906 as the first control object, and an arithmetic unit 407 that generates an operation amount MV1 based on the first signal S1a and the disturbance signal N. Additionally, the controller 932 may include a subtractor 405a that generates the position control deviation E1, which is the difference between the position target value PR1 and the control amount (position measurement value) CV1 of the on-board stage 906. The first compensator 1001 for board position control may be, for example, a PID compensator.

[0077] The controller 932 may include a synchronization switch 1004 that switches the synchronization control between the on-board stage 906 and the mask stage 903. The synchronization switch 1004 may generate a position control deviation E based on the position control deviation E2 of the mask stage 903 as the second control object and the synchronization position deviation Es, which is the difference between the position control deviations E1 and E2. The position control deviation E is obtained by selecting either the position control deviation E2 or the synchronization position deviation Es. The synchronization switch 1004 may output the position control deviation E by selectively switching between the position control deviation E2 and the synchronization position deviation Es.

[0078] The controller 932 may include a first compensator 1002 for mask position control that generates a first signal S1b based on the position control deviation E, and a second compensator 1003 for mask position control that generates a second signal S2 based on the position control deviation E through a neural network. The controller 932 may include an arithmetic unit 406 that generates an operation amount MV2 based on the first signal S1b and the second signal S2. Additionally, the controller 932 may include a subtractor 405b that generates the position control deviation E2, which is the difference between the position target value PR2 and the control amount (position measurement value) CV2 of the mask stage 903.

[0079] The exposure apparatus EXP according to the third embodiment can include a learning unit 201 and a disturbance generator 202. The learning unit 201 can be configured to perform learning of parameter values of a neural network for determining the second compensator 1003 for mask position control. In order to perform the learning implemented by the learning unit 201, the first recording unit 404 can record the action history HL required for the learning implemented by the learning unit 201, and provide the recorded action history HL to the learning unit 201. The action history HL can be a history (time series data) of the second index obtained during the execution of the learning sequence. The second index can be, for example, the position control deviation E, which is the input data for the second compensator 1003 for mask position control, and / or the output data of the second compensator 1003, which is the second signal S2. However, other data and signals can be included or additionally included as an alternative. In addition, the disturbance generator 202 can generate a disturbance signal N based on the action history HP during the execution of the production sequence, and provide it to the controller 932 (arithmetic unit 407). The action history HP can be a history (time series data) of the first index obtained during the execution of the production sequence. The first index can be, for example, the position control deviation E, but is not limited thereto.

[0080] Next, Figures 6A - 6B , the operation of the exposure apparatus EXP according to the third embodiment will be exemplarily described. In step S601, the exposure apparatus EXP (controller 932) executes the production sequence. Specifically, the sequence unit 101 can generate a position target value PR1 of the substrate stage 906 and a position target value PR2 of the mask stage 903 based on the given production sequence, and provide them to the exposure apparatus EXP (controller 932). The controller 932 (subtractor 405a) can obtain the control amount CV1 of the substrate stage 906 detected (observed) by the position sensor 930a, and calculate the difference between the control amount CV1 and the position target value PR1 as the position control deviation E1. Similarly, the controller 932 (subtractor 405b) can obtain the control amount CV2 of the mask stage 903 detected (observed) by the position sensor 930b, and calculate the difference between the control amount CV2 and the position target value PR2 as the position control deviation E2.

[0081] The controller 932 can control the position of the substrate stage 906 based on the position control deviation E1. In addition, the controller 932 can control the position of the mask stage 903 based on the position control deviation E2 switched by the synchronous switch 1004 or the difference between the position control deviations E1 and E2, that is, the synchronous position deviation Es.

[0082] In step S602, the controller 932 (the second recording unit 501) may record the history of the first index (e.g., the position control deviation E) obtained in the production sequence of step S601 as the operation history HP. Further, in steps S603 to S612, the exposure apparatus EXP (the disturbance adjustment unit 503) may generate a disturbance signal N for applying a virtual disturbance to the stage 906 by adjusting the amplitude of the reference disturbance signal NO based on the operation history HP recorded in the second recording unit 501.

[0083] In step S613, the controller 932 may start applying vibration to the stage 906 based on the disturbance signal N. In step S614, the controller 932 executes a learning sequence. Specifically, the sequence unit 101 may generate position target values PR1 and PR2 based on the learning sequence and supply them to the exposure apparatus EXP (the controller 932). The controller 932 may control the stage 906 and the mask stage 903 based on the position target values PR1 and PR2, respectively. In step S614, while applying a virtual disturbance to the stage 906, the stage 906 and the mask stage 903 are controlled based on the learning sequence. Here, the virtual disturbance may be continuously applied to the stage 906 or intermittently applied.

[0084] In step S615, the controller 932 may stop applying vibration to the stage 906 based on the disturbance signal N. In step S616, the controller 932 (the first recording unit 404) may record the history of the second index (e.g., the position control deviation E) obtained in step S614 as the operation history HL. In step S617, the controller 932 may determine whether the second index of the operation history HL recorded in step S616 is lower than a specified value. For example, when using the position control deviation E as the second index, the controller 932 may determine whether the position control deviation E recorded in step S616 is lower than the specified value. If the position control deviation E recorded in step S616 is lower than the specified value, the process ends. In this case, the controller 932 determines the parameter value of the neural network of the second compensator 1003 for mask position control currently in use as the parameter value to be used in the future. On the other hand, if the position control deviation E recorded in step S616 is equal to or greater than the specified value, the process proceeds to step S618. In step S618, the controller 932 (the learning unit 201) determines (updates) the parameter value of the neural network of the second compensator 1003 for mask position control by performing machine learning based on the operation history HL recorded in step S616.

[0085] As described above, in the third embodiment, while applying virtual interference to the on-board stage 906 through the interference signal N, the on-board stage 906 and the mask stage 903 are controlled in accordance with the learning sequence. Then, learning is performed based on the second index (e.g., the position control deviation E) obtained through this control, and the parameter values of the neural network of the second compensator 1003 are determined in such a way as to reduce the control deviation between the on-board stage 906 and the mask stage 903. According to the third embodiment, as in the first and second embodiments, learning for determining the parameter values of the neural network is effectively performed, and the control accuracy is improved in the control of the on-board stage 906 using the neural network. In addition, according to the third embodiment, as in the first and second embodiments, the time required for learning the parameter values of the neural network of the second compensator 1003 can be shortened. Further, by disposing the interference generator 202 on the synchronization host side, it is easy to simulate the interference environment during synchronization. As a result, a decrease in the productivity of the exposure apparatus EXP can be suppressed.

[0086] Here, the third embodiment is not only applied to the control of the stage in the exposure apparatus, but can also be applied to the control of the stage in other lithography apparatuses such as an imprint apparatus and an electron beam lithography apparatus. In addition, the first embodiment or the second embodiment can be applied, for example, to the control of a movable part in a transfer mechanism for transferring an article, such as a robot hand for holding an article.

[0087] <Embodiments of the article manufacturing method>

[0088] The above-described lithography apparatus can be used to implement an article manufacturing method for manufacturing various articles (semiconductor IC elements, liquid crystal display elements, MEMS, etc.). The article manufacturing method may include a forming step of forming a pattern on a substrate using the above-described lithography apparatus (a transfer step of transferring the pattern of the reticle to the substrate), a processing step of processing the substrate that has undergone the forming step, and a manufacturing step of manufacturing an article from the substrate that has undergone the processing step. When the lithography apparatus is an exposure apparatus, the forming step may include an exposure step of exposing the substrate through the reticle and a developing step of developing the substrate that has undergone the exposure step. Further, the above-described article manufacturing method includes other known steps (baking, cooling, cleaning, oxidation, film formation, evaporation, doping, planarization, etching, resist stripping, cutting, bonding, packaging, etc.). The article manufacturing method of the present embodiment is advantageous in at least one of the performance, quality, productivity, and production cost of the article as compared with the conventional method.

[0089] The invention is not limited to the above embodiments, and various changes and modifications can be made without departing from the spirit and scope of the invention. Therefore, in order to disclose the scope of the invention, claims are attached.

[0090] This application claims priority based on Japanese Patent Application No. 2022-170869 filed on October 25, 2022, and incorporates all of its descriptions herein by reference.

Claims

1. A control method, controlling a controlled object by a control signal generated based on a first signal and a second signal, where the first signal is generated by a first compensator based on a control deviation of the controlled object, and the second signal is generated by a second compensator based on the control deviation and through a neural network, the control method is characterized by comprising: a first control process of controlling the controlled object based on a target value specified by a first sequence; a second control process of controlling the controlled object based on a target value specified by a second sequence while applying a virtual disturbance set based on a first index obtained through the first control process to the controlled object; and a determination process of learning based on a second index obtained through the second control process to determine a parameter value of the neural network.

2. The control method according to claim 1, characterized in that The second sequence defines a time-varying target value for bringing the controlled object to a stop state.

3. The control method according to claim 1 or 2, characterized in that The virtual disturbance is set such that the difference between the first index obtained when the virtual disturbance is applied to the controlled object and the first index obtained through the first control process is lower than a threshold value.

4. The control method according to any one of claims 1 to 3, characterized in that The first index includes the control deviation of the controlled object.

5. The control method according to any one of claims 1 to 4, characterized in that The first index includes the control signal generated based on the first signal and the second signal.

6. The control method according to any one of claims 1 to 5, characterized in that The second control process and the determination process are repeatedly executed such that the second index obtained when the second control process is performed using the parameter value determined through the determination process is lower than a specified value.

7. The control method according to any one of claims 1 to 6, characterized in that The second index includes the control deviation of the controlled object.

8. The control method according to any one of claims 1 to 7, characterized in that The second index includes the second signal.

9. The control method according to any one of claims 1 to 8, characterized in that In the determination process, the parameter value is determined by machine learning.

10. The control method according to any one of claims 1 to 9, characterized in that The time for controlling the controlled object by the second sequence is shorter than the time for controlling the controlled object by the first sequence.

11. A control device, a first compensator that generates a first signal based on a control deviation of a controlled object, and a second compensator that generates a second signal based on the control deviation and through a neural network, and controls the controlled object by a control signal generated according to the first signal and the second signal The control device is characterized by performing the following steps: A first control step of controlling the controlled object based on a target control amount specified by a first sequence; A second control step of controlling the controlled object based on a target value specified by a second sequence while applying a virtual disturbance set based on a first index obtained through the first control step to the controlled object; and A determination step of performing learning based on a second index obtained through the second control step to determine a parameter value of the neural network.

12. A lithographic apparatus for forming a pattern on a substrate, the lithographic apparatus being characterized in that it includes the control device according to claim 11 configured to control the position of the substrate or the position of a reticle having a pattern transferred to the substrate.

13. An article manufacturing method, characterized in that Comprising: A forming process of forming a pattern on a substrate using the lithographic apparatus according to claim 12; A processing process of processing the substrate that has undergone the forming process; And A manufacturing process of manufacturing an article from the substrate that has undergone the processing process.

Images