Control device, lithography apparatus, and production method of article

Abstract

To provide a technique advantageous for shortening a learning time and securing learning stability.SOLUTION: A control device includes: a control part for controlling a control object according to a learning model having a parameter value determined by learning; and a learning part for re-determining a parameter value of the learning model by learning using an operation history of the control object controlled by the control part, wherein the learning part executes second learning after first learning by using only a second operation history acquired after the first operation history without using a first operation history used in the first learning.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present invention relates to a control apparatus, a lithographic apparatus, and an article manufacturing method. [Background technology]

[0002] In recent years, the demand for improved control accuracy has become stricter, and conventional feedback control alone may not be able to achieve the required accuracy. Therefore, efforts are being made to configure neural network controllers (Patent Document 1). Neural network controllers adjust their parameters through machine learning. For example, in fields where complex models are required to achieve high levels of control accuracy, reinforcement learning, a type of machine learning, is increasingly being introduced.

[0003] Reinforcement learning is one type of machine learning, along with supervised learning and unsupervised learning. In reinforcement learning, learning is achieved by repeatedly acquiring learning data, calculating parameters, and updating the parameters. However, when learning is performed, production by the equipment must be stopped. In other words, performing learning leads to a decrease in productivity. To address this issue, a method has been proposed that shortens learning time by performing parameter updates and learning data acquisition asynchronously in parallel (Patent Document 2). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2022-046317 [Patent Document 2] Japanese Patent Publication No. 2022-121112 Summary of the Invention [Problem to be solved by the invention]

[0005] Reinforcement learning is performed on the neural network controller by repeatedly acquiring learning data by executing a predetermined learning sequence and calculating parameters using the acquired learning data. Therefore, in the learning process, it takes time to acquire learning data and calculate parameters.

[0006] In Patent Document 2, learning data acquisition and parameter calculation are performed asynchronously, thereby shortening the learning time. Furthermore, if the learning data acquisition time is short compared to the parameter calculation time, the learning data will be a consistent behavioral history based on fixed parameter values, resulting in stable learning. However, if the learning data acquisition time is long compared to the parameter calculation time, parameter values ​​will be updated during learning data acquisition, resulting in behavioral histories based on multiple parameter values ​​being mixed in the learning data. This may result in the learning data containing inconsistent behavioral histories, potentially reducing the stability of learning.

[0007] The present invention provides an advantageous technique for shortening learning time and ensuring learning stability. [Means for solving the problem]

[0008] According to one aspect of the present invention, there is provided a control device comprising: a control unit that controls a control object according to a learning model whose parameter values ​​have been determined by learning; and a learning unit that redetermines the parameter values ​​of the learning model by learning using the operation history of the control object controlled by the control unit, wherein the learning unit performs second learning after the first learning using only operation history acquired after the past operation history, without using the past operation history used in the first learning. [Effects of the Invention]

[0009] According to the present invention, it is possible to provide an advantageous technique for shortening the learning time and ensuring the learning stability. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 shows the system configuration. [Figure 2] This is a hardware configuration diagram when the system in Figure 1 is applied to a stage device. [Figure 3] FIG. 1 is a diagram showing the configuration of a system having a learning unit for determining parameter values. [Figure 4] FIG. 4 is a control block diagram when the system of FIG. 3 is applied to a stage device. [Figure 5] FIG. 3 is a block diagram showing a detailed configuration of a learning unit. [Figure 6] 10 is a flowchart showing the procedure of a learning process. [Figure 7] 10 is a flowchart showing details of a parameter calculation flow. [Figure 8] 10 is a flowchart showing details of a learning data acquisition flow. [Figure 9] FIG. 1 is a diagram showing an example of the configuration of an exposure apparatus. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the invention claimed. Although multiple features are described in the embodiments, not all of these multiple features are necessarily essential to the invention, and multiple features may be combined arbitrarily. Furthermore, in the accompanying drawings, the same reference numerals are used to designate the same or similar components, and redundant explanations will be omitted.

[0012] First Embodiment FIG. 1 is a diagram showing the configuration of a system SS according to the first embodiment. The system SS can be applied to, for example, a manufacturing apparatus for manufacturing an article. The manufacturing apparatus can include, for example, a processing apparatus for processing a material or member of the article or a component that constitutes a part of the article. The processing apparatus can be, for example, a lithography apparatus that transfers a pattern to the material or member, a film formation apparatus that forms a film on the material or member, an apparatus that etches the material or member, or a heating apparatus that heats the material or member.

[0013] The system SS may include, for example, a sequence unit 101, a control unit 102 (control device), and a controlled object 103. When the system is applied to a production system, a production sequence may be provided to the sequence unit 101. The production sequence may define a procedure for production. The sequence unit 101 may generate a target value for controlling the controlled object 103 based on the production sequence and provide the target value to the control unit 102. The control unit 102 outputs an operation amount to the controlled object 103 based on a control amount detected from the controlled object 103 and a target value set by the sequence unit 101. By repeating this process, the control unit 102 controls the controlled object 103.

[0014] 2 is a diagram showing an example of the hardware configuration when the system SS is applied to a stage device. The stage device can include a controller 501, a current driver 502, a motor 503, a stage 504, and a sensor 505. The controller 501 outputs a current command to the current driver 502. The current driver 502 outputs a current to the motor 503 configured inside the stage 504. The result of driving the stage 504 is observed by the sensor 505. The sensor 505 provides position information to the controller 501.

[0015] FIG. 3 is a diagram showing the configuration of a system SS having a learning unit. The control unit 102 controls the control object 103 according to a learning model whose parameter values ​​have been determined by learning. The learning unit 201 is configured to redetermine the parameter values ​​of the learning model by learning using the operation history of the control object 103 controlled by the control unit 102. The learning unit 201 sends a predetermined learning sequence to the sequence unit 101. The sequence unit 101 sets a target value according to the learning sequence. The control unit 102 sends the operation history of the control object 103 to the learning unit 201. The learning unit 201 calculates the parameter values ​​of the learning model using the operation history and sends the obtained parameter values ​​to the control unit 102. Reinforcement learning is used as the parameter adjustment method.

[0016] FIG. 4 is a control block diagram when the system of FIG. 3 is applied to a stage apparatus. A position error, which is the difference between a position target value and position information of a stage 504 observed by a sensor 505, is input to a controller 501 having a neural network as a learning model. The controller 501 outputs a current command to a current driver 502. The current driver 502 supplies a current corresponding to the current command to a motor 503. The driving force of the motor 503 due to the supplied current becomes a thrust and acts on the stage 504. An operation history is stored in a learning data storage unit 202 of the learning unit 201 to perform learning to determine parameter values ​​of the neural network. The operation history stored in the learning data storage unit 202 may include the position error, which is the input of the neural network, and the current command, which is the output, but may also include other information required for learning. The learning unit 201 may be configured, for example, by a CPU (Central Processing Unit) configured in the stage apparatus. Alternatively, the learning unit 201 may be configured by an external computer connected to the stage apparatus via a network.

[0017] In this embodiment, a decrease in the stability of learning caused by a mixture of behavioral histories based on a plurality of new and old parameter values ​​in the learning data is suppressed. Specifically, in this embodiment, the learning unit 201 executes second learning after the first learning by using only a second behavioral history acquired after the first behavioral history, without using the first behavioral history used in the first learning. A specific configuration for achieving this will be described below.

[0018] 5 shows a detailed configuration example of the learning unit 201. The learning unit 201 may include a memory unit 20, a learning control unit 203, a parameter calculation unit 205, and a parameter storage unit 206. The memory unit 20 includes a learning data storage unit 202 that provides a first storage area and a learning data duplication unit 204 that provides a second storage area. The first storage area of ​​the learning data storage unit 202 and the second storage area of ​​the learning data duplication unit 204 may be logically divided data storage areas within a single storage device shared by the learning data storage unit 202 and the learning data duplication unit 204. Alternatively, the first storage area of ​​the learning data storage unit 202 and the second storage area of ​​the learning data duplication unit 204 may be physically different data storage areas, i.e., data storage areas in separate storage devices.

[0019] The learning control unit 203 can also function as a storage control unit that controls the learning data storage unit 202 and the learning data duplication unit 204. The learning data storage unit 202 stores data including the operation history of the control target 103 transmitted from the control unit 102 to the learning unit 201. In response to receiving a data duplication command from the learning control unit 203, the learning data storage unit 202 transfers a copy of the learning data stored in the learning data storage unit 202 to the learning data duplication unit 204. The learning data duplication unit 204 stores the data transferred from the learning data storage unit 202. At this time, the learning data duplication unit 204 overwrites and saves the data stored therein with the data transferred from the learning data storage unit 202. Alternatively, the learning data duplication unit 204 may delete all data stored therein before storing the data transferred from the learning data storage unit 202. Furthermore, in response to receiving a data deletion command from the learning control unit 203, the learning data storage unit 202 deletes the learning data stored in the learning data storage unit 202. For example, after the learning data stored in the learning data storage unit 202 is duplicated in the learning data duplication unit 204, the learning control unit 203 outputs a data deletion command to the learning data storage unit 202. In response to the data deletion command, the learning data in the learning data storage unit 202 is deleted. This prevents behavioral histories based on multiple new and old parameter values ​​from being mixed in the learning data.

[0020] The parameter calculation unit 205 calculates parameter values ​​by performing reinforcement learning using the learning data stored in the learning data duplication unit 204. After completing the calculation of the parameter values, the parameter calculation unit 205 transmits the parameter values ​​to the parameter storage unit 206. The parameter storage unit 206 stores the parameter values ​​calculated by the parameter calculation unit 205. The learning control unit 203 controls the transfer of the parameter values ​​stored in the parameter storage unit 206 to the control unit 102. For example, the learning control unit 203 outputs a parameter update command. In response to the parameter update command output from the learning control unit 203, the parameter storage unit 206 outputs the updated parameter values ​​to the control unit 102. The learning control unit 203 transmits a learning sequence to the sequence unit 101 and receives a learning sequence completion signal from the sequence unit 101. After receiving the learning sequence completion signal from the sequence unit 101, the learning control unit 203 outputs a parameter update command to the parameter storage unit 206 before the learning sequence starts. Furthermore, after receiving the learning sequence completion signal, the learning control unit 203 outputs a data duplication command to the learning data storage unit 202. In response to receiving the data duplication command from the learning control unit 203, the learning data storage unit 202 transfers a copy of the learning data stored in the learning data storage unit 202 to the learning data duplication unit 204. After outputting the data duplication command, the learning control unit 203 outputs a data deletion command to the learning data storage unit 202. In response to receiving the data deletion command from the learning control unit 203, the learning data storage unit 202 deletes the learning data stored in the learning data storage unit 202. Note that instead of actually deleting the learning data from the learning data storage unit 202, it is also possible to simply invalidate the learning data so that it will not be used in subsequent learning.

[0021] As described above, the storage unit of the learning unit 201 includes a learning data storage unit 202 (first storage area) that stores learning data including the behavior history (operation history) of the control target 103, and a learning data duplication unit 204 (second storage area). The learning unit 201 (learning control unit 203) duplicates the learning data stored in the learning data storage unit 202 to the learning data duplication unit 204, and then deletes or invalidates the learning data in the learning data storage unit 202. The learning control unit 203 then executes learning using the learning data stored in the learning data duplication unit 204 and writing new learning data provided from the control unit 102 to the learning data storage unit 202 in parallel. The writing of new learning data provided from the control unit 102 to the learning data storage unit 202 is performed while the control unit 102 is controlling the control target 103. In this way, the learning unit 201 can perform the second learning after the first learning by using only the second operation history acquired after the first operation history, without using the first operation history used in the first learning. This limits the learning data used for learning to data acquired under conditions where parameter values ​​are fixed, preventing a decrease in learning stability. Furthermore, since the storage of learning data including the operation history and the calculation of parameter values ​​in the learning phase can be performed in parallel, the learning time required to stop production can be shortened.

[0022] The configuration of this embodiment is particularly useful in situations where the learning data acquisition time is long compared to the parameter calculation time. For example, let Ta be the time it takes for the control unit 102 to control the control target 103 and for the amount of learning data required for learning in the learning unit 201 to be accumulated in the learning data storage unit 202. This Ta is the learning data acquisition time. Also, let Tb be the time it takes for the learning unit 201 to execute learning. This Tb is the parameter calculation time. Therefore, a situation where the learning data acquisition time is long compared to the parameter calculation time is a situation where Ta>Tb. According to this embodiment, even under such a situation, it is possible to prevent parameter values ​​from being updated during learning data acquisition, preventing behavioral histories based on multiple parameter values ​​from being mixed in the learning data.

[0023] 6 is a flowchart showing the sequence of the learning process. In S400, the learning control unit 203 sets parameters for the neural network (NN) of the control unit 102. When learning parameters from scratch, the parameter values ​​may be initialized with random numbers. When re-learning is performed, the parameter values ​​that have been used up until that point may be used.

[0024] In S401, the learning control unit 203 initializes the learning data storage unit 202, the learning data duplication unit 204, and the parameter storage unit 206. In S402, the learning control unit 203 transmits a learning sequence to the sequence unit 101.

[0025] In S403, the learning data storage unit 202 stores the operation history data sent from the control unit 102. In S404, the learning control unit 203 waits to receive a learning sequence completion signal from the sequence unit 101. When the learning sequence completion signal is received, the process proceeds to S405.

[0026] In S405, the learning control unit 203 outputs a data duplication command to the learning data storage unit 202, causing a copy of the data stored in the learning data storage unit 202 to be stored in the learning data duplication unit 204. Thereafter, in S406, the learning control unit 203 outputs a data deletion command to the learning data storage unit 202, causing the data stored in the learning data storage unit 202 to be deleted (or invalidated).

[0027] In S407, a parameter calculation flow is performed, and in S408, a learning data acquisition flow is performed. The parameter calculation flow in S407 and the learning data acquisition flow in S408 can be performed in parallel. Details of the parameter calculation flow in S407 and the learning data acquisition flow in S408 will be described later. After the parameter calculation flow and the learning data acquisition flow are completed, in S409, the learning control unit 203 outputs a data duplication command to the learning data storage unit 202, and causes a copy of the data stored in the learning data storage unit 202 to be stored in the learning data duplication unit 204.

[0028] In S410, the learning control unit 203 updates the parameters of the neural network of the control unit 102. Next, in S411, the learning control unit 203 outputs a data deletion command to the learning data storage unit 202 to delete (or invalidate) the data stored in the learning data storage unit 202. Thereafter, in S412, the learning control unit 203 determines whether or not learning is complete. The determination of learning completion can be made, for example, by determining whether the number of times the learning sequence has been executed has reached a predetermined number of times. Alternatively, the determination of learning completion may be made by determining whether the control accuracy has reached a predetermined accuracy value. If learning is not complete, the process returns to S407 and is repeated.

[0029] FIG. 7 is a flowchart showing details of the parameter calculation flow in S407. In S416, the parameter calculation unit 205 calculates parameter values ​​using the learning data stored in the learning data duplication unit 204. In S426, the learning control unit 203 determines whether the parameter value calculation is complete. Specifically, for example, the learning control unit 203 determines whether the parameter value calculation using the learning data in the learning data duplication unit 204 has been performed a specified number of times. The specified number of times is a number determined in advance before the start of learning and may be any integer equal to or greater than 1. Since the execution time of the learning data acquisition flow S408 is determined by the execution time of the learning sequence, the specified number of times may be determined so that the execution time of the parameter calculation flow S407 is equal to or less than the execution time of the learning data acquisition flow S408. If the parameter value calculation is not complete, the process returns to S416, and the parameter value calculation is repeated. As a result, the learning unit 201 can perform learning multiple times using the learning data stored in the learning data duplication unit 204 until the amount of learning data required for learning is accumulated in the learning data storage unit 202 by the learning data acquisition flow in S408. Upon completion of the calculation of the parameter values, in S436 the learning control unit 203 stores the parameter values ​​obtained by the calculation in the parameter storage unit 206.

[0030] 8 is a flowchart showing the details of the learning data acquisition flow of S408. In S417, the learning control unit 203 transmits a learning sequence to the sequence unit 101. As a result, under the control of the control unit 102, control of the control object 103 is executed in accordance with the learning sequence. In S427, an operation history is stored in the learning data storage unit 202. The operation history may include at least the control amount output from the control unit 102, the operation amount, and the next control amount generated by the control object 103 actually operating in response to the control amount. In S437, the learning control unit 203 waits to receive a learning sequence completion signal from the sequence unit 101. When the learning sequence completion signal is received, the process proceeds to S409.

[0031] The following describes a learning method when the learning process sequence shown in Figures 6 to 8 is applied to a stage device. Learning is performed when the stage device is installed and at appropriate times when it is necessary to improve the stage control accuracy due to environmental changes, changes over time in the equipment, etc.

[0032] First, the learning control unit 203 transmits a learning sequence to the sequence unit 101. The learning sequence may be a production sequence. Alternatively, the learning sequence may be a learning sequence for vibrating a stage. The controller 501 outputs an operation amount based on a target value set by the sequence unit 101, and drives the stage 504, which is the object of control. The operation history of the stage 504 is stored in the learning data storage unit 202.

[0033] When the learning sequence of stage 504 is completed, the sequence unit 101 outputs a learning sequence completion signal to the learning control unit 203. When the learning control unit 203 receives the learning sequence completion signal, it outputs a data duplication command to the learning data storage unit 202. In response to receiving the data duplication command, the learning data storage unit 202 transfers a copy of the learning data stored in the learning data storage unit 202 to the learning data duplication unit 204. After outputting the data duplication command, the learning control unit 203 outputs a data deletion command to the learning data storage unit 202. In response to receiving the data deletion command from the learning control unit 203, the learning data storage unit 202 deletes the learning data stored in the learning data storage unit 202.

[0034] The parameter calculation unit 205 calculates parameter values ​​using the learning data stored in the learning data duplication unit 204. In parallel with the parameter value calculation, the learning control unit 203 transmits a learning sequence to the sequence unit 101 to start acquiring learning data, which is the operation history of the stage 504. Executing the parameter value calculation and the learning sequence in parallel reduces the learning time. The area storing the learning data used for the parameter value calculation (the learning data duplication unit 204) and the area storing the operation history during the operation of the learning sequence (the learning data storage unit 202) are separated into separate areas. This limits the data used for the learning data to data acquired under conditions where the parameter values ​​are fixed, preventing a decrease in learning stability. Furthermore, if the parameter value calculation is completed earlier than the learning sequence, it is also possible to learn the data stored in the learning data duplication unit 204 multiple times. For example, if the execution of the learning sequence takes three times the parameter calculation time, the parameter value calculation can be performed up to three times. This improves learning stability.

[0035] According to this embodiment, when obtaining optimal parameter values ​​through learning depending on the installation environment of the device and changes over time, it is possible to perform parameter value calculation and learning data acquisition in parallel while preventing a decrease in learning stability.

[0036] Second Embodiment FIG. 9 shows an example of the configuration of an exposure apparatus EXP according to the second embodiment. The exposure apparatus EXP may be configured as a scanning exposure apparatus. The exposure apparatus EXP may include, for example, an illumination light source 600, an illumination optical system 601, a mask stage 603, a projection optical system 604, and a plate stage 606. The illumination light source 600 may include, but is not limited to, a mercury lamp, an excimer laser light source, or an EUV light source. Exposure light 610 from the illumination light source 600 is shaped by the illumination optical system 601 to have uniform illuminance and to fit the shape of the irradiation area of ​​the projection optical system 604. In one example, the exposure light 610 is shaped into a rectangle whose length is in the X direction, which is the axis perpendicular to the plane defined by the Y and Z axes. Depending on the type of the projection optical system 604, the exposure light 610 may be shaped into an arc shape. The shaped exposure light 610 is irradiated onto the pattern of the mask 602, and the exposure light 610 that passes through the pattern of the mask 602 forms an image of the pattern of the mask 602 on the surface of a plate 605 (substrate) via a projection optical system 604.

[0037] The mask 602 is held by vacuum suction or the like on the mask stage 603. The plate 605 is held by vacuum suction or the like by a chuck 607 of the plate stage 606. The positions of the mask stage 603 and the plate stage 606 can be controlled by a multi-axis position control device equipped with a position sensor 630 such as a laser interferometer or a laser scale, a drive system 631 such as a linear motor, and a controller 632. The position measurement value output from the position sensor 630 can be provided to the controller 632. The controller 632 generates a control signal (operation amount signal) based on a position control deviation, which is the difference between the target position value and the measured position value, and provides this to the drive system 631 to drive the mask stage 603 and the plate stage 606. The pattern of the mask 602 is transferred to the plate 605 (the photosensitive agent thereon) by scanning and exposing the plate 605 while synchronously driving the mask stage 603 and the plate stage 606 in the Y direction.

[0038] The first embodiment will be described below when applied to the control of a plate stage 606. In FIG. 4, the controller 501 corresponds to the controller 632, the current driver 502 and motor 503 correspond to the drive system 631, the stage 504 corresponds to the plate stage 606, and the sensor 505 corresponds to the position sensor 630. Applying a controller having a neural network to the control of the plate stage 606 can reduce position control deviations for the plate stage 606. This can improve overlay accuracy and other parameters. Neural network parameter values ​​are learned when the state of the controlled object or the disturbance environment changes. Learning may also be performed periodically to maintain control performance. According to this embodiment, learning can be performed in a short time by performing parameter calculation and the learning sequence in parallel. Learning stability can be improved by performing learning any number of times while acquiring learning data. As a result, the number of times the learning sequence is executed can be reduced. As described above, learning can be performed in a short time, minimizing any decrease in productivity of the exposure apparatus and maintaining control accuracy.

[0039] This embodiment may be applied to the mask stage 603 in the same way as when it is applied to the plate stage 606. In this case, the same effects can be obtained.

[0040] The above describes the application of the first embodiment to control of a stage in an exposure apparatus. However, the first embodiment can be applied not only to control of a stage in an exposure apparatus, but also to control of a stage in other lithography apparatuses such as an imprint apparatus and an electron beam lithography apparatus. Furthermore, the first embodiment can also be applied to control, for example, a movable part in a transport mechanism that transports an object, such as a hand that holds the object.

[0041] The above-described lithography apparatus can be used to implement an article manufacturing method for manufacturing various articles (such as semiconductor IC elements, liquid crystal display elements, and MEMS). The article manufacturing method includes a transfer step of transferring a pattern of an original onto a substrate using the above-described lithography apparatus, and a processing step of processing the substrate that has undergone the transfer step, thereby obtaining an article from the substrate that has undergone the processing step. When the lithography apparatus is an exposure apparatus, the transfer step can include an exposure step of exposing the substrate through an original, and a development step of developing the substrate that has undergone the exposure step.

[0042] The disclosure of the present specification includes at least the following techniques. (Item 1) a control unit that controls a control target in accordance with a learning model whose parameter values ​​have been determined by learning; a learning unit that redetermines parameter values ​​of the learning model by learning using an operation history of the control object controlled by the control unit; Equipped with A control device characterized in that the learning unit performs second learning after the first learning using only second operation history acquired after the first operation history, without using the first operation history used in the first learning. (Item 2) the learning unit has a storage unit that provides a first storage area for storing learning data including an operation history of the control target and a second storage area different from the first storage area; The learning unit The learning data stored in the first storage area is copied to the second storage area, and then the learning data in the first storage area is deleted or invalidated; learning using the learning data stored in the second storage area and writing new learning data provided from the control unit into the first storage area in parallel; 2. The control device according to item 1, (Item 3) 3. The control device according to item 2, wherein new learning data provided from the control unit is written to the first storage area while the control unit is controlling the control target. (Item 4) 4. The control device according to item 3, wherein the time required for the amount of learning data required for learning by the learning unit to be accumulated in the first storage area is longer than the time required for the learning unit to perform the learning. (Item 5) 5. The control device according to item 4, wherein the learning unit executes learning using the learning data stored in the second storage area multiple times until the amount of learning data required for learning by the learning unit is accumulated in the first storage area. (Item 6) The learning unit a parameter calculation unit that calculates parameter values ​​of the learning model by performing reinforcement learning using the learning data stored in the second storage area; a parameter storage unit that stores the parameter values ​​calculated by the parameter calculation unit; a learning control unit that controls transfer of the parameter values ​​stored in the parameter storage unit to the control unit; 6. The control device according to any one of items 2 to 5, further comprising: (Item 7) 1. A lithography apparatus for transferring a pattern of an original onto a substrate, comprising: 7. A control device according to any one of items 1 to 6, configured to control the position of the substrate or the master. 1. A lithography apparatus comprising: (Item 8) A transfer step of transferring a pattern of an original onto a substrate using the lithography apparatus according to item 7; a processing step of processing the substrate that has undergone the transfer step, A method for manufacturing an article, characterized in that an article is obtained from the substrate that has been subjected to the processing step.

[0043] The invention is not limited to the above-described embodiments, and various changes and modifications can be made without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0044] 101: sequence unit, 102: control unit, 103: control target, 201: learning unit, 202: learning data storage unit, 203: learning control unit, 204: learning data duplication unit, 205: parameter calculation unit, 206: parameter storage unit

Claims

1. a control unit that controls a control target in accordance with a learning model whose parameter values ​​have been determined by learning; a learning unit that redetermines parameter values ​​of the learning model by learning using an operation history of the control object controlled by the control unit; Equipped with A control device characterized in that the learning unit performs second learning after the first learning using only second operation history acquired after the first operation history, without using the first operation history used in the first learning.

2. the learning unit has a storage unit that provides a first storage area for storing learning data including an operation history of the control target and a second storage area different from the first storage area, The learning unit The learning data stored in the first storage area is copied to the second storage area, and then the learning data in the first storage area is deleted or invalidated; learning using the learning data stored in the second storage area and writing new learning data provided by the control unit into the first storage area in parallel; 2. The control device according to claim 1.

3. 3. The control device according to claim 2, wherein new learning data provided by the control unit is written to the first storage area while the control unit is controlling the controlled object.

4. 4. The control device according to claim 3, wherein the time required for the amount of learning data required for learning by the learning unit to be accumulated in the first storage area is longer than the time required for the learning unit to perform the learning.

5. The control device according to claim 4, characterized in that the learning unit performs learning using the learning data stored in the second storage area multiple times until the amount of learning data required for learning in the learning unit is accumulated in the first storage area.

6. The learning unit a parameter calculation unit that calculates parameter values ​​of the learning model by performing reinforcement learning using the learning data stored in the second storage area; a parameter storage unit that stores the parameter values ​​calculated by the parameter calculation unit; a learning control unit that controls transfer of the parameter values ​​stored in the parameter storage unit to the control unit; 3. The control device according to claim 2, further comprising:

7. 1. A lithography apparatus for transferring a pattern of an original onto a substrate, comprising: A control device according to any one of claims 1 to 6, configured to control the position of the substrate or the original.

1. A lithography apparatus comprising:

8. a transfer step of transferring a pattern of an original onto a substrate using the lithography apparatus according to claim 7; a processing step of processing the substrate that has undergone the transfer step, A method for manufacturing an article, characterized in that an article is obtained from the substrate that has been subjected to the processing step.

Images