Management device, prediction method, and prediction program

Abstract

The present invention aims to provide a structure that predicts changes in process values in a control target and utilizes the prediction results. A management device of the present invention includes a prediction model section that learns an input-output relationship between a multivariate control value at time T in a control target and a multivariate process value at time T+ΔT in the control target, and an optimization model section that explores a multivariate control value at time T that minimizes each difference between the multivariate process value at time T+ΔT output by the prediction model section and a corresponding target value, and controls the control target using the explored multivariate control value, wherein, when there is a request from an agent section that manages the prediction model section, the prediction model section predicts the multivariate process value after time ΔT in the control target in a case where the control target is controlled with a specified control value, and outputs to the agent section.

Description

Technical Field

[0001] This invention relates to a management device, a forecasting method, and a forecasting program. Background Technology

[0002] In recent years, various efforts have been made towards the realization of smart factories in the field of substrate manufacturing processes. Specifically, the development of digital twin technology, which involves a management device collecting various data measured in the substrate manufacturing process (data in physical space) and reproducing the physical space in cyberspace, continues to advance.

[0003] At this point, when reproducing the physical space, for example, it is sought to accurately predict changes in processing values ​​within the controlled objects (physical space) using a model (network space) that reproduces the controlled objects (physical space) in each substrate processing apparatus performing the substrate manufacturing process. Furthermore, it is sought to construct a structure that effectively utilizes the processing values ​​predicted by the model.

[0004] Patent Document 1: International Publication No. 2020 / 050072

[0005] Patent Document 2: Japanese Patent Publication No. 2020-518079

[0006] Patent Document 3: Japanese Patent Application Publication No. 2018-092511 Summary of the Invention

[0007] <Problem to be solved by this invention>

[0008] This invention provides a structure for predicting changes in processed values ​​in a controlled object and utilizing the prediction results.

[0009] <Methods for solving problems>

[0010] A management device according to one aspect of the present invention, for example, has the following configuration. That is, it includes:

[0011] The predictive model unit learned the input-output relationship between the multivariate control value at time T and the multivariate processing value at time T+ΔT for the controlled object; and

[0012] The optimization model unit explores a multivariate control value at time T that minimizes the differences between the multivariate processing value at time T+ΔT output by the prediction model unit and the corresponding target value, and uses the explored multivariate control value to control the controlled object.

[0013] When a request is received from the agent unit that manages the prediction model unit, the prediction model unit predicts the multivariate processing value after time ΔT in the controlled object when the controlled object is controlled with a specified control value, and outputs it to the agent unit.

[0014] <The Effects of the Invention>

[0015] According to the present invention, a structure can be provided for predicting changes in processed values ​​in a controlled object and utilizing the prediction results. Attached Figure Description

[0016] Figure 1 This is a diagram illustrating an example of a cyberphysical system configuration that includes multiple substrate processing devices for performing substrate manufacturing processes.

[0017] Figure 2 This is a diagram illustrating an example of the hardware configuration of a management device.

[0018] Figure 3 This is a diagram illustrating an example of the functional configuration of a cyber-physical system during the control phase.

[0019] Figure 4 This is an example of a flowchart representing the control process.

[0020] Figure 5 This is a diagram illustrating an example of the functional configuration of a cyber-physical system during the simulation phase.

[0021] Figure 6 This is an example of a flowchart representing the process of simulation processing.

[0022] Figure 7 The first diagram is an example of the functional configuration of a cyber-physical system during the learning phase.

[0023] Figure 8 This is an example of a flowchart representing the process of the first learning step.

[0024] Figure 9 The second figure is an example of the functional configuration of a cyber-physical system during the learning phase.

[0025] Figure 10 This is an example of a flowchart representing the process of the second learning process. Detailed Implementation

[0026] Hereinafter, each embodiment will be described with reference to the accompanying drawings. Furthermore, in this specification and the accompanying drawings, components having substantially the same functional configuration are labeled with the same symbols, and repeated descriptions are omitted.

[0027] (First Implementation)

[0028] <System Configuration of Cyber-Physical Systems>

[0029] First, the system configuration of the cyber-physical system with multiple substrate processing devices that perform substrate manufacturing processes is explained. Figure 1 This is a diagram illustrating an example of a network physical system configuration that includes multiple substrate processing devices for performing substrate manufacturing processes.

[0030] like Figure 1 As shown, the network physical system 100 includes server devices 110_1 to 110_3, management devices 120_1 to 120_n, baseboard processing devices 130_1 to 130_n, and an administrator terminal 140.

[0031] In the network physical system 100, server devices 110_1 to 110_3, management devices 120_1 to 120_n, and administrator terminal 140 are connected via network 150 and can communicate with each other.

[0032] Server devices 110_1 to 110_3 are devices that manage the entire network physical system 100. For example, server devices 110_1 to 110_3 perform manufacturing management, data management, device management, and management of the substrate manufacturing process executed by each substrate processing device 130_1 to 130_n, as well as the management of the models used by each management device 120_1 to 120_n in cyberspace.

[0033] The management devices 120_1 to 120_n are respectively connected to the substrate processing devices 130_1 to 130_n to form a management system.

[0034] Furthermore, the management devices 120_1 to 120_n have various models that reproduce each controlled object within their corresponding substrate processing devices 130_1 to 130_n, forming a network space. The management devices 120_1 to 120_n use the control values ​​calculated through these various models to perform control processing for each controlled object.

[0035] Specifically, the management devices 120_1 to 120_n calculate a control value that minimizes the difference between the processing value predicted by the various models and the target value, and then control the controlled object. That is, the management devices 120_1 to 120_n can control the controlled object after predicting changes in the processing value.

[0036] Furthermore, the management devices 120_1 to 120_n collect data from the physical space acquired by the substrate processing devices 130_1 to 130_n.

[0037] • Monitor the status of substrate processing devices 130_1 to 130_n.

[0038] • Detects events occurring in the substrate processing apparatus 130_1 to 130_n.

[0039] And to respond appropriately to various situations that occur in physical space.

[0040] Furthermore, when dealing with various events occurring in the physical space, management devices 120_1 to 120_n perform simulation processing in the cyberspace using the aforementioned models. Thus, management devices 120_1 to 120_n can, for example, predict how the processing value in the controlled object (physical space) will change after time ΔT if the control value is changed at time T.

[0041] That is, the management devices 120_1 to 120_n can respond to the situation after the change of the processing value in the controlled object (physical space) after the time ΔT.

[0042] The substrate processing apparatuses 130_1 to 130_n are apparatuses that perform substrate manufacturing processes and constitute a physical space. The substrate processing apparatuses 130_1 to 130_n may include, for example, apparatuses for performing film deposition processes, apparatuses for performing photolithography processes, apparatuses for performing etching processes, and apparatuses for performing cleaning processes.

[0043] The substrate processing apparatuses 130_1 to 130_n have multiple control objects, and each control object is controlled based on a control value calculated by the management devices 120_1 to 120_n. Furthermore, the substrate processing apparatuses 130_1 to 130_n transmit data acquired in the physical space during the execution of the substrate manufacturing process to the management devices 120_1 to 120_n.

[0044] Administrator terminal 140 is a terminal operated by an administrator managing the network physical system 100. Administrator terminal 140 generates, for example, various models used by management devices 120_1 to 120_n. Specifically, first, administrator terminal 140 acquires measured processing values ​​and measured control values ​​of controlled objects from data collected in the physical space by management devices 120_1 to 120_n, and generates learning data. Then, administrator terminal 140 uses the generated learning data for learning processing, thereby generating various models.

[0045] It should be noted that, Figure 1 The network physical system 100 shown depicts a configuration where the management devices 120_1 to 120_n and the substrate processing devices 130_1 to 130_n are configured separately. However, the management devices 120_1 to 120_n and the substrate processing devices 130_1 to 130_n can also be configured as a single unit.

[0046] <Hardware Configuration of Management Devices and Administrator Terminals>

[0047] Next, the hardware configuration of management devices 120_1 to 120_n and administrator terminal 140 will be described. It should be noted that management devices 120_1 to 120_n and administrator terminal 140 all have the same hardware configuration; therefore, the hardware configuration of management devices 120_1 to 120_n will be described here. Figure 2 A unified explanation. Figure 2 This is a diagram illustrating an example of the hardware configuration of a management device.

[0048] like Figure 2 As shown, the management devices 120_1 to 120_n include a processor 201, a memory 202, an auxiliary storage device 203, an I / F (Interface) device 204, a communication device 205, and a drive device 206. It should be noted that the hardware components of the management devices 120_1 to 120_n are interconnected via a bus 207.

[0049] The processor 201 has various computing devices such as a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit). The processor 201 reads various programs (e.g., prediction programs described below) from the memory 202 and executes them.

[0050] The memory 202 includes main storage devices such as ROM (Read Only Memory) and RAM (Random Access Memory). The processor 201 and the memory 202 form what is commonly referred to as a computer, which performs various functions by having the processor 201 execute various programs read from the memory 202.

[0051] The auxiliary storage device 203 stores various programs and various data used when the processor 201 executes the various programs.

[0052] I / F device 204 is a connection device that connects an external device, namely a substrate processing device 130_1 to 130_n, to a management device 120_1 to 120_n.

[0053] The communication device 205 is a communication device used to communicate with other devices (in this embodiment, server devices 110_1 to 110_3, other management devices, administrator terminal 140, etc.) via the network 150.

[0054] The drive device 206 is a means for setting the recording medium 210. The recording medium 210 mentioned here includes media that record information in an optical, electrical, or magnetic manner, such as CD-ROM, floppy disk, and magneto-optical disk. In addition, the recording medium 210 may also include semiconductor memory such as ROM and flash memory that record information electrically.

[0055] It should be noted that various programs installed to the auxiliary storage device 203 can be installed, for example, by placing the distributed recording medium 210 on the drive device 206 and having the drive device 206 read the various programs recorded on the recording medium 210. Alternatively, various programs installed to the auxiliary storage device 203 can also be downloaded from the network via the communication device 205.

[0056] <Functional Configuration and Various Processing of Network Physical Systems>

[0057] Next, the functional configuration of the cyber-physical system 100 and the details of the various processes performed within it will be explained. As described above, in the cyber-physical system 100, different functions are activated at the following stages:

[0058] • In the control phase, management devices 120_1 to 120_n utilize various models for reproducing the controlled object to perform control processing of the controlled object.

[0059] • During the simulation phase, management devices 120_1 to 120_n monitor the state of the board processing device and, upon detecting an incident, perform simulation processing using various models for reproducing the controlled object to respond to the incident.

[0060] • During the learning phase, the administrator terminal 140 uses the measured processing values ​​and measured control values ​​in the controlled object to generate learning data, and uses the generated learning data for learning processing to generate various models.

[0061] The following section will explain the functional configuration and processing details of the network physical system 100 in stages.

[0062] (1) Control phase

[0063] (1-1) Functional Configuration

[0064] First, the functional configuration of the cyber-physical system 100 in the control phase will be explained. Figure 3 This is a diagram illustrating an example of the functional configuration of a cyber-physical system during the control phase.

[0065] As described above, prediction programs are installed in management devices 120_1 to 120_n respectively. Furthermore, by executing these prediction programs in management devices 120_1 to 120_n, an agent unit 320 and a model prediction control unit 330 are implemented in each management device in the network space 310 during the control phase.

[0066] During the control phase, the agent unit 320 sets multivariate target values ​​for the model prediction control unit 330. These target values ​​refer to the target values ​​used by the model prediction control unit 330 when performing control processing on the controlled object (physical space 340) within the substrate processing apparatus 130_1 to 130_n. It should be noted that... Figure 3 The example shows that the controlled object (physical space 340) has sensors 341_1 to 341_n and actuators 342_1 to 342_m.

[0067] like Figure 3 As shown, for a controlled object having sensors 341_1 to 341_n and actuators 342_1 to 342_m, in order to make the multivariable (n) measured processed values ​​consistent with the corresponding multivariable (n) target values, the operation of multiple (m) actuators is controlled. Therefore, the agent unit 320 sets the multivariable (n) target values ​​to the model prediction control unit 330.

[0068] The model prediction control unit 330 includes a prediction model unit 331, an objective function unit 332, an optimization model unit 333, and a verification unit 334.

[0069] The prediction model unit 331 is a model that models the behavior of the controlled object (physical space 340). It is a model that learns the input-output relationship between the multivariate measured control value of the controlled object at time T and the multivariate measured processing value of the controlled object at time T+ΔT.

[0070] The objective function unit 332 calculates the differences between the multivariate prediction processing value at time T+ΔT output by the prediction model unit 331 and the multivariate target value set by the agent unit 320, and notifies the optimization model unit 333.

[0071] The optimization model unit 333 explores the multivariate control value at time T that minimizes the differences notified by the objective function unit 332. Furthermore, the optimization model unit 333 inputs the explored multivariate control values ​​to the prediction model unit 331, and again obtains the differences between the multivariate predicted processed value at time T+ΔT output by the prediction model unit 331 and the multivariate target value. In the optimization model unit 333, these processes are repeatedly performed to minimize the differences, and the optimal multivariate control value at time T is calculated.

[0072] In addition, the optimization model unit 333 sends the optimal multivariable control value at time T to the actuators 342_1 to 342_m of the controlled object (physical space 340) (actually, it sends the value to the corresponding board processing device, and then notifies each actuator from there).

[0073] Therefore, the optimization model unit 333 can control the controlled object (physical space 340) after predicting the multivariate prediction processing value.

[0074] The verification unit 334 obtains the optimal multivariate control value from the optimization model unit 333. Furthermore, in response to the transmission of the optimal multivariate control value to the actuators 342_1 to 342_m of the controlled object (physical space 340), the verification unit 334 obtains the measured multivariate processing value from the physical space 340.

[0075] Furthermore, the verification unit 334 determines whether the control value is appropriate based on the optimal multivariate control value and the acquired multivariate measured processing value, and verifies the prediction accuracy of the prediction model unit 331, adjusting the model parameters of the prediction model unit 331 as needed. Thus, the verification unit 334 can make the behavior of the prediction model unit 331 consistent with the behavior of the controlled object in the physical space 340 (that is, it can make the predicted processing value consistent with the measured processing value).

[0076] On the other hand, the physical space 340 formed by the substrate processing apparatuses 130_1 to 130_n contains a plurality of control objects. Figure 3 The example represents a controlled object that has sensors 341_1 to 341_n and actuators 342_1 to 342_m.

[0077] Sensors 341_1 to 341_n measure and process values ​​such as flow rate and pressure information. The flow rate and pressure information measured by sensors 341_1 to 341_n are provided to network space 310 as actual measured values.

[0078] Actuators 342_1 to 342_m operate based on instructions from cyberspace 310. Figure 3 The example shows that actuators 342_1 to 342_m operate based on the optimal multivariable control values ​​calculated by the model predictive control unit 330.

[0079] (1-2) Control Processing

[0080] Next, the flow of control processing performed by the cyber-physical system 100 during the control phase will be explained. Figure 4 This is an example of a flowchart representing the control process.

[0081] In step S401, the model prediction control unit 330 determines whether a new multivariate target value has been set by the agent unit 320.

[0082] If, in step S401, it is determined that no new multivariate target value has been set (the case where "No" is true in step S401), proceed to step S404.

[0083] On the other hand, if it is determined in step S401 that a new multivariate target value has been set (the case where it is "yes" in step S401), proceed to step S402.

[0084] In step S402, the model prediction control unit 330 explores the optimal multivariate control value at time T, so as to output the multivariate prediction processing value at time T+ΔT from the prediction model unit 331 for minimizing the differences with the new multivariate target value.

[0085] In step S403, the model prediction control unit 330 sends the optimal multivariate control value at time T obtained through exploration to the actuators 342_1 to 342_m of the controlled object (physical space 340).

[0086] In step S404, the model prediction control unit 330, in response to the transmission of the optimal multivariate control value at time T, acquires the multivariate measured processing value at time T+ΔT provided by the physical space 340.

[0087] In step S405, the model prediction control unit 330 determines whether the control value at time T is appropriate based on the optimal multivariate control value at time T and the obtained multivariate measured processing value at time T+ΔT, and verifies the prediction accuracy of the prediction model unit 331.

[0088] In step S406, the model prediction control unit 330 determines whether it is necessary to adjust the model parameters of the prediction model unit 331.

[0089] If it is determined in step S406 that no adjustment of the model parameters is required (the case where "No" is determined in step S406), proceed to step S408.

[0090] On the other hand, in step S406, if it is determined that the model parameters need to be adjusted (if "yes" is in step S406), proceed to step S407.

[0091] In step S407, the model prediction control unit 330 adjusts the model parameters of the prediction model unit 331.

[0092] In step S408, the model prediction control unit 330 determines whether to end the control process.

[0093] If it is determined in step S408 that the control process should not be terminated (the case where "No" is determined in step S408), return to step S401.

[0094] On the other hand, if it is determined in step S409 that the control process is to end (the case where it is "yes" in step S408), the control process ends.

[0095] (2) Simulation phase

[0096] (2-1) Function Configuration

[0097] Next, the functional configuration of the cyber-physical system 100 during the simulation phase will be explained. Figure 5 This is a diagram illustrating an example of the functional configuration of a cyber-physical system during the simulation phase.

[0098] Its and already utilized Figure 3 The difference in the functional configuration of the network physical system 100 in the control phase described lies in the function of the agent unit 320 and the composition of the physical space 340.

[0099] During the simulation phase, Agent 320 collects data from the physical space (any data other than measured processed values, such as particle information, maintenance information, device configuration information, operation information, etc.).

[0100] • Monitor the status of substrate processing devices 130_1 to 130_n.

[0101] • Detects events occurring in the substrate processing apparatus 130_1 to 130_n.

[0102] Such as these, and appropriately responding to various situations occurring in physical space 340. It should be noted that... Figure 5 The data in the physical space shown is just one example; for instance, it could also include information measured about the processing state of the substrate, in place of particle information (or in addition to particle information). Furthermore, Figure 5 In the example, the machine used to measure particle information is designated as an external measuring machine. However, the machine used to measure particle information is not limited to an external measuring machine; it can also be an internal measuring machine installed within the substrate processing apparatuses 130_1 to 130_n. For example, it could be a machine that measures the internal state of the substrate processing apparatuses 130_1 to 130_n through a window provided on the wall of the substrate processing apparatuses 130_1 to 130_n. Furthermore, the machine used to measure particle information could also be a machine that observes the state on the substrate to be processed, or it could be a machine that acquires the state of the processing space where the substrate to be processed is processed.

[0103] Here, it is assumed that when the agent unit 320 responds to a situation, it determines that it is necessary to change the control value sent to the controlled object (physical space 340). At this time, the agent unit 320 requests the model prediction control unit 330 to perform simulation processing to predict "how the measured processing value in the controlled object (physical space 340) will change after time ΔT when the control value sent to the controlled object is changed".

[0104] Figure 5 The example shows that the agent unit 320 requests the model prediction control unit 330 to predict the state of the prediction processing value at time T+ΔT "when the multivariate control value is changed at time T". Specifically, the agent unit 320 inputs the changed multivariate control value at time T to the prediction model unit 331.

[0105] Therefore, the agent unit 320 can obtain the predicted processing value at time T+ΔT from the model prediction control unit 330. As a result, the agent unit 320 is able to respond to the situation after knowing the change in the processing value of the controlled object (physical space 340) at time ΔT when the control value changes at time T.

[0106] (2-2) Simulation Processing

[0107] Next, the simulation process performed by the cyber-physical system 100 during the simulation phase will be explained. Figure 6 This is an example of a flowchart representing the process of simulation processing.

[0108] In step S601, the model prediction control unit 330 determines whether the modified multivariate control value specified by the agent unit 320 has been input.

[0109] If, in step S601, it is determined that the specified modified multivariate control value has not yet been entered (in the case of "No" in step S601), proceed to step S604.

[0110] On the other hand, if it is determined in step S601 that the specified modified multivariate control value has been entered (if it is "yes" in step S601), proceed to step S602.

[0111] In step S602, the model prediction control unit 330 calculates the prediction processing value after prediction time ΔT.

[0112] In step S603, the model prediction control unit 330 outputs the prediction processing value after time ΔT to the agent unit 320.

[0113] In step S604, the model prediction control unit 330 determines whether to end the simulation process. If it is determined in step S604 that the simulation process should not be ended (the case where it is "No" in step S604), the process returns to step S601.

[0114] On the other hand, in step S604, if it is determined that the simulation process is to be terminated (the case where "yes" is in step S604), the simulation process is terminated.

[0115] (3) Learning Phase (Predictive Model Department)

[0116] (3-1) Functional Configuration

[0117] Next, the functional configuration of the cyber-physical system 100 during the learning phase, in which the prediction model is processed, will be explained. Figure 7 The first diagram is an example of the functional configuration of a cyber-physical system during the learning phase.

[0118] A first learning program is installed on the administrator terminal 140. By executing the first learning program, the administrator terminal 140 functions as the first learning unit 720.

[0119] The first learning unit 720 stores the first learning data in the first learning data storage unit 723. Specifically, the first learning unit 720 stores the first learning data in the management devices 120_1 to 120_n. Figure 7 (Not shown in the figure) Obtain the multivariate measured control value at time T and the multivariate measured processed value at time T+ΔT provided by physical space 340.

[0120] Figure 7 An example shows that in physical space 340, substrate processing apparatus 130_1 to 130_n has a controlled object (sensors 341_1 to 341_n, actuators 342_1 to 342_m) and a controller 730 for controlling the controlled object.

[0121] The controller 730 controls the actuators 342_1 to 342_m so that the multivariate measured values ​​obtained by the sensors 341_1 to 341_n are close to the multivariate target values ​​set by the controller 730.

[0122] The first learning unit 720 associates and obtains the multivariate measured control value at time T with the multivariate measured processing value at time T+ΔT, thereby generating first learning data and accumulating it in the first learning data accumulation unit 723.

[0123] Furthermore, the first learning unit 720 has a prediction model unit 721 and a comparison / modification unit 722, and performs learning processing on the prediction model unit 721.

[0124] Specifically, the first learning unit 720 reads the first learning data from the first learning data accumulation unit 723 and inputs the multivariate measured control value at time T to the prediction model unit 721. As a result, the prediction model unit 721 outputs the multivariate predicted processing value at time T+ΔT.

[0125] Furthermore, the first learning unit 720 inputs the multivariate measured processing value at time T+ΔT as the forward solution data to the comparison / modification unit 722. The comparison / modification unit 722 then calculates the errors between the multivariate predicted processing value at time T+ΔT and the multivariate measured processing value at time T+ΔT, and backpropagates these errors to update the model parameters of the prediction model unit 721. Thus, the first learning unit 720 can learn the input-output relationship between the multivariate measured control value at time T and the multivariate measured processing value at time T+ΔT in the controlled object, and can generate the prediction model unit 331 after learning.

[0126] (3-2) First learning process

[0127] Next, the flow of the first learning process performed by the cyber-physical system 100 during the learning phase will be explained. Figure 8 This is an example of a flowchart representing the process of the first learning step.

[0128] In step S801, in the physical space 340, the controller 730 in the substrate processing apparatus 130_1 to 130_n begins the control processing of the controlled object.

[0129] In step S802, the first learning unit 720 of the administrator terminal 140 acquires the multivariate measured control values ​​at time T provided by the physical space 340.

[0130] In step S803, the first learning unit 720 of the administrator terminal 140 acquires the multivariate measured processing value at time T+ΔT provided by the physical space 340.

[0131] In step S804, the first learning unit 720 of the administrator terminal 140 associates the acquired multivariate measured control value at time T with the multivariate measured processing value at time T+ΔT, and generates the first learning data.

[0132] In step S805, the first learning unit 720 of the administrator terminal 140 uses the generated first learning data to perform learning processing on the prediction model unit 721.

[0133] In step S806, the first learning unit 720 of the administrator terminal 140 determines whether the learning process performed on the prediction model unit 721 has been completed. If it is determined in step S806 that the learning process has not been completed (the case where it is "No" in step S806), the process returns to step S802.

[0134] On the other hand, if it is determined in step S806 that the learning process has been completed (if it is "yes" in step S806), proceed to step S807.

[0135] In step S807, the first learning unit 720 of the administrator terminal 140 outputs the prediction model unit that has completed learning.

[0136] (4) Learning Phase (Optimization Model Department)

[0137] (4-1) Functional Configuration

[0138] Next, the functional configuration of the cyber-physical system 100 during the learning phase, which performs learning processing on the optimization model, will be explained. Figure 9 The second figure is an example of the functional configuration of a cyber-physical system during the learning phase.

[0139] A second learning program is installed on the administrator terminal 140. By executing the second learning program, the administrator terminal 140 functions as the second learning unit 920.

[0140] The second learning unit 920 stores the second learning data in the second learning data storage unit 924. Specifically, the second learning unit 920 acquires the differences at time T+ΔT calculated by the difference calculation unit 923, and the multivariable measured control values ​​at time T provided from the physical space 340. Furthermore, the difference calculation unit 923 transmits the data via management devices 120_1 to 120_n (in... Figure 9 (Not shown in the figure) Obtain the measured processing value and target value at time T+ΔT provided by physical space 340, and then calculate the differences.

[0141] Figure 9 An example shows that in physical space 340, substrate processing apparatus 130_1 to 130_n has a controlled object (sensors 341_1 to 341_n, actuators 342_1 to 342_m) and a controller 730 for controlling the controlled object.

[0142] The controller 730 controls the actuators 342_1 to 342_m so that the multivariate measured values ​​obtained by the sensors 341_1 to 341_n are close to the multivariate target values ​​set by the controller 730.

[0143] The difference calculation unit 923 acquires the multivariate target value at this time and the multivariate measured processing value at time T+ΔT, and calculates the differences at time T+ΔT. Furthermore, the second learning unit 920 acquires the multivariate measured control value at time T, and correlates it with the differences at time T+ΔT calculated by the difference calculation unit 923, thereby generating second learning data, which is then stored in the second learning data storage unit 924.

[0144] Furthermore, the second learning unit 920 has an optimization model unit 921 and a comparison / change unit 922, and performs learning processing on the optimization model unit 921.

[0145] Specifically, the second learning unit 920 reads the second learning data from the second learning data accumulation unit 924 and inputs the differences at time T+ΔT to the optimization model unit 921. As a result, the optimization model unit 921 outputs the multivariate control value at time T.

[0146] Furthermore, the second learning unit 920 inputs the multivariate measured control value at time T as forward solution data to the comparison / modification unit 922. The comparison / modification unit 922 then calculates the errors between the multivariate control value at time T and the multivariate measured control value at time T, and backpropagates the calculated errors to update the model parameters of the optimization model unit 921. Thus, the second learning unit 920 can learn the input-output relationship between the difference at time T+ΔT and the control value at time T, and generate the optimized model unit 333, which has completed the learning process.

[0147] (4-2) Second learning process

[0148] Next, the flow of the second learning process performed by the cyber-physical system 100 during the learning phase will be explained. Figure 10 This is an example of a flowchart representing the process of the second learning process.

[0149] In step S1001, in the physical space 340, the controller 730 in the substrate processing apparatus 130_1 to 130_n begins the control processing of the controlled object.

[0150] In step S1002, the second learning unit 920 of the administrator terminal 140 acquires the multivariate target value provided by the physical space 340. Furthermore, the second learning unit 920 of the administrator terminal 140 acquires the multivariate measured processing value at time T+ΔT provided by the physical space 340.

[0151] In step S1003, the second learning unit 920 of the administrator terminal 140 calculates the differences between the multivariate target value and the multivariate measured processing value at time T+ΔT.

[0152] In step S1004, the second learning unit 920 of the administrator terminal 140 acquires the multivariate measured control values ​​at time T provided by the physical space 340.

[0153] In step S1005, the second learning unit 920 of the administrator terminal 140 correlates the calculated differences at time T+ΔT with the acquired multivariate measured control values ​​at time T, and generates second learning data.

[0154] In step S1006, the second learning unit 920 of the administrator terminal 140 uses the generated second learning data to perform learning processing on the optimization model unit 921.

[0155] In step S1007, the second learning unit 920 of the administrator terminal 140 determines whether the learning process performed on the optimization model unit 921 has been completed. If it is determined in step S1007 that the learning process has not been completed (the case where "No" is determined in step S1007), the process returns to step S1002.

[0156] On the other hand, if it is determined in step S1007 that the learning process has been completed (if it is "yes" in step S1007), proceed to step S1008.

[0157] In step S1008, the second learning unit 920 of the administrator terminal 140 outputs the optimized model unit that has completed learning.

[0158] <Summary>

[0159] As can be seen from the above description, in the management device of the first embodiment:

[0160] • It has a predictive model unit that learns the input-output relationship between the multivariate measured control value at time T and the multivariate measured processing value at time T+ΔT in the controlled object.

[0161] • It has an optimization model unit that explores the multivariate control value at time T that minimizes the differences between the multivariate processing value at time T+ΔT output by the prediction model unit and the corresponding target value, and uses the explored optimal multivariate control value at time T to control the controlled object.

[0162] When a request is received from the agent department, the prediction model department predicts the multivariate processing value of the control object after time ΔT when the control object is controlled by the control value specified at time T, and outputs it to the agent department.

[0163] Thus, according to the management device of the first embodiment, by having a prediction model unit, changes in processing values ​​in the controlled object can be predicted with high accuracy. Furthermore, according to the management device of the first embodiment, by employing a configuration that calculates the optimal control value based on the predicted processing value, the predicted processing value can be used to perform control processing on the controlled object. Moreover, according to the management device of the first embodiment, by employing a configuration that predicts the processing value in response to a request from the agent and outputs it to the agent, the predicted processing value can be used to respond to events occurring in the physical space.

[0164] That is, according to the first embodiment, a structure can be provided that predicts changes in the processed value in the controlled object and utilizes the prediction results.

[0165] (Second Implementation)

[0166] In the first embodiment described above, the management devices 120_1 to 120_n are each configured as separate management devices, but the management devices 120_1 to 120_n can also be configured as a single device. In this case, n management devices can be virtually operated (i.e., as virtual machines) on a single device.

[0167] Furthermore, in the first embodiment described above, the management devices 120_1 to 120_n corresponding to the substrate processing devices 130_1 to 130_n are described as devices that execute the prediction program individually. However, the management device (e.g., management device 120_1) corresponding to one substrate processing device (e.g., substrate processing device 130_1) may be composed of multiple computers. Furthermore, the prediction program can be installed on multiple computers respectively, thereby executing the prediction program in a distributed computing manner.

[0168] Furthermore, in the first embodiment described above, as an example of a method for installing the prediction program into the auxiliary storage device 203 of the management devices 120_1 to 120_n, a method of downloading and installing via a network is mentioned. While the download source is not specifically mentioned, in the case of installation using this method, the download source could be, for example, a server device that stores the prediction program in an accessible manner. Furthermore, this server device could also be a cloud-based device that accepts access from each of the management devices 120_1 to 120_n via a network and downloads the prediction program on a paid basis. That is, the server device could be a cloud-based device that provides a prediction program service.

[0169] Furthermore, while the first embodiment described above illustrates the case where a network space is formed within a management system comprising multiple management devices 120_1 to 120_n, a network space can also be formed outside the management system. For example, a network space can be formed within server devices 110_1 to 110_3.

[0170] Furthermore, in the first embodiment described above, the case where the first learning process and the second learning process are executed in the administrator terminal 140 is explained. However, the first learning process and the second learning process may also be executed in the management devices 120_1 to 120_n or the server devices 110_1 to 110_3.

[0171] In other words, the functions of actions in each stage—control, simulation, and learning—can be implemented in separate devices or in a single integrated device.

[0172] Furthermore, the first embodiment described above illustrates the case where control processing and simulation processing are performed at different stages, but they can also be performed in parallel at the same stage.

[0173] Furthermore, in the first embodiment described above, the output objects of the prediction model unit and the optimization model unit that have completed the learning process are not mentioned, but the prediction model unit and the optimization model unit that have completed the learning process can be output to the management devices 120_1 to 120_n. Alternatively, they can be logged to the server devices 110_1 to 110_n in a downloadable manner.

[0174] Furthermore, while no details about the model are mentioned in the first embodiment described above, the model used in the first embodiment may be, for example, a machine learning model incorporating deep learning, such as:

[0175] • RNN (Recurrent Neural Network)

[0176] • LSTM (Long Short-Term Memory) network

[0177] • CNN (Convolutional Neural Network)

[0178] R-CNN (Region-based Convolutional Neural Network)

[0179] YOLO (You Only Look Once): an object detection algorithm that requires only one look.

[0180] • SSD (Single Shot MultiBox Detector)

[0181] • GAN (Generative Adversarial Network)

[0182] • SVM (Support Vector Machine)

[0183] Decision tree

[0184] Random Forest

[0185] Any one of them.

[0186] It should be noted that, alternatively, models using genetic algorithms such as GA (Genetic Algorithm) and GP (Genetic Programming) or models learned through reinforcement learning can also be used.

[0187] Alternatively, the model used in the first embodiment described above can also be a general statistical analysis other than deep learning, such as PCR (Principal Component Regression), PLS (Partial Least Squares), LASSO, ridge regression, linear polynomial, autoregressive model, moving average model, autoregressive moving average model, or ARX model. Alternatively, the above models can be combined.

[0188] It should be noted that the present invention is not limited to the structures listed in the above embodiments, combinations with other elements, or the structures shown herein. In this regard, changes can be made without departing from the spirit of the invention, and it can be appropriately specified according to its application.

[0189] This application is based on and claims priority to Japanese Patent Application No. 2020-218532, filed on December 28, 2020, the entire contents of which are incorporated herein by reference.

[0190] Explanation of reference numerals in the attached figures

[0191] 100 Network Physical Systems

[0192] 120_1~120_n Management Device

[0193] 130_1~130_n Substrate Processing Apparatus

[0194] 140 Administrator Terminal

[0195] 310 Cyberspace

[0196] 320 Agency

[0197] 330 Model Predictive Control Unit

[0198] 340 physical space

[0199] 720 First Study Department

[0200] 920 Second Study Department.

Claims

1. A management device, comprising: The predictive model unit learned the input-output relationship between the multivariate control value at time T and the multivariate processing value at time T+ΔT of the controlled object in the physical space. as well as The optimization model unit explores the multivariate control value at time T that minimizes the differences between the multivariate processing value at time T+ΔT output by the prediction model unit and the corresponding target value, and uses the explored multivariate control value to control the controlled object. The prediction model unit is configured to, when determining that a request needs to be issued from the agent unit to change the multivariate control value based on events occurring in the physical space, predict the multivariate processing value after time ΔT in the controlled object when the controlled object is controlled with the specified control value, and output it to the agent unit. The agency is configured to respond to the events occurring in the physical space based on the predicted multivariate processing values.

2. The management device according to claim 1, further comprising: The first learning unit performs learning processing on the prediction model unit so that the multivariate output of the prediction model unit, when given the multivariate measured control value at time T, is close to the multivariate measured processing value of the controlled object at time T + ΔT.

3. The management device according to claim 1, wherein, Determine whether a new multivariate target value has been set by the aforementioned agency. When it is determined that the new multivariate target value has been set by the agent, after exploring the optimal multivariate control value for minimizing the differences between the multivariate processed value and the new multivariate target value, and sending the explored optimal multivariate control value to the controlled object, the measured multivariate processed value is obtained from the physical space. When it is determined that the new multivariate target value has not yet been set by the agent, the measured multivariate processing value is obtained from the physical space, without exploring the optimal multivariate control value.

4. The management device according to claim 1, further comprising: The second learning unit performs learning processing on the optimization model unit so that the multivariate control value at time T, which is output by the optimization model unit based on the differences between the multivariate measured processing value at time T+ΔT input to the controlled object and the corresponding target value, is close to the multivariate measured control value at time T when the controlled object is controlled.

5. A prediction method, comprising the following steps: The steps of the predictive model unit are as follows: the predictive model unit learns the input-output relationship between the multivariate control value at time T and the multivariate processing value at time T+ΔT of the controlled object in the physical space; and The optimization model unit performs the following steps: it explores a multivariate control value at time T that minimizes the differences between the multivariate processing value at time T + ΔT output by the prediction model unit and the corresponding target value, and uses the explored multivariate control value to control the controlled object. in, In the step of executing the prediction model unit, when a request is issued from the agent unit to change the multivariate control value based on the events occurring in the physical space, the multivariate processing value after time ΔT in the controlled object is predicted when the controlled object is controlled with the specified control value, and then output to the agent unit. The agency responds to the events occurring in the physical space based on the predicted multivariate processing values.

6. A prediction program product for causing a computer to perform the following steps: The steps of the predictive model unit are as follows: the predictive model unit learns the input-output relationship between the multivariate control value at time T and the multivariate processing value at time T+ΔT of the controlled object in the physical space; and The optimization model unit performs the following steps: it explores a multivariate control value at time T that minimizes the differences between the multivariate processing value at time T + ΔT output by the prediction model unit and the corresponding target value, and uses the explored multivariate control value to control the controlled object. in, In the step of executing the prediction model unit, when a request is issued from the agent unit to change the multivariate control value based on the events occurring in the physical space, the multivariate processing value after time ΔT in the controlled object is predicted when the controlled object is controlled with the specified control value, and then output to the agent unit. The agency is configured to respond to the events occurring in the physical space based on the predicted multivariate processing values.

Images