Management device, prediction method, and prediction program

Abstract

The present invention provides a setup that predicts changes in process values for a control target and utilizes the prediction results. A management device that has: a prediction model unit that has learned the input / output relationships between multivariate control values for a control target at time T and multivariate process values for the control target at time T+ΔT; and an optimization model unit that retrieves multivariate control values for time T that minimize the differences between the multivariate process values outputted by the prediction model unit for time T+ΔT and corresponding target values and uses the retrieved multivariate control values to control the control target. When there has been a request from an agent that manages the prediction model unit, the prediction model unit predicts multivariate process values for the control target after time ΔT assuming control of the control target using designated control values and outputs the predicted process values to the agent.

Description

[Technical field]

[0001] The present disclosure relates to a management device, a prediction method, and a prediction program. [Background technology]

[0002] In recent years, various efforts have been made in the field of circuit board manufacturing processes to realize smart factories. Specifically, progress is being made in the development of digital twin technology, in which a management device collects various data (data in the physical space) measured in the circuit board manufacturing process and reproduces the physical space in cyberspace.

[0003] Here, when reproducing the physical space, for example, a model (cyber space) reproducing a controlled object (physical space) in each substrate processing apparatus that executes a substrate manufacturing process is required to accurately predict changes in a process value of the controlled object. Also, it is required to construct a mechanism for effectively using the process value predicted by the model. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2020 / 050072 [Patent Document 2] Special Publication No. 2020-518079 [Patent Document 3] JP 2018-092511 A Summary of the Invention [Problem to be solved by the invention]

[0005] The present disclosure provides a mechanism for predicting a change in a process value of a controlled object and utilizing the prediction result. [Means for solving the problem]

[0006] A management device according to an aspect of the present disclosure has, for example, the following configuration. A prediction model unit that learns an input / output relationship between a multivariate control value of a control object at time T and a multivariate process value of the control object at time T+ΔT; an optimization model unit that searches for a multivariate control value at time T that minimizes each difference between a multivariate process value at time T+ΔT outputted from the prediction model unit and a corresponding target value, and controls the controlled object using the searched multivariate control value; The prediction model unit is an agent that manages the prediction model unit in response to events occurring in a physical space. When the agent determines that it is necessary to change the control value, When a request is received from the agent, a process value of a multivariate in the control object after a time ΔT is predicted when the control object is controlled with a specified control value, and the process value is output to the agent; A function for setting the target value The agent is The change in the multivariate process value of the controlled object after time ΔT is grasped and the event is dealt with. do. Effect of the Invention

[0007] According to the present disclosure, it is possible to provide a mechanism for predicting a change in a process value of a controlled object and utilizing the prediction result. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing an example of a system configuration of a cyber-physical system including a plurality of substrate processing apparatuses that execute a substrate manufacturing process. [Diagram 2] FIG. 2 is a diagram illustrating an example of a hardware configuration of the management apparatus. [Diagram 3] FIG. 3 is a diagram illustrating an example of a functional configuration of a cyber-physical system in the control phase. [Figure 4] FIG. 4 is an example of a flowchart showing the flow of the control process. [Diagram 5] FIG. 5 is a diagram illustrating an example of a functional configuration of a cyber-physical system in the simulation phase. [Figure 6]FIG. 6 is an example of a flowchart showing the flow of the simulation process. [Figure 7] FIG. 7 is a first diagram illustrating an example of a functional configuration of a cyber-physical system in the learning phase. [Figure 8] FIG. 8 is an example of a flowchart showing the flow of the first learning process. [Figure 9] FIG. 9 is a second diagram illustrating an example of the functional configuration of the cyber-physical system in the learning phase. [Figure 10] FIG. 10 is an example of a flowchart showing the flow of the second learning process. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, each embodiment will be described with reference to the accompanying drawings. In this specification and the drawings, components having substantially the same functional configurations are denoted by the same reference numerals, and redundant description will be omitted.

[0010] [First embodiment] <System configuration of cyber-physical system> First, a system configuration of a cyber-physical system including a plurality of substrate processing apparatuses that execute a substrate manufacturing process will be described. Fig. 1 is a diagram showing an example of a system configuration of a cyber-physical system including a plurality of substrate processing apparatuses that execute a substrate manufacturing process.

[0011] As shown in FIG. 1, the cyber-physical system 100 includes server devices 110_1 to 110_3, management devices 120_1 to 120_n, substrate processing devices 130_1 to 130_n, and an administrator terminal 140.

[0012] In the cyber-physical system 100, the server devices 110_1 to 110_3, the management devices 120_1 to 120_n, and the administrator terminal 140 are connected via a network 150 so as to be able to communicate with each other.

[0013] The server devices 110_1-110_3 are devices that control the entire cyber-physical system 100. The server devices 110_1-110_3 perform, for example, manufacturing management, data management, and device management of the substrate manufacturing process executed by each of the substrate processing devices 130_1-130_n, and management of models used in cyberspace by each of the management devices 120_1-120_n.

[0014] The management apparatuses 120_1 to 120_n are connected to the substrate processing apparatuses 130_1 to 130_n, respectively, to form a management system.

[0015] Moreover, the management devices 120_1 to 120_n have various models that reproduce each control object in the corresponding substrate processing apparatuses 130_1 to 130_n, and form a cyberspace. The management devices 120_1 to 120_n execute control processing of each control object using control values ​​calculated using the various models.

[0016] Specifically, the management devices 120_1 to 120_n calculate a control value that minimizes a difference between a process value predicted by the various models and a target value, and control the controlled object. That is, the management devices 120_1 to 120_n can predict a change in the process value and then control the controlled object.

[0017] Furthermore, the management apparatuses 120_1 to 120_n collect data in the physical space acquired in the substrate processing apparatuses 130_1 to 130_n, Grasping the state of the substrate processing apparatuses 130_1 to 130_n, Detection of events occurring in the substrate processing apparatuses 130_1 to 130_n, etc., to appropriately deal with various events that occur in the physical space.

[0018] In addition, the management devices 120_1 to 120_n execute simulation processing in cyberspace using the above-mentioned various models to deal with various events occurring in the physical space. As a result, the management devices 120_1 to 120_n can predict how a process value in a controlled object (physical space) will change after time ΔT when a control value is changed at time T, for example.

[0019] That is, the management devices 120_1 to 120_n can grasp the change in the process value in the controlled object (physical space) after time ΔT and then deal with the event.

[0020] The substrate processing apparatuses 130_1 to 130_n are apparatuses for performing a substrate manufacturing process and constitute a physical space. The substrate processing apparatuses 130_1 to 130_n include, for example, an apparatus for performing a film formation process, an apparatus for performing a lithography process, an apparatus for performing an etching process, an apparatus for performing a cleaning process, and the like.

[0021] The substrate processing apparatuses 130_1 to 130_n each have a plurality of control targets, and each control target is controlled based on a control value calculated by the management devices 120_1 to 120_n. Furthermore, the substrate processing apparatuses 130_1 to 130_n transmit data in the physical space acquired during the execution of the substrate manufacturing process to the management devices 120_1 to 120_n.

[0022] The administrator terminal 140 is a terminal operated by an administrator who manages the cyber-physical system 100. The administrator terminal 140 generates various models to be used in, for example, the management devices 120_1 to 120_n. Specifically, the administrator terminal 140 first acquires actual measured process values ​​and actual measured control values ​​of a control target from data in the physical space collected by the management devices 120_1 to 120_n, and generates learning data. Then, the administrator terminal 140 performs a learning process using the generated learning data to generate various models.

[0023] 1 illustrates a case in which the management devices 120_1 to 120_n and the substrate processing devices 130_1 to 130_n are configured as separate entities in the cyber-physical system 100. However, the management devices 120_1 to 120_n and the substrate processing devices 130_1 to 130_n may be configured as an integrated entity.

[0024] <Hardware configuration of management device and administrator terminal> Next, a description will be given of the hardware configurations of the management devices 120_1 to 120_n and the administrator terminal 140. Since the management devices 120_1 to 120_n and the administrator terminal 140 all have the same hardware configuration, the management devices 120_1 to 120_n will be collectively described here with reference to Fig. 2. Fig. 2 is a diagram showing an example of the hardware configuration of the management device.

[0025] 2, the management devices 120_1 to 120_n each include a processor 201, a memory 202, an auxiliary storage device 203, an I / F (Interface) device 204, a communication device 205, and a drive device 206. The hardware components of the management devices 120_1 to 120_n are connected to each other via a bus 207.

[0026] The processor 201 has various arithmetic devices such as a central processing unit (CPU), a graphics processing unit (GPU), etc. The processor 201 reads various programs (for example, a prediction program described later) onto the memory 202 and executes them.

[0027] The memory 202 has a main storage device such as a Read Only Memory (ROM) and a Random Access Memory (RAM). The processor 201 and the memory 202 form a so-called computer, and the processor 201 executes various programs read onto the memory 202, causing the computer to realize various functions.

[0028] The auxiliary storage device 203 stores various programs and various data used when the processor 201 executes the various programs.

[0029] The I / F apparatus 204 is a connection device that connects the substrate processing apparatuses 130_1 to 130_n, which are examples of external apparatuses, with the management apparatuses 120_1 to 120_n.

[0030] The communication apparatus 205 is a communication device for communicating with other apparatuses (in this embodiment, the server apparatuses 110_1 to 110_3, other management apparatuses, the administrator terminal 140, etc.) via the network 150.

[0031] The drive device 206 is a device for setting the recording medium 210. The recording medium 210 here includes media that record information optically, electrically, or magnetically, such as a CD-ROM, a flexible disk, a magneto-optical disk, etc. The recording medium 210 may also include semiconductor memories that record information electrically, such as ROM, flash memory, etc.

[0032] The various programs to be installed in the auxiliary storage device 203 are installed, for example, by setting the distributed recording medium 210 in the drive device 206 and reading out the various programs recorded in the recording medium 210 by the drive device 206. Alternatively, the various programs to be installed in the auxiliary storage device 203 may be installed by being downloaded from a network via the communication device 205.

[0033] <Functional configuration and various processing of cyber-physical systems> Next, the functional configuration of the cyber-physical system 100 and the details of various processes executed in the cyber-physical system 100 will be described. As described above, the cyber-physical system 100 A control phase in which the management devices 120_1 to 120_n perform control processing of the control object using various models that reproduce the control object; A simulation phase in which the management devices 120_1 to 120_n grasp the state of the substrate processing apparatus and, when an event is detected, perform a simulation process using various models that reproduce the control target in order to deal with the event; A learning phase in which the administrator terminal 140 generates learning data using actual measured process values, actual measured control values, etc. of the controlled object, and performs learning processing using the generated learning data to generate various models; In each of the phases, different functions operate. Therefore, in the following, the functional configuration of the cyber-physical system 100 and the details of various processes will be described for each phase.

[0034] (1) Control Phase (1-1) Functional configuration First, a functional configuration of the cyber-physical system 100 in the control phase will be described. Fig. 3 is a diagram showing an example of the functional configuration of the cyber-physical system in the control phase.

[0035] As described above, the prediction programs are installed in the management devices 120_1 to 120_n, respectively. Then, the management devices 120_1 to 120_n execute the respective prediction programs, whereby an agent unit 320 and a model prediction control unit 330 are realized for each management device in the cyberspace 310 in the control phase.

[0036] In the control phase, the agent unit 320 sets multivariate target values ​​for the model predictive control unit 330. The target values ​​referred to here refer to target values ​​used when the model predictive control unit 330 executes control processing of a control target (physical space 340) in the substrate processing apparatuses 130_1 to 130_n. Note that the example of Fig. 3 indicates that the control target (physical space 340) has sensors 341_1 to 341_n and actuators 342_1 to 342_m.

[0037] 3, in a controlled object having sensors 341_1 to 341_n and actuators 342_1 to 342_m, the operations of a plurality of (m) actuators are controlled so that actual process values ​​of the (n) multivariates coincide with target values ​​of the corresponding (n) multivariates. For this reason, the agent unit 320 sets target values ​​of the (n) multivariates for the model predictive control unit 330.

[0038] The model prediction control unit 330 includes a prediction model unit 331 , an objective function unit 332 , an optimization model unit 333 , and a verification unit 334 .

[0039] The prediction model unit 331 models the behavior of the controlled object (physical space 340) and is a model that learns the input / output relationship between the multivariate actual control value of the controlled object at time T and the multivariate actual process value of the controlled object at time T+ΔT.

[0040] The objective function unit 332 calculates each difference value between the predicted process value of the multivariate at time T+ΔT output from the prediction model unit 331 and the target value of the multivariate set by the agent unit 320, and notifies the optimization model unit 333.

[0041] The optimization model unit 333 searches for multivariate control values ​​at time T that minimize each difference value notified by the objective function unit 332. The optimization model unit 333 also inputs the searched multivariate control values ​​to the prediction model unit 331, and again acquires each difference value between the predicted process values ​​of the multivariates at time T+ΔT output by the prediction model unit 331 and the target values ​​of the multivariates. The optimization model unit 333 repeats these processes to minimize each difference value, and calculates the optimal multivariate control value at time T.

[0042] Furthermore, the optimization model unit 333 transmits the optimal multivariate control values ​​at time T to the actuators 342_1 to 342_m to be controlled (physical space 340) (actually, the control values ​​are transmitted to the corresponding substrate processing apparatuses, and then notified to each actuator).

[0043] This allows the optimization model unit 333 to predict the predicted multivariate process values ​​and then control the controlled object (physical space 340).

[0044] The verification unit 334 acquires optimal multivariate control values ​​from the optimization model unit 333. Furthermore, the verification unit 334 acquires actual measured process values ​​of multivariates from the physical space 340 in response to the optimal multivariate control values ​​being transmitted to the actuators 342_1 to 342_2 of the control target (physical space 340).

[0045] Furthermore, the verification unit 334 determines whether the control values ​​are appropriate based on the optimal multivariate control values ​​and the acquired multivariate actual process values, verifies the prediction accuracy of the prediction model unit 331, and adjusts the model parameters of the prediction model unit 331 as necessary. This enables the verification unit 334 to match the behavior of the prediction model unit 331 with the behavior of the controlled object in the physical space 340 (i.e., to match the predicted process values ​​with the actual process values).

[0046] Meanwhile, a physical space 340 constituted by the substrate processing apparatuses 130_1 to 130_n includes a plurality of control objects. The example of Fig. 3 shows one control object having sensors 341_1 to 341_n and actuators 342_1 to 342_m.

[0047] The sensors 341_1 to 341_n measure process values ​​such as flow rate information, pressure information, etc. The flow rate information, pressure information, etc. measured by the sensors 341_1 to 341_n are provided to the cyberspace 310 as actual measured process values.

[0048] The actuators 342_1 to 342_m operate based on instructions from the cyberspace 310. The example of Fig. 3 shows that each of the actuators 342_1 to 342_m operates based on an optimal multivariate control value calculated by the model predictive control unit 330.

[0049] (1-2) Control processing Next, a description will be given of the flow of control processing executed by the cyber-physical system 100 in the control phase. Fig. 4 is an example of a flowchart showing the flow of the control processing.

[0050] In step S401, the model prediction control unit 330 determines whether or not new multivariate target values ​​have been set by the agent unit 320.

[0051] In step S401, if it is determined that new multivariate target values ​​have not been set (NO in step S401), the process proceeds to step S404.

[0052] On the other hand, if it is determined in step S401 that new multivariate target values ​​have been set (YES in step S401), the process proceeds to step S402.

[0053] In step S402, the model predictive control unit 330 searches for optimal multivariate control values ​​at time T so that the predictive model unit 331 outputs multivariate predicted process values ​​at time T+ΔT that minimize each difference value with the new multivariate target value.

[0054] In step S403, the model predictive control unit 330 transmits the optimal multivariate control values ​​at time T obtained by the search to the actuators 342_1 to 342_2 of the control target (physical space 340).

[0055] In step S404, in response to the transmission of the optimal multivariate control values ​​at time T, the model predictive control unit 330 acquires the actual measured process values ​​of the multivariate at time T+ΔT provided from the physical space 340.

[0056] In step S405, the model predictive control unit 330 determines whether the control value at time T is appropriate based on the optimal multivariate control value at time T and the acquired actual multivariate process value at time T+ΔT, and verifies the predictive accuracy of the predictive model unit 331.

[0057] In step S406, the model prediction control unit 330 determines whether or not the model parameters of the prediction model unit 331 need to be adjusted.

[0058] In step S406, if it is determined that adjustment of the model parameters is not necessary (NO in step S406), the process proceeds to step S408.

[0059] On the other hand, if it is determined in step S406 that adjustment of the model parameters is necessary (YES in step S406), the process proceeds to step S407.

[0060] In step S407, the model prediction control unit 330 adjusts the model parameters of the prediction model unit 331.

[0061] In step S408, the model prediction control unit 330 determines whether or not to end the control process.

[0062] If it is determined in step S408 that the control process should not be ended (NO in step S408), the process returns to step S401.

[0063] On the other hand, if it is determined in step S409 that the control process should be ended (YES in step S408), the control process is ended.

[0064] (2) Simulation Phase (2-1) Functional configuration Next, a functional configuration of the cyber-physical system 100 in the simulation phase will be described. Fig. 5 is a diagram showing an example of the functional configuration of the cyber-physical system in the simulation phase.

[0065] The difference from the functional configuration of the cyber-physical system 100 in the control phase described with reference to FIG.

[0066] In the simulation phase, the agent unit 320 collects data in the physical space (in addition to actual measured process values, any data such as particle information, maintenance information, device configuration information, and operation information) to Grasping the state of the substrate processing apparatuses 130_1 to 130_n, Detection of events occurring in the substrate processing apparatuses 130_1 to 130_n, and so on to appropriately deal with various events occurring in the physical space 340. Note that the data in the physical space shown in FIG. 5 is an example, and may include information obtained by measuring the processing state of the substrate instead of (or in addition to) the particle information. In the example of FIG. 5, the device for measuring the particle information is an external measuring device, but the device for measuring the particle information is not limited to the external measuring device and may be an internal measuring device installed in the substrate processing apparatuses 130_1-130_n. For example, the device may be a device for measuring the internal state of the substrate processing apparatuses 130_1-130_n through a window provided in the wall of the substrate processing apparatuses 130_1-130_n. In addition, the device for measuring the particle information may be a device for observing the state on the substrate to be processed, or a device for acquiring the state of the processing space in which the substrate to be processed is processed.

[0067] Here, it is assumed that the agent unit 320 determines that it is necessary to change the control value to be sent to the control target (physical space 340) in order to deal with the event. In this case, the agent unit 320 makes a request to the model prediction control unit 330, "When the control value to be sent to the controlled object is changed, how does the actual measured process value in the controlled object (physical space 340) change after time ΔT?" is predicted by executing a simulation process.

[0068] 5 shows a state in which the agent unit 320 makes a request to the model predictive control unit 330 to predict a predicted process value at time T+ΔT when the multivariate control values ​​are changed at time T. Specifically, the agent unit 320 inputs the changed multivariate control values ​​at time T to the predictive model unit 331.

[0069] This allows the agent unit 320 to obtain the predicted process value at time T+ΔT from the model predictive control unit 330. As a result, the agent unit 320 can handle the event after grasping the change in the process value in the controlled object (physical space 340) after time ΔT when the control value is changed at time T.

[0070] (2-2) Simulation processing Next, a description will be given of the flow of the simulation process executed by the cyber-physical system 100 in the simulation phase. Fig. 6 is an example of a flowchart showing the flow of the simulation process.

[0071] In step S601, the model predictive control unit 330 determines whether or not the changed multivariate control values ​​designated by the agent unit 320 have been input.

[0072] In step S601, if it is determined that the specified changed multivariate control values ​​have not been input (NO in step S601), the process proceeds to step S604.

[0073] On the other hand, if it is determined in step S601 that the specified changed multivariate control values ​​have been input (YES in step S601), the process proceeds to step S602.

[0074] In step S602, the model prediction control unit 330 predicts a predicted process value after time ΔT.

[0075] In step S603, the model prediction control unit 330 outputs the predicted process value after time ΔT to the agent unit 320.

[0076] In step S604, the model prediction control unit 330 determines whether or not to end the simulation process. If it is determined in step S604 that the simulation process is not to end (NO in step S604), the process returns to step S601.

[0077] On the other hand, if it is determined in step S604 that the simulation process is to be ended (YES in step S604), the simulation process is ended.

[0078] (3) Learning phase (predictive model section) (3-1) Functional configuration Next, a functional configuration of the cyber-physical system 100 in the learning phase, in which learning processing is performed on the prediction model unit, will be described. Fig. 7 is a first diagram showing an example of the functional configuration of the cyber-physical system in the learning phase.

[0079] A first learning program is installed in the administrator terminal 140, and the administrator terminal 140 functions as a first learning unit 720 by executing the first learning program.

[0080] The first learning unit 720 accumulates the first learning data in the first learning data accumulation unit 723. Specifically, the first learning unit 720 acquires the actual measured control values ​​of the multivariates at time T and the actual measured process values ​​of the multivariates at time T+ΔT provided from the physical space 340 via the management devices 120_1 to 120_n (not shown in FIG. 7).

[0081] The example of FIG. 7 shows that in the physical space 340, the substrate processing apparatuses 130_1 to 130_n each have control objects (sensors 341_1 to 341_n and actuators 342_1 to 342_m) and a controller 730 that controls the control objects.

[0082] The controller 730 controls the actuators 342_1 to 342_m so that actual process values ​​of the multivariates measured by the sensors 341_1 to 341_n approach target values ​​of the multivariates set in the controller 730.

[0083] The first learning unit 720 generates first learning data by correspondingly acquiring the multivariate actual control value at time T and the multivariate actual process value at time T+ΔT, and stores the first learning data in the first learning data storage unit 723.

[0084] The first learning unit 720 has a prediction model unit 721 and a comparison / change unit 722 , and performs a learning process on the prediction model unit 721 .

[0085] Specifically, the first learning unit 720 reads out the first learning data from the first learning data storage unit 723, and inputs the multivariate actual control value at time T to the prediction model unit 721. As a result, the prediction model unit 721 outputs the multivariate predicted process value at time T+ΔT.

[0086] Further, the first learning unit 720 inputs the multivariate actual process values ​​at time T+ΔT as correct answer data to the comparison / change unit 722. As a result, the comparison / change unit 722 calculates each error between the multivariate predicted process values ​​at time T+ΔT and the multivariate actual process values ​​at time T+ΔT, and updates the model parameters of the prediction model unit 721 by backpropagating each calculated error. As a result, the first learning unit 720 can learn the input / output relationship between the multivariate actual control values ​​of the controlled object at time T and the multivariate actual process values ​​of the controlled object at time T+ΔT, and can generate the learned prediction model unit 331.

[0087] (3-2) First learning process Next, a description will be given of the flow of the first learning process executed by the cyber-physical system 100 in the learning phase. Fig. 8 is an example of a flowchart showing the flow of the first learning process.

[0088] In step S801, in the physical space 340, the controllers 730 in the substrate processing apparatuses 130_1 to 130_n start control processing of the control targets.

[0089] In step S802, the first learning unit 720 of the administrator terminal 140 acquires the actual measured control values ​​of the multivariates at time T provided from the physical space 340.

[0090] In step S803, the first learning unit 720 of the administrator terminal 140 acquires the multivariate actual measured process values ​​at time T+ΔT provided from the physical space 340.

[0091] In step S804, the first learning section 720 of the manager terminal 140 generates first learning data by associating the acquired actual measured control values ​​of multivariates at time T with the acquired actual measured process values ​​of multivariates at time T+ΔT.

[0092] In step S805, the first learning unit 720 of the administrator terminal 140 performs a learning process on the prediction model unit 721 using the generated first learning data.

[0093] In step S806, the first learning unit 720 of the administrator terminal 140 determines whether or not the learning process has been completed for the prediction model unit 721. If it is determined in step S806 that the learning process has not been completed (NO in step S806), the process returns to step S802.

[0094] On the other hand, if it is determined in step S806 that the learning process is completed (YES in step S806), the process proceeds to step S807.

[0095] In step S807, the first learning unit 720 of the administrator terminal 140 outputs the learned prediction model unit.

[0096] (4) Learning phase (optimization model section) (4-1) Functional configuration Next, a functional configuration of the cyber-physical system 100 in the learning phase, in which learning processing is performed on the optimization model unit, will be described. Fig. 9 is a second diagram showing an example of the functional configuration of the cyber-physical system in the learning phase.

[0097] A second learning program is installed in the administrator terminal 140, and the administrator terminal 140 functions as a second learning unit 920 by executing the second learning program.

[0098] The second learning unit 920 accumulates the second learning data in the second learning data accumulation unit 924. Specifically, the second learning unit 920 acquires each difference value at time T+ΔT calculated by the difference calculation unit 923 and the actual measured control value of the multivariate at time T provided from the physical space 340. The difference calculation unit 923 calculates each difference value by acquiring the actual measured process value and the target value at time T+ΔT provided from the physical space 340 via the management devices 120_1 to 120_n (not shown in FIG. 9).

[0099] The example of FIG. 9 shows that in the physical space 340, the substrate processing apparatuses 130_1 to 130_n each have control objects (sensors 341_1 to 341_n and actuators 342_1 to 342_m) and a controller 730 that controls the control objects.

[0100] The controller 730 controls the actuators 342_1 to 342_m so that actual process values ​​of the multivariates measured by the sensors 341_1 to 341_n approach target values ​​of the multivariates set in the controller 730.

[0101] The difference calculation unit 923 acquires the multivariate target values ​​at this time and the multivariate actual process values ​​at time T+ΔT, and calculates each difference value at time T+ΔT. The second learning unit 920 acquires the multivariate actual control values ​​at time T at this time, and generates second learning data by correlating them with each difference value at time T+ΔT calculated by the difference calculation unit 923. The second learning data accumulation unit 924 accumulates the second learning data.

[0102] The second learning unit 920 has an optimization model unit 921 and a comparison / change unit 922 , and performs a learning process on the optimization model unit 921 .

[0103] Specifically, the second learning unit 920 reads out the second learning data from the second learning data storage unit 924, and inputs each difference value at time T+ΔT to the optimization model unit 921. As a result, the optimization model unit 921 outputs the multivariate control value at time T.

[0104] Further, the second learning unit 920 inputs the actual measurement control values ​​of the multivariates at time T as correct answer data to the comparison / change unit 922. As a result, the comparison / change unit 922 calculates each error between the control values ​​of the multivariates at time T and the actual measurement control values ​​of the multivariates at time T, and updates the model parameters of the optimization model unit 921 by backpropagating each calculated error. As a result, the second learning unit 920 can learn the input / output relationship between the difference value at time T+ΔT and the control value at time T, and can generate the trained optimization model unit 333.

[0105] (4-2) Second learning process Next, a description will be given of the flow of the second learning process executed by the cyber-physical system 100 in the learning phase. Fig. 10 is an example of a flowchart showing the flow of the second learning process.

[0106] In step S1001, in the physical space 340, the controllers 730 in the substrate processing apparatuses 130_1 to 130_n start control processing of the control targets.

[0107] In step S1002, the second learning unit 920 of the administrator terminal 140 acquires target values ​​of multivariates provided from the physical space 340. In addition, the second learning unit 920 of the administrator terminal 140 acquires actual measured process values ​​of multivariates at time T+ΔT provided from the physical space 340.

[0108] In step S1003, the second learning unit 920 of the manager terminal 140 calculates each difference value between the target value of the multivariate and the actual measured process value of the multivariate at time T+ΔT.

[0109] In step S1004, the second learning unit 920 of the administrator terminal 140 acquires the actual measured control values ​​of the multivariates at time T provided from the physical space 340.

[0110] In step S1005, the second learning unit 920 of the administrator terminal 140 associates each calculated difference value at time T+ΔT with the acquired actual measured control value of the multivariate at time T to generate second learning data.

[0111] In step S1006, the second learning unit 920 of the administrator terminal 140 performs a learning process on the optimization model unit 921 using the generated second learning data.

[0112] In step S1007, the second learning unit 920 of the administrator terminal 140 determines whether or not the learning process has been completed for the optimization model unit 921. If it is determined in step S1007 that the learning process has not been completed (NO in step S1007), the process returns to step S1002.

[0113] On the other hand, if it is determined in step S1007 that the learning process is completed (YES in step S1007), the process proceeds to step S1008.

[0114] In step S1008, the second learning unit 920 of the administrator terminal 140 outputs the trained optimization model unit.

[0115] <Summary> As is clear from the above description, the management device according to the first embodiment The system has a prediction model that learns the input / output relationship between the multivariate actual control values ​​of the controlled object at time T and the multivariate actual process values ​​of the controlled object at time T+ΔT. The system has an optimization model section that searches for a multivariate control value at time T that minimizes each difference value between the multivariate process value at time T+ΔT output from the prediction model section and the corresponding target value, and controls the controlled object using the searched optimal multivariate control value at time T. When there is a request from the agent unit, the prediction model unit predicts the multivariate process values ​​of the controlled object after time ΔT when the controlled object is controlled with the specified control value at time T, and outputs the predictions to the agent unit.

[0116] In this way, the management device according to the first embodiment has a prediction model unit, and thus can accurately predict changes in the process value of the controlled object. Also, the management device according to the first embodiment is configured to calculate an optimal control value based on the predicted process value, and thus can use the predicted process value in the control process of the controlled object. Furthermore, the management device according to the first embodiment is configured to predict a predicted process value in response to a request from an agent unit and output the predicted process value to the agent unit, and thus can use the predicted process value to deal with an event occurring in a physical space.

[0117] That is, according to the first embodiment, it is possible to provide a mechanism for predicting a change in a process value of a controlled object and utilizing the prediction result.

[0118] [Second embodiment] In the above-described first embodiment, the management devices 120_1 to 120_n are configured as separate management devices, but the management devices 120_1 to 120_n may be configured as an integrated device. In this case, the n management devices may be configured to operate virtually (i.e., as virtual machines) on the integrated device.

[0119] In the above first embodiment, the management devices 120_1 to 120_n corresponding to the substrate processing apparatuses 130_1 to 130_n have been described as executing the prediction program independently. However, a management device (e.g., the management device 120_1) corresponding to one substrate processing apparatus (e.g., the substrate processing apparatus 130_1) may be configured with, for example, a plurality of computers. The prediction program may be installed in each of the plurality of computers, so that the prediction program is executed in a distributed computing format.

[0120] In the above first embodiment, a method of downloading and installing the prediction program via a network has been mentioned as an example of a method of installing the prediction program in the auxiliary storage device 203 of the management devices 120_1 to 120_n. At this time, no particular reference has been made to a download source, but in the case of installing by such a method, the download source may be, for example, a server device that stores the prediction program in an accessible manner. Furthermore, the server device may be a device on a cloud that accepts access from each of the management devices 120_1 to 120_n via a network and downloads the prediction program on the condition of payment. In other words, the server device may be a device on a cloud that provides a service of providing the prediction program.

[0121] In the above-described first embodiment, the cyberspace is formed in the management system including the plurality of management devices 120_1 to 120_n, but the cyberspace may be formed in a system other than the management system. For example, the cyberspace may be formed in the server devices 110_1 to 110_3.

[0122] In the above first embodiment, the first learning process and the second learning process have been described as being executed in the administrator terminal 140. However, the first learning process and the second learning process may be executed in the management devices 120_1 to 120_n or the server devices 110_1 to 110_3.

[0123] In other words, the functions operated in each of the control phase, the simulation phase, and the learning phase may be realized in separate devices or in a single device.

[0124] Furthermore, in the above first embodiment, the control process and the simulation process are described as being executed in separate phases, but they may be executed in parallel in the same phase.

[0125] In the above-described first embodiment, no reference is made to the output destination of the prediction model unit and optimization model unit that have completed the learning process, but the prediction model unit and optimization model unit that have completed the learning process may be output to the management devices 120_1 to 120_n, or may be registered in the server devices 110_1 to 110_n so as to be downloadable.

[0126] In addition, in the first embodiment, details of the model were not mentioned. However, the model used in the first embodiment may be, for example, a machine learning model including deep learning. For example, ·RNN (Recurrent Neural Network), ·LSTM(Long Short-Term Memory), ·CNN(Convolutional Neural Network), ·R-CNN (Region based Convolutional Neural Network), ·YOLO (You Only Look Once), ·SSD (Single Shot MultiBox Detector), ·GAN (Generative Adversarial Network), ·SVM(Support Vector Machine), Decision trees, Random Forest It may be any of the above.

[0127] Alternatively, the model may be a model using a genetic algorithm such as GA (Genetic Algorism) or GP (Genetic Programming), or a model trained by reinforcement learning.

[0128] Alternatively, the model used in the first embodiment may be a model obtained by general statistical analysis other than deep learning, such as Principal Component Regression (PCR), Partial Least Square (PLS), LASSO, ridge regression, linear polynomial, autoregressive model, moving average model, autoregressive moving average model, ARX model, etc. Alternatively, the above models may be used in combination.

[0129] The present invention is not limited to the configurations shown in the above embodiments, combinations with other elements, etc. These points can be changed without departing from the spirit of the present invention, and can be appropriately determined according to the application form.

[0130] This application claims priority based on Japanese Patent Application No. 2020-218532, filed on December 28, 2020, the entire contents of which are incorporated herein by reference. [Explanation of symbols]

[0131] 100: Cyber-Physical Systems 120_1~120_n: Management device 130_1 to 130_n: Substrate processing equipment 140: Administrator terminal 310: Cyberspace 320: Agent Department 330: Model prediction control unit 340: Physical Space 720: First Learning Section 920: Second Learning Section

Claims

1. a prediction model unit that learns an input / output relationship between a multivariate control value of a control object at time T and a multivariate process value of the control object at time T+ΔT; an optimization model unit that searches for a multivariate control value at time T that minimizes each difference between a multivariate process value at time T+ΔT outputted from the prediction model unit and a corresponding target value, and controls the controlled object using the searched multivariate control value; the prediction model unit, upon receipt of a request from an agent managing the prediction model unit that a control value needs to be changed in response to an event occurring in a physical space, predicts multivariate process values ​​of the control object after a time ΔT when the control object is controlled with a designated control value, and outputs the predicted process values ​​to the agent; The agent, which has a function of setting the target value, grasps a change in a multivariate process value after time ΔT in the controlled object and deals with the event.

2. The management device according to claim 1, further comprising a first learning unit that performs a learning process on the prediction model unit so that the output of a multivariate when an actual control value of a multivariate at time T is input approaches the actual process value of the multivariate in the controlled object at time T + ΔT.

3. The management device according to claim 1, further comprising a second learning unit that performs a learning process on the optimization model unit so that a multivariate control value at time T that is output by inputting each difference between a multivariate actual process value at time T+ΔT in the controlled object and a corresponding target value approaches the multivariate actual control value at time T when the controlled object is controlled.

4. Executing a prediction model unit that has learned an input / output relationship between a multivariate control value at time T in a controlled object and a multivariate process value at time T+ΔT in the controlled object; and executing an optimization model unit that searches for a multivariate control value at time T that minimizes each difference between a multivariate process value at time T+ΔT outputted from the prediction model unit and a corresponding target value, and controls the controlled object using the searched multivariate control value. The step of executing the prediction model unit includes, when an agent managing the prediction model unit determines that it is necessary to change a control value in response to an event occurring in a physical space, predicting a multivariate process value of the control object after a time ΔT when the control object is controlled with a specified control value upon receiving a request from the agent, and outputting the predicted process value to the agent; The step of executing an agent having a function of setting the target value grasps a change in a multivariate process value after a time ΔT in the controlled object and deals with the event.

5. Executing a prediction model unit that has learned an input / output relationship between a multivariate control value at time T in a controlled object and a multivariate process value at time T+ΔT in the controlled object; a step of searching for a multivariate control value at time T that minimizes each difference between a multivariate process value at time T+ΔT outputted from the prediction model unit and a corresponding target value, and executing an optimization model unit that controls the controlled object using the searched multivariate control value, The step of executing the prediction model unit includes, when an agent managing the prediction model unit determines that it is necessary to change a control value in response to an event occurring in a physical space, predicting a multivariate process value of the control object after a time ΔT when the control object is controlled with a specified control value upon receiving a request from the agent, and outputting the predicted process value to the agent; The step of executing an agent having a function of setting the target value is a prediction program that grasps changes in multivariate process values ​​of the controlled object after time ΔT and deals with the event.

Images