Control of a technical system by means of a data-based adjustment model

Abstract

The present invention relates to a method and a device for configuring a regulator, and a method and a regulator for controlling a technical system by means of a data-based regulation model, in particular a model based on reinforcement learning. The data-based regulation model (RL) is configured according to a model predictive regulation model (MPC). Here, the configuration parameters of the data-based regulation model are set by mapping the model predictive regulation model to the data-based regulation model such that the data-based regulation model reproduces the output data of the model predictive regulation model based on the read-in state data of the technical system, and the regulation parameters optimized in such a configured manner are determined. Thus, computationally intensive training methods for configuring the data-based regulation model can be avoided.

Description

Field of the Invention

[0001] The present invention relates to a computer-implemented method and apparatus for configuring a regulator to control a technical system, a computer-implemented method and a regulator for controlling a technical system by means of such a configured data-based regulation model, and a computer program product. Background Art

[0002] The efficient operation of technical systems, such as process plants in the chemical industry or power plants, typically takes place by means of parallel operation regulation. Regulation is usually carried out based on predetermined regulation parameters or manipulated variables starting from the detected system state, where the optimal regulation parameters or manipulated variables for this purpose can be determined by means of model predictive regulation methods. However, this accompanying regulation is difficult or even impossible due to the high computational effort, especially for complex systems.

[0003] US2020 / 183338A1 discloses a control device for controlling an object based on reinforcement learning. US2019 / 041811A1 discloses a combined prediction model for a building management system consisting of an artificial neural network, a distance metric-based prediction, and a regression model. US2019 / 031204A1 discloses a method for implementing optimized control of a complex dynamic system using machine learning-based scenario control heuristics. Summary of the Invention

[0004] Therefore, the object of the present invention is to create a possibility for simplifying the accompanying operation control of a technical system, especially a complex technical system.

[0005] This object is solved by the measures described in the independent claims. Advantageous refinements of the invention are shown in the dependent claims.

[0006] According to a first aspect, the present invention relates to a computer-implemented method for configuring a regulator to control a technical system,

[0007] a) reading in a model predictive regulation model for the regulator, wherein the model predictive regulation model is configured to output optimized regulation parameters for controlling the technical system as output data based on the simulated and / or measured state data of the technical system,

[0008] b) reading in a data-based regulation model,

[0009] c) setting configuration parameters of the data-based regulation model according to the model predictive regulation model such that the data-based regulation model reproduces the output data of the model predictive regulation model based on the read-in state data of the technical system, and determining optimized regulation parameters in such a configured manner, and

[0010] d) Output the data-based adjustment model configured in this way to the adjuster for controlling the technical system.

[0011] The data-based adjustment model configured in this way can in particular be output to the adjuster for controlling the technical system. Thus, in particular, the device according to the invention can be coupled to such an adjuster.

[0012] According to another aspect, the invention relates to a device for configuring an adjuster to control a technical system, the device comprising

[0013] - an interface which is set up to read in a model predictive adjustment model for the adjuster, wherein the model predictive adjustment model is set up to output optimized adjustment parameters for controlling the technical system as output data based on the simulated and / or measured state data of the technical system, and to read in a data-based adjustment model,

[0014] - a configurator which is set up to set configuration parameters of the data-based adjustment model according to the model predictive adjustment model such that the data-based adjustment model reproduces the output data of the model predictive adjustment model based on the read-in state data of the technical system, and to determine optimized adjustment parameters in such a configured manner,

[0015] - an output module which is set up to output the configured data-based adjustment model to the adjuster for controlling the technical system.

[0016] The method according to the invention can in particular be implemented in a computer-aided manner. As long as not stated otherwise in the following description, the terms "implement", "calculate", "computer-aided", "calculate", "determine", "generate", "configure", "reconfigure", etc. preferably relate to actions and / or processes and / or processing steps of changing and / or generating data and / or converting data into other data, where the data can in particular be represented as physical parameters or can exist as physical parameters, for example as electrical pulses. In particular, the term "computer" should be interpreted as broadly as possible so as to in particular cover all electronic devices having data processing characteristics. Thus, a computer can be, for example, a personal computer, a server, a programmable logic controller (PLC), a handheld computer system, a palmtop device, a mobile radio device and other communication devices that can process data in a computer-aided manner, a processor and other electronic devices for data processing. The device according to the invention can be designed in hardware and / or software. If the device is designed in hardware, the device can in particular include at least one processor. In connection with the invention, a processor can be understood, for example, as a machine or an electronic circuit. A processor can in particular be a main processor (English: Central Processing Unit (CPU)), a microprocessor or a microcontroller, for example an application-specific integrated circuit or a digital signal processor, possibly in combination with a storage unit for storing program instructions, etc. A processor can also be understood as a virtualized processor, a virtual machine or a soft CPU.

[0017] In particular with regard to data and / or information, "provide" in connection with the invention can be understood, for example, as being provided in a computer-aided manner. For example, it is provided via an interface, such as a network interface or an interface to a storage unit. For example, corresponding data and / or information can be transmitted and / or sent and / or called and / or received via such an interface when providing.

[0018] A "model predictive control model" can in particular be understood as a time-discrete dynamic model of the process to be controlled of a technical system. A model predictive control model can in particular be established based on the method of model predictive control (English: Model Predictive Control; MPC). A model predictive control model / model predictive control can calculate the future process behavior based on an input signal or a control parameter. Thus, in particular, an optimized input signal or an optimized control parameter can be determined in order to achieve an optimized output signal.

[0019] Thus, in connection with the invention, a "control parameter" can in particular be understood as an input signal or a manipulated variable for controlling / regulating a technical system. The state of the technical system is changed according to the control parameter.

[0020] The model predictive control model is established based on the simulated and / or measured state data of the technical system. In particular, the model predictive control model can be established based on the provided computer-aided simulation model of the technical system, so as to output optimized control parameters for controlling the technical system as output data based on the state data generated by the technical system with the aid of the computer-aided simulation model.

[0021] The technical system can in particular be understood as a facility, a process plant, a power plant, a device, a machine (such as a turbine), a robot, a vehicle, an autonomous vehicle or an infrastructure network (such as for water, gas, electric current or oil).

[0022] The "data-based control model" can in particular be a computer-aided model which is established to output control parameters for controlling the technical system based on data (here state data). The data-based control model is preferably based on machine learning methods or artificial intelligence. For example, the data-based control model is an artificial neural network.

[0023] The advantage of the present invention is that a data-based control model, in particular a model based on reinforcement learning, is established in a simple manner for outputting optimized control parameters. Here, since configuration parameters, such as the weights of an artificial neural network, are set according to the model predictive control model such that the output of the model predictive control model is reproduced by the data-based control model, the data-based control model is not trained or is trained with less computational effort by means of training data. In addition, since the data-based control model is trained not only limited to the training data space, this method enables a larger state space or behavior space to be covered. In other words, when mapping the model predictive control model to the data-based control model, a larger behavior space, such as also extreme scenarios, is considered, which typically occur rarely in the measured data, but in which the control should also function correctly. This is achieved in particular by covering the valid range in the case of model predictive control if the physics-based equations are valid.

[0024] Therefore, compared with the common training of data-based models (such as reinforcement learning models), the present method requires less computational effort. Therefore, information from model predictive control is used and extracted in order to configure the data-based control model. In particular, the initial configuration of the data-based control model can be determined using this method.

[0025] In an advantageous embodiment of the computer-implemented method, the configuration parameters of the data-based regulation model can be set by mapping the model predictive regulation model to the data-based regulation model.

[0026] The model predictive regulation model can exist, for example, as a state space model or in state space representation. Here, all relationships of the input, output, and state variables can be represented in the form of matrices and vectors. Thereby, a mapping of the model predictive regulation model to the data-based regulation model, in particular a geometric mapping, can be achieved. Thus, the configuration parameters of the data-based regulation model are set by the mapping. For example, the model representation of the data-based regulation model can be adapted from the information of the model predictive regulation, such as the state and the manipulated variable, such that the output of the data-based regulation model reproduces the output of the model predictive regulation model. The mapping matrix or the mapping function can be determined, for example, based on the output to be obtained of the model predictive regulation model and the data-based regulation model. In other words, the configuration parameters of the data-based regulation model are preferably set such that, according to the input value, the output value of the model predictive regulation and the output value of the data-based regulation model are approximated.

[0027] In an embodiment of the computer-implemented method, the data-based regulation model can be established based on machine learning methods.

[0028] In an embodiment of the computer-implemented method, the data-based regulation model can be established as an agent of the reinforcement learning method (English: Reinforcement Learning).

[0029] Thus, for example, the model predictive regulation model can be mapped to the agent of the reinforcement learning method, and the configuration parameters are set accordingly in order to reproduce the output of the model predictive regulation model.

[0030] In an embodiment of the computer-implemented method, the configuration parameters of the data-based regulation model are adapted according to other state data and by means of the reinforcement learning method for determining other optimized regulation parameters.

[0031] In particular, the configured data-based regulation model can be regarded as preconfigured or trained, that is, the configuration parameters are preset according to the model predictive regulation model. The accuracy of the data-based regulation model can be improved by means of other training data, that is, other state data of the technical system.

[0032] In an embodiment of the computer-implemented method, the model predictive regulation model can be established based on the method of model predictive regulation.

[0033] According to another aspect, the invention relates to a computer-implemented method for controlling a technical system, the computer-implemented method comprising the following method steps:

[0034] - Read in the data-based adjustment model configured according to the present invention,

[0035] - Read in the measured status data of the technical system,

[0036] - Determine optimized adjustment parameters for controlling the technical system by evaluating the configured data-based adjustment model based on the measured status data,

[0037] - Output the optimized adjustment parameters for controlling the technical system, and

[0038] - Control the technical system by means of the optimized adjustment parameters.

[0039] In particular, the method can be implemented by a regulator, which includes a data-based adjustment model configured according to the present invention for controlling the technical system. The regulator can in particular be understood as a device that sets or adjusts the desired gear, correct level, intensity, etc. of something in the case of a technical system (especially as part of a control loop).

[0040] According to another aspect, the present invention relates to a regulator for controlling a technical system, the regulator including

[0041] - A first interface for reading in the measured status data of the technical system,

[0042] - An adjustment module, which is configured to receive a data-based adjustment model configured according to the present invention and output optimized adjustment parameters by evaluating the data-based adjustment model based on the measured status data of the technical system, and

[0043] - A second interface for outputting the optimized adjustment parameters for controlling the technical system.

[0044] Furthermore, the present invention relates to a computer program product that can be directly loaded into a programmable computer, the computer program product including program code portions that cause the computer to perform the steps of the method according to the present invention when the program is executed by the computer.

[0045] The computer program product can be provided or supplied, for example, on a storage medium, such as a memory card, USB stick, CD-ROM, DVD, non-transitory storage medium, or can also be provided or supplied as a downloadable file from a server in a network. Description of the Drawings

[0046] Embodiments of the present invention are exemplarily illustrated in the drawings and will be explained in more detail according to the following description. Among them:

[0047] Figure 1 Shows an embodiment of a method for configuring a regulator to control a technical system according to the present invention;

[0048] Figure 2 Shows an embodiment of a device for configuring a regulator to control a technical system according to the present invention;

[0049] Figure 3 Shows an embodiment of a method for controlling a technical system according to the present invention; and

[0050] Figure 4 Shows an embodiment of a regulator for controlling a technical system according to the present invention. Detailed Description of the Invention

[0051] Parts corresponding to each other in all the figures are provided with the same reference numerals.

[0052] In particular, the following embodiments only show exemplary implementation possibilities of how such an implementation according to the teachings of the present invention might look, since it is not possible and also not convenient or necessary for understanding the present invention to name all these implementation possibilities.

[0053] In particular, all possibilities for implementing the present invention that are common in the prior art are of course also known to those skilled in the art (relevant) to understanding the one or more method claims, such that in particular no separate disclosure in the specification is required.

[0054] Figure 1 Shows an embodiment of a computer-implemented method for configuring a regulator to control a technical system according to the present invention. In particular, the technical system can be a complex technical facility, such as a factory facility. The computer-implemented method includes the following method steps:

[0055] First, a model predictive control model is provided, step S0. The model predictive control model preferably enables the determination and output of optimized control parameters for controlling the technical system. The model predictive control model is thus set up to output optimized control parameters as output data based on the (computer-based) simulated and / or measured state data of the technical system. In this case, the determination of the optimized control parameters is based on an optimization method.

[0056] For example, a model predictive regulation model can be generated based on methods of model predictive regulation or Lyapunov functions, where in particular the target behavior of the system to be controlled can be defined by defining formal specifications such as regulation quality, time requirements, prohibited operating ranges and behavioral functions, and other constraints such as logistics.

[0057] State data can be measured and provided, for example, by means of at least one sensor. Additionally or alternatively, state data can also be determined and provided by means of computer-aided simulation of a technical system. In this case, a model predictive regulation model can be generated based on a computer-aided simulation model of the technical system.

[0058] In the next step S1, the model predictive regulation model is read in. In particular, the model predictive regulation model can be used to determine optimized regulation parameters based on the provided state data of the technical system. Thus, a data pair consisting of the state data and the assigned optimized regulation parameters can be generated.

[0059] In the next step S2, a data-based regulation model is read in. The data-based regulation model can in particular be a reinforcement learning model implemented as an artificial neural network, for example. The data-based regulation model can in particular be preconfigured, i.e., for example, the data-based regulation model is suitable for controlling a technical system, but has not yet been optimized for this purpose. In other words, the form or preconfiguration of the data-based regulation model can thus preferably be pre-given.

[0060] In the next step S3, the configuration parameters of the data-based regulation model are adapted to reproduce the output data of the model predictive regulation model. For this purpose, the data pair consisting of the state data and the assigned optimized regulation parameters, which are provided by model predictive regulation, is used. These data pairs represent the first implementable result. The data-based regulation model (RL algorithm) is now set up such that the data-based regulation model reproduces this result.

[0061] The model predictive regulation model is in particular mapped to the data-based regulation model in order to set the configuration parameters. For example, the mapping can be carried out analytically or also numerically.

[0062] In other words, the internal structure of the model predictive regulation model is mapped to the internal structure of the data-based regulation model, such as by means of a geometric mapping, such that the data-based regulation model reproduces the model behavior of the model predictive regulation model. For this purpose, the configuration parameters of the data-based regulation model are set such that the data-based regulation model outputs the regulation parameters determined by the model predictive regulation model based on the input state data. This can be handled, for example, like an inverse problem by means of Bayesian fitting.

[0063] Formally, this mapping can be described as follows in other words:

[0064] Typically, a technical system has at least one observable, time-dependent state x(t), which depends on the adjustment parameter u(t). The following optimization problem can be solved by means of model predictive control: the function J(u,x) specifying the desired requirements for the technical system is minimized with respect to the adjustment parameter u in order to determine the optimized adjustment parameter u'. The optimization can be solved not only numerically but also analytically or by means of a black-box solver. This is preferably carried out for a large number of given initial states x0 in order to obtain the optimized adjustment parameters u'(x0) separately. These data pairs (x0,u'(x0)) can be used to configure a data-based control model. The model on which the data-based control model is based, such as an artificial neural network, can be designated as RL(w;x_0), where w represents the configuration parameters of the data-based control model. These parameters can be determined by mapping the model predictive control model to the data-based control model such that the output of the data-based control model agrees with the output of the model predictive model within a tolerance range. The configuration parameters w determined in this way can in particular be understood as the initial configuration parameters of the data-based control model, i.e., the data-based control model can be further adapted to the technical system by further training with the state data of the technical system.

[0065] In the next step S4, the thus-configured data-based control model is output to the controller. The technical system can be controlled by the controller by means of the data-based control model. This is in particular less computationally intensive and can therefore enable the control of the accompanying operation even in the case of complex technical systems.

[0066] The configured, data-based control model can preferably be further adapted to the technical system with the aid of other training data. For example, the data-based control model can be set up as an agent of a reinforcement learning method and can be adapted on the basis of other state data, for example during the continuous operation of the technical system, by means of the reinforcement learning method. Thus, the data-based control model / agent can be continuously improved.

[0067] Figure 2 An embodiment of a computer-implemented method for controlling a technical system by means of a configured, data-based control model according to the invention is shown.

[0068] In step S10, the data-based control model is read in. The data-based control model is preferably in accordance with as exemplarily according to Figure 1The method shown is configured such that a data-based regulation model is established for outputting optimized regulation parameters for a technical system based on given status data of the technical system. A data-based regulation model configured in this way is read in or loaded by the regulator, for example, in order to control the technical system.

[0069] In the next step S20, the measured status data of the technical system are read in. The status data are detected, for example, by means of at least one sensor. Based on these status data, the technical system should be optimally controlled by means of the regulator configured accordingly.

[0070] For this purpose, in the next step S30, optimized regulation parameters are determined by means of the data-based regulation model. The data-based regulation model is executed such that optimized regulation parameters are output based on the read-in status data, step S40. The regulator can control the technical system by means of the optimized regulation parameters, step S50.

[0071] Figure 3 An embodiment of a device 100 for configuring a regulator to control a technical system according to the invention is shown in block diagram. The device 100 is preferably coupled to the regulator.

[0072] The device 100 includes an interface 101, a configurator 102, and an output module 103.

[0073] The interface 101 is configured to read in a model predictive control model MPC and a data-based regulation model RL. The two regulation models MPC, RL can be stored externally, for example, and called from there.

[0074] The model predictive control model MPC is preferably generated based on a computer-aided simulation model SIM and is configured to determine optimized regulation parameters for controlling the technical system based on the status data generated by the simulation model SIM. The data-based regulation model RL can in particular be a reinforcement learning model.

[0075] The configurator 102 is configured to set configuration parameters K of the data-based regulation model RL according to the model predictive control model such that the data-based regulation model RL reproduces the output of the model predictive control model based on the read-in status data. The configuration parameters can be set, for example, by mapping the internal structure of the model predictive control model MPC to the internal structure of the data-based regulation model RL. For example, in this case this can be a geometric mapping.

[0076] The data-based regulation model RL(K) configured according to these configuration parameters K is transmitted to the output module 103. The output module 103 is configured to output the configured data-based regulation model RL(K) to the regulator for controlling the technical system TS.

[0077] Figure 4 FIG. shows an embodiment of a regulator R for controlling a technical system TS, such as a machine or a plant facility, according to the present invention.

[0078] The regulator R is preferably coupled to a device 100 for configuring the regulator according to the present invention as described, for example, according to Figure 3 or alternatively includes such a device 100 (not shown). The device 100 provides a configured, data-based regulation model RL(K), which is set up to output optimized regulation parameters for controlling the technical system TS.

[0079] The regulator R includes a first interface R1, a regulation module R2, and a second interface R3.

[0080] The configured, data-based regulation model RL(K) is received and read in from the device 100 via the first interface R1. In addition, the measured state data ZD of the technical system TS is read in via the first interface R1.

[0081] The regulation module R2 is set up to receive the data-based regulation model RL(K), or alternatively to store the data-based regulation model in an internal memory (not shown) and call it from there.

[0082] The regulation module R2 is furthermore set up to execute the data-based regulation model RL(K). Here, at least one optimized regulation parameter RP for controlling the technical system TS is determined and output according to the read-in state data ZD.

[0083] At least one optimized regulation parameter RP is output from the second interface R3 to the technical system TS. Thus, the technical system TS can be controlled by the regulator R according to the regulation parameter RP.

[0084] All the described and / or depicted features can be advantageously combined with each other within the scope of the present invention. The present invention is not limited to the described embodiments.

Claims

1. A computer-implemented method for configuring a regulator to control a technical system, a) reading in (Sl) a model predictive control model (MPC) for the regulator, wherein the model predictive control model is configured to output optimized control parameters for controlling the technical system as output data based on the simulated and / or measured state data of the technical system, wherein the model predictive control model (MPC) is configured based on a model predictive control method, b) reading in (S2) a data-based control model (RL), c) setting (S3) the configuration parameters (K) of the data-based control model (RL) according to the model predictive control model (MPC) such that the data-based control model reproduces the output data of the model predictive control model based on the read-in state data of the technical system, and determining optimized control parameters in such a configured manner, wherein the configuration parameters of the data-based control model are set by mapping the internal structure of the model predictive control model to the internal structure of the data-based control model, and d) outputting (S4) the thus-configured data-based control model to the regulator for controlling the technical system.

2. The computer-implemented method according to claim 1, wherein the data-based control model is configured based on a machine learning method.

3. The computer-implemented method according to claim 1 or 2, wherein the data-based control model is configured as an agent of a reinforcement learning method.

4. The computer-implemented method according to claim 3, wherein the configuration parameters of the data-based control model are adapted based on other state data and by means of a reinforcement learning method for determining other optimized control parameters.

5. A device (100) for configuring a regulator to control a technical system, the device comprising - an interface (101) configured to read in a model predictive control model (MPC) for the regulator, wherein the model predictive control model (MPC) is configured to output optimized control parameters for controlling the technical system as output data based on the simulated and / or measured state data of the technical system, and to read in a data-based control model, wherein the model predictive control model (MPC) is configured based on a model predictive control method, - A configurator (102) configured to set configuration parameters of the data-based adjustment model according to the model prediction adjustment model such that the data-based adjustment model reproduces output data of the model prediction adjustment model based on the read state data of the technical system, and to determine adjustment parameters optimized in such a configured manner, wherein the configuration parameters of the data-based adjustment model are set by mapping the internal structure of the model prediction adjustment model to the internal structure of the data-based adjustment model. - An output module (103) configured to output the configured data-based adjustment model to the regulator for controlling the technical system.

6. A computer-implemented method for controlling a technical system, the method comprising the following method steps: - Reading in (S10) a data-based adjustment model configured according to any one of claims 1 to 4. - Reading in (S20) the measured state data of the technical system. - Determining (S30) optimized adjustment parameters for controlling the technical system by evaluating the configured data-based adjustment model based on the measured state data. - Outputting (S40) the optimized adjustment parameters for controlling the technical system, and - Controlling (S50) the technical system by means of the optimized adjustment parameters.

7. A regulator (R) for controlling a technical system, the regulator comprising - A first interface (R1) for reading in the measured state data of the technical system. - An adjustment module (R2) configured to receive a data-based adjustment model configured according to any one of claims 1 to 4 and to output optimized adjustment parameters by evaluating the data-based adjustment model based on the measured state data of the technical system, and - A second interface (R3) for outputting the optimized adjustment parameters (RP) for controlling the technical system.

8. A computer program product that can be directly loaded into a programmable computer, the computer program product comprising program code portions suitable for implementing the steps of the method according to any one of claims 1 to 4.

Images