CONTROL DEVICE FOR CONTROLLING A TECHNICAL SYSTEM AND METHOD FOR CONFIGURING SUCH A CONTROL DEVICE - Patent application

Abstract

According to the present invention, the operation signal (BS) of the technical system is fed to a first machine learning module (NN1), which is trained to reproduce the inherent behavior signal of the technical system that occurs without the actual application of a control action based on the operation signal (BS) of the technical system and output the reproduced behavior signal (VSR1) as a first output signal. The first output signal (VSR1) is fed to a second machine learning module (NN2), which is trained to reproduce the resulting behavior signal of the technical system based on the control action signal (AS) and output the reproduced behavior signal (VSR2) as a second output signal. Furthermore, the operation signal (BS) of the technical system is fed to a third machine learning module (NN3), and the third output signal (AS) of the third machine learning module (NN3) is fed to the trained second machine learning module (NN2). The control action performance (Q) is determined based on the second output signal (VSR2).

Description

[Technical field]

[0001] Machine learning methods are increasingly used in the control of complex technical systems such as gas turbines, wind turbines, combustion engines, robots, manufacturing equipment or power systems. Such learning methods can be used to train a machine learning model of a control device based on training data, so as to determine a control action for controlling the technical system that produces a desired or optimized behavior specific to the technical system based on the actual operating signals of the technical system and optimizes its performance. Such machine learning models for controlling technical systems are often also called policies or control models. Many known training methods are available for training such policies, such as, for example, reinforcement learning methods.

[0002] However, in control optimization in industrial environments, many known training methods suffer from convergence problems and / or reproducibility problems in the learning process, for example, because only a small part of the state space of the technical system is considered, the sensor data of the technical system are noisy, and / or the control actions are generally accompanied by time delays, and different control actions often result in different time delays. The above symptoms often occur in complex real systems and significantly impair the learning effect. Summary of the Invention [Problem to be solved by the invention]

[0003] It is an object of the present invention to provide a control device and a method for configuring said control device which allows more effective training for controlling a technical system. [Means for solving the problem]

[0004] This problem is solved by a method having the features of claim 1, a control device having the features of claim 12, a computer program product having the features of claim 13 and a computer-readable storage medium having the features of claim 14.

[0005] To configure a controller for a technical system, an operating signal of the technical system is provided to a first machine learning module, which is trained to reproduce a behavior signal specific to the technical system that occurs without the actual application of a control action based on the operating signal of the technical system and to output the reproduced behavior signal as a first output signal. The first output signal is provided to a second machine learning module, which is trained to reproduce a resulting behavior signal of the technical system based on the control action signal and to output the reproduced behavior signal as a second output signal. Furthermore, the operating signal of the technical system is provided to a third machine learning module, and a third output signal of the third machine learning module is provided to the trained second machine learning module. A control action performance is determined based on the second output signal. The third machine learning module is thereby trained to optimize the control action performance based on the operating signal of the technical system. The controller is finally configured to control the technical system based on the third machine learning module and using the third output signal of the third machine learning module.

[0006] To carry out the method according to the invention, a controller, a computer program product and a preferably non-volatile computer readable storage medium are provided.

[0007] The method according to the present invention and the control device according to the present invention can be executed or implemented, for example, by one or more computers, processors, application specific integrated circuits (ASICs), digital signal processors (DSPs), and / or field programmable gate arrays (FPGAs).

[0008] The present invention allows the control device to be configured or trained significantly more effectively. Since the trained second machine learning module is used in training the third machine learning module, essential components of the system behavior generally do not need to be potentially learned or displayed any more when training the third machine learning module. In many cases, this leads to a significant improvement in the convergence behavior and / or a better reproducibility of the training results. The training is also often more stable and / or more robust against variations in the training data. Furthermore, in many cases, less training data, computation time, and / or computation resources are required.

[0009] Advantageous embodiments and developments of the invention are set forth in the dependent claims.

[0010] According to an advantageous embodiment of the invention, the third machine learning module can be trained on the basis of the first output signal, which allows the third machine learning module to be trained particularly effectively, since unique information about the system behavior is available to the third machine learning module without the actual application of a control action.

[0011] According to a particularly advantageous embodiment of the invention, the control action performance for the respective time point can be determined on the basis of a single time increment of the behavior signal. In many cases, a complex determination or estimation of future effects on the performance is not necessary. Thus, dynamic effects that are performed on different time scales can also be efficiently taken into account. This time increment can furthermore be of different length depending on the control action and / or the behavior signal and can also represent the future effects of the control action.

[0012] Advantageously, the first and / or second parts of the operating signals of the technical system can be selected in particular according to whether they comprise a control action or not, whereby the first part of the operating signals not comprising a control action can be used in particular for training the first machine learning module and / or the second part of the operating signals comprising a control action can be used in particular for training the second machine learning module. A particular selection of the training data for the respective training purpose allows a particularly effective training of the first and / or second machine learning modules.

[0013] According to another advantageous embodiment of the invention, a behavior signal setpoint can be read and the second output signal can be compared with this behavior signal setpoint. In this way, depending on the result of the comparison, the control performance can be determined. In particular, the deviation between the second output signal and the behavior signal setpoint can be determined, for example in the form of the absolute value of the difference or the square of the difference. The control performance can then be determined depending on this deviation, the greater the deviation, generally the lower the control performance.

[0014] Additionally, the behavior signal setpoints can be provided to a third machine learning module, such that the third machine learning module can be trained to optimize control operation performance based on the behavior signal setpoints.

[0015] According to another advantageous embodiment of the invention, the control operation performance can be determined on the basis of the first output signal. In this case, in particular the deviation between the first output signal and the second output signal can be determined, for example in the form of the absolute value of the difference or the square of the difference. Alternatively or additionally, the deviation between the sum of the first and second output signals and the behavior signal setpoint can be determined. The control operation performance can then be determined as a function of the deviation thus determined. In this case, it is possible to evaluate, in particular, how the system behavior with application of the control action differs from the system behavior without application of the control action. It has been found that using this difference, the determination of the control operation performance can often be significantly improved.

[0016] According to an advantageous development of the invention, the first and / or second machine learning module can be trained to reproduce separately a plurality of behavior signals of various processes executed in the technical system. The control operation performance can then be determined depending on the reproduced behavior signals. The first and / or second machine learning module can include for this purpose in particular a set of machine learning models or sub-models, each of which models a particular process executed in the technical system in a process-specific manner. Such separate training has often been found to be more effective than combined training, since the respective underlying individual dynamics generally have a simpler response behavior than the combined system dynamics.

[0017] Since the present invention allows the control operation performance to be determined at each time point based on a single, possibly adjustable, time increment of the behavior signal, synchronization problems between multiple processes with different execution speeds generally occur less, especially in the training of the third machine learning module. In many cases, it is possible to perform a relatively accurate and robust evaluation of the control operation performance for multiple process-specific machine learning models in a single step.

[0018] Furthermore, for each one of the behavior signals, a unique behavior signal setting value can be loaded. The control performance can then be determined based on a comparison between the reproduced behavior signal and the unique behavior signal setting value.

[0019] The third machine learning module can be trained to specifically optimize control operation performance based on specific behavior signal set points.

[0020] An embodiment of the invention will now be described in more detail with reference to the drawings, all of which are schematic diagrams. [Brief description of the drawings]

[0021] [Figure 1] 1 shows a gas turbine equipped with a control device according to the present invention; [Diagram 2] FIG. 2 shows a control device according to the invention in a first training phase. [Diagram 3] FIG. 13 shows a control device in a second training phase. [Figure 4] FIG. 13 illustrates the control device in a third training stage. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0022] 1 shows a gas turbine as an example of a technical system TS with a control device CTL, which may alternatively or additionally include a wind turbine, a combustion engine, a manufacturing installation, a chemical, metallurgical or pharmaceutical manufacturing process, a robot, a car, a power grid, a 3D printer or another machine, another device or another installation.

[0023] The gas turbine TS is connected to a control device CTL, which can be implemented as part of the gas turbine TS or wholly or partly outside the gas turbine TS. In FIG. 1, the control device CTL is shown for the sake of clarity outside the technical system TS.

[0024] The control device CTL is used to control the technical system TS and for this purpose is trained by machine learning methods. Control of the technical system TS here also means the regulation of the technical system TS as well as the output and use of data or signals which are related to the control, i.e. which contribute to the control of the technical system TS.

[0025] Such control-related data or signals include, in particular, control operation signals, prediction data, monitoring signals, status data and / or classification data, which can be used, in particular, for optimizing the operation, monitoring or maintenance and / or identifying wear or damage of the technical system TS.

[0026] The gas turbine TS has a number of sensors S which continuously measure one or more operating parameters of the technical system TS and output them as measured values. The measured values ​​of the sensors S and possibly also the operating parameters of the technical system TS detected in other ways are transmitted from the technical system TS to the control device CTL as operating signals BS.

[0027] The operating signals BS may include, in particular, physical, chemical, regulation-related, efficiency-related and / or structure-related operating variables, characteristic data, power data, efficiency data, status signals, behavior signals, system data, preset values, control data, control action signals, sensor data, measured values, peripheral data, monitoring data, forecast data, analysis data and / or other data occurring during the operation of the technical system TS and / or describing the operating state or control action of the technical system TS. These are, for example, data relating to the temperature, pressure, emissions, vibrations, vibration state or fuel consumption of the technical system TS. In particular in the case of a gas turbine, the operating signals BS relate to turbine power, rotational speed, vibration frequency, vibration amplitude, combustion dynamics, combustion alternating pressure amplitude or nitrous oxide concentration.

[0028] The trained control device CTL determines, based on the operating signals BS, control actions that optimize the performance of the technical system TS, in particular with regard to power, yield, speed, service life, precision, failure rate, fuel requirement, efficiency, pollutant emissions, stability, wear, service life and / or other target parameters of the technical system TS.

[0029] The determined performance-optimizing control actions are delivered to the technical system TS by transmitting corresponding control action signals AS from the control device CTL, which can, for example, regulate the gas supply, the gas distribution or the air supply in the case of a gas turbine.

[0030] 2 is a schematic diagram of a learning-based control device CTL according to the invention in a first training phase, which control device CTL is configured for controlling a technical system TS. Where identical or corresponding reference numbers are used in the figures, these refer to identical or corresponding objects.

[0031] In the present embodiment, a control device CTL is connected to the technical system TS. This control device CTL comprises one or more processors PROC for executing the method according to the invention and one or more memories MEM for storing method data.

[0032] The control device CTL receives a number of operating signals BS as training data from the technical system TS. These operating signals in particular comprise a number of time series, i.e. the time history of a number of operating parameter values ​​of the technical system TS. In the present embodiment, the operating signals BS comprise a status signal SS which specifies a state of the technical system TS over time, a control action signal AS which specifies or executes a control action of the technical system TS, and a behavior signal VS which specifies a system behavior of the technical system TS. The behavior signals can specify, for example, a combustion alternating pressure amplitude, an emission amount, a speed of the gas turbine or a change in temperature. The behavior signals VS can be understood to be status signals of the technical system TS which relate in particular to the performance of the technical system TS.

[0033] At least some of these operating signals BS can be received or derived from a technical system similar to this technical system TS, from a database containing stored operating signals of this technical system TS or a similar technical system, and / or from a simulation of this technical system TS or a similar technical system.

[0034] Furthermore, the control device CTL comprises a first machine learning module NN1, a second machine learning module NN2 and a third machine learning module NN3, where each machine learning module NN1, NN2 to NN3 can in particular be configured as an artificial neural network or as a collection of several neural sub-networks, the first machine learning module NN1 in particular being configured as a sub-module of the third machine learning module NN3.

[0035] The machine learning modules NN1, NN2 and / or NN3 may preferably use or implement supervised learning methods, reinforcement learning methods, recurrent neural networks, convolutional neural networks, Bayesian neural networks, autoencoders, deep learning architectures, support vector machines, data-driven trainable regression models, k-nearest neighbor classifiers, physical models, decision trees and / or random forests. Numerous effective implementations are available for the above mentioned variants and their training.

[0036] Training here generally means optimizing the reproduction of an input signal into an output signal. This reproduction is optimized according to pre-given and / or learned criteria during a training phase. In this case, as a criterion, for example, the prediction error can be used in a prediction model, the classification error in a classification model or the result of a control action in a control model. By means of this training, for example, the networking structure of a number of neurons of a neural network and / or the weights of the connections between the neurons can be adjusted or optimized in such a way that pre-given criteria are met as well as possible. Training can therefore be understood as an optimization problem. In the field of machine learning, many effective optimization methods are available for such optimization problems. Gradient descent, particle swarm optimization and / or genetic optimization methods can in particular be used.

[0037] In a first training phase, shown in Fig. 2, a first machine learning module NN1 is trained. It is trained to predict or reproduce the behavior of the technical system TS based on its operating signals, which is deployed without actually performing any control actions.

[0038] In order to improve the training effect, preferably, a plurality of training data BS are filtered by a filter F1 connected to the first machine learning module NN1 in order to obtain training data without control actions or without the influence of control actions. For this purpose, a plurality of driving signals BS are supplied to the filter F1. The filter F1 comprises a control action detector ASD for detecting a control action in the driving signals BS based on a control action signal AS included in the driving signals.

[0039] In response to the detection of the control action by the control action detector ASD, a first part of the operating signal BS is selected by the filter F1 and extracted from the operating signal BS. In this case, the first part of the operating signal BS that does not include the control action and / or does not include the influence of the control action is preferably selected. Thus, the first part of the operating signal BS can be extracted, for example, from a time window after the actual control action is detected, and in this case, the time window is selected so that the control action cannot yet affect the system behavior.

[0040] The first part of the filtered driving signal BS includes a first part SS1 of the state signal SS and a first part VS1 of the behavior signal VS. These first parts SS1 and VS1 are output through a filter F1 and used to train a first machine learning module NN1.

[0041] The first part SS1 of the state signal SS is supplied as an input signal to the first machine learning module NN1 for training. The objective of this training is that the first machine learning module NN1 reproduces as well as possible an operating signal of the technical system TS based on the operating signal of the technical system TS, which would occur without the actual application of a control action. This means that the output signal VSR1 of the first machine learning module NN1 (hereinafter referred to as the first output signal) corresponds as well as possible to the actual behavior signal of the technical system TS. For this purpose, a deviation D1 between the first output signal VSR1 and the corresponding first part VS1 of the behavior signal VS is determined. In this case, the deviation D1 represents the reproduction or prediction error of the first machine learning module NN1. The deviation D1 is in particular calculated as follows: D1=(VS1-VSR1) 2 , or it can be calculated as the square or absolute value of the difference, in particular the vector difference, by D1=│VS1-VSR1│.

[0042] As shown by the dashed arrow in FIG. 2, the deviation D1 is fed back to the first machine learning module NN1. Based on the fed back deviation D1, the first machine learning module NN1 is trained in such a way that this deviation D1, and thus the reproduction error, is minimized. To minimize the deviation D1, as already mentioned above, a large number of optimization methods are available, such as gradient descent, particle swarm optimization or genetic optimization. In this way, the first machine learning module NN1 is trained by a supervised learning method. The first output signal VSR1 of the trained first machine learning module NN1 reproduces the operating signal of the technical system TS that occurs without the actual application of a control action.

[0043] Since the filtered operating signals SS1 and VS1 are used for training, the first machine learning module NN1 is trained particularly effectively for this training purpose. It should be noted that this first machine learning module NN1 can also be trained outside the control device CTL.

[0044] The above-described training method can be used in a particularly advantageous manner to reproduce several behavioral signals of various processes executed in the technical system TS separately. For this purpose, the first machine learning module NN1 can comprise several process-specific neural sub-networks, which are trained separately, i.e. individually, with the process-specific behavioral signals, as described above. Such separate training often proves to be more effective than combined training, since the respective underlying individual dynamics themselves generally have a simpler and / or more uniform response behavior.

[0045] 3 shows the control device CTL in a second training phase, in which the second machine learning module NN2 is trained to predict or reproduce, based on the operating signals BS of the technical system TS, in particular on the control action signals AS contained therein, the behavior of the technical system TS induced by the respective control actions.

[0046] To train the second machine learning module NN2, the control device CTL receives from the technical system TS as training data the operating signals BS of the technical system TS. As already mentioned, the operating signals BS include, in particular, a plurality of time series of state signals SS, control action signals AS and behavior signals VS. To train the second machine learning module NN2, the trained first machine learning module NN1 is additionally used. In this embodiment, when training the second machine learning module NN2, the training of the first machine learning module NN1 has already been completed.

[0047] To improve the training effect, the training data BS is filtered by a filter F2 connected to the second machine learning module NN2 in order to preferably obtain training data including multiple control actions or the effects of these control actions.

[0048] For this purpose, the driving data BS is fed to a filter F2. The filter F2 includes a control action detector ASD for detecting the control actions in the driving signal BS individually based on the control action signals AS contained therein. Depending on the detection of the control actions by the control action detector ASD, a second part of the driving signal BS is selected by the filter F2 and extracted from the driving signal BS. In this case, preferably, the second part of the driving signal BS including the control actions and / or the effects of these control actions is selected. The second part of the driving signal BS can be extracted, for example, from a time window related to the respectively detected control action and / or from a time window in which the effects of the respective control action are expected. The filtered second part of the driving signal BS includes in particular a second part VS2 of the behavior signal VS and a second part AS2 in which the control action signals are included. The second parts AS2 and VS2 of the driving signal BS are output through the filter F2 and used to train a second machine learning module NN2.

[0049] For this training, the second part AS2 of the driving signal BS is fed as an input signal to the second machine learning module NN2. Furthermore, the driving signal BS is fed to the already trained first machine learning module NN1, which derives therefrom a behavior signal VSR1 and outputs it as a first output signal. This behavior signal VSR1 reproduces, as described above, the behavior signal of the technical system TS, which occurs without the actual application of a control action. This behavior signal VSR1 is fed as a further input signal to the second machine learning module NN2.

[0050] The objective of this training is for the second machine learning module NN2, on the basis of an operating signal comprising a plurality of control actions, here AS2, and on the basis of a behavior signal resulting without the actual application of the control actions, here VSR1, to reproduce as accurately as possible the behavior signals of the technical system TS derived by these control actions, i.e. the objective is that the output signal VSR2 of the second machine learning module NN2 (hereinafter referred to as the second output signal) corresponds as well as possible to the actual behavior signals of the technical system TS under the influence of the plurality of control actions.

[0051] In the course of this training, a deviation D2 between the second output signal VSR2 and the corresponding second part VS2 of the behavior signal VS is determined. In this case, this deviation D2 represents the reproduction or prediction error of the second machine learning module NN2. This deviation D2 can be calculated, for example, as follows: D2=(VS2-VSR2) 2 , or it can be calculated as the square or absolute value of the difference, in particular the vector difference, by D2 = |VS2-VSR2|.

[0052] As shown by the dashed arrow in Fig. 3, the deviation D2 is fed back to the second machine learning module NN2. Based on the fed back deviation D2, the second machine learning module NN2 is trained so that this deviation D2, and thus the recall error, is minimized. As already mentioned, many known optimization methods, in particular supervised learning methods, can be used to minimize the deviation D2.

[0053] The second output signal VSR2 of the trained second machine learning module NN2 reproduces the behavior signal of the technical system TS derived by the actual application of the control action.

[0054] The second machine learning module NN2 is trained particularly effectively for this training purpose, since the filtered driving signals AS2 and VS2 are used for this training. Furthermore, by supplying the behavior signal VSR1 to the second machine learning module NN2, it is often possible to significantly increase the training effect, since the second machine learning module NN2 is provided with unique information on the difference between the system behavior induced by the control action and the system behavior without the control action. It should be noted that the second machine learning module NN2 can also be trained outside the control device CTL.

[0055] The above-described training method can be used in a particularly advantageous manner to reproduce a plurality of behavior signals of different processes executed in the technical system TS separately. For this purpose, the second machine learning module NN2, like the first machine learning module NN1, can comprise a plurality of process-specific neural sub-networks, which are each trained separately, i.e. individually with the process-specific behavior signals, as described above.

[0056] FIG. 4 shows the control device CTL in a third training phase. In the third training phase, the third machine learning module NN3 is trained to generate control action signals based on the operating signals of the technical system TS, optimizing the performance for controlling this technical system. In this case, optimization also means approaching an optimum value. By training the third machine learning module NN3, the control device CTL is configured to control the technical system TS.

[0057] To train the third machine learning module NN3, the control device CTL receives from the technical system TS the operating signals BS of the technical system TS as training data. For this training, in particular the first machine learning module NN1 and the second machine learning module NN2, which have been trained as described above, are used. In this embodiment, when training the third machine learning module NN3, the training of the machine learning modules NN1 and NN2 has already been completed.

[0058] In addition to the above mentioned components, the control device CTL comprises a performance evaluator EV connected to the machine learning modules NN1, NN2 and NN3. Furthermore, the first machine learning module NN1 is connected to the machine learning modules NN2 and NN3, and the second machine learning module NN2 is connected to the third machine learning module NN3.

[0059] The performance evaluator EV is used to determine, for each control action, the performance of the behavior of the technical system TS caused by that control action. For this purpose, a reward function Q is evaluated. The reward function Q determines and quantifies the reward, i.e., the performance of the actual system behavior, as already mentioned several times here. Such a reward function is often also called a cost function, a loss function, an objective function, a reward function or a value function. The reward function Q can be implemented, for example, as a function of the operating states, of the control actions and of one or more set points OB for the system behavior.

[0060] In the case where multiple behavior signals are evaluated by the machine learning modules NN1, NN2, NN3 and / or by the performance evaluator EV, multiple behavior signal setting values ​​OB can be pre-set specifically for each of the behavior signals.

[0061] To train the third machine learning module NN3, a plurality of driving signals BS are provided as input signals to the trained machine learning modules NN1 and NN2, as well as to the third machine learning module NN3.

[0062] Based on these operating signals BS, the trained first machine learning module NN1 reproduces a behavior signal VSR1 of the technical system TS that would occur without the actual application of a control action. This reproduced behavior signal VSR1 is provided from the first machine learning module NN1 to the second machine learning module NN2, to the third machine learning module NN3 and to the performance evaluator EV. Furthermore, one or more behavior signal set values ​​OB are provided to the third machine learning module NN3 and to the performance evaluator EV.

[0063] An output signal AS of the third machine learning module NN3 resulting from the driving signal BS, from the reproduced behavior signal VSR1 and from one or more behavior signal setpoints OB (hereinafter referred to as the third output signal) is then supplied as an input signal to the trained second machine learning module NN2. Based on the third output signal AS, the reproduced behavior signal VSR1 and the driving signal BS, the trained second machine learning module NN2 reproduces a behavior signal VSR2 induced by a control action of the technical system TS, which is supplied from the trained second machine learning module NN2 to the performance evaluator EV.

[0064] The performance evaluator EV quantifies the actual performance of the technical system TS based on the reconstructed behavior signal VSR2 and taking into account the reconstructed first behavior signal VSR1 and one or more behavior signal setpoints OB. In this case, the performance evaluator EV determines, in particular, a first deviation of the control action-induced behavior signal VSR2 from one or more behavior signal setpoints OB. If the deviation increases, the control action performance is usually reduced. Furthermore, a second deviation between the control action-induced behavior signal VSR2 and the behavior signal VSR1 is also determined. Based on this second deviation, the performance evaluator EV can evaluate how the system behavior with application of the control action differs from the system behavior without application of the control action. It has been found that this difference often allows a significant improvement of the performance evaluation.

[0065] The control action performance determined by the reward function Q is fed back to the third machine learning module NN3 as shown by the dashed arrow in Fig. 4. Based on the fed back control action performance, the third machine learning module NN3 is trained to maximize its control action performance. As already mentioned many times, many known optimization methods can be used to maximize the control action performance.

[0066] As long as the second machine learning module NN2 is able to reliably obtain as input signals in particular a control action signal, the third machine learning module NN3 is implicitly trained to output such a control action signal, here AS. By optimizing this control action performance, the third machine learning module NN3 is trained to output a control action signal AS that optimizes performance.

[0067] Since in addition to the driving signal BS, the reproduced behavior signal VSR1 is also used to train the third machine learning module NN3, the third machine learning module NN3 can be trained particularly effectively, since unique information about the system behavior without control actions is available to the third machine learning module NN3.

[0068] A particular advantage of the present invention is that when training the third machine learning module NN3, it is often sufficient to use the performance evaluator EV to evaluate only a single, optionally adjustable, time increment of the behavior signal for each one time point. In many cases, complex determination or estimation of future rewards is not necessary. Thus, effects that are performed at multiple different time scales can also be efficiently taken into account.

[0069] Furthermore, since multiple varying behavior signal setpoints OB are pre-given for the behavior signal VSR2, the pre-given data set of driving signals can be used multiple times to train the third machine learning module NN3, so that different, setpoint-specific control action signals can be learned from multiple identical driving signals, and a better coverage of the control action space can be achieved.

[0070] Due to this training of the third machine learning module NN3, the control device CTL is configured to control the technical system TS to optimize its performance using the control action signal AS of the trained third machine learning module NN3.

Claims

1. A computer-implemented method for configuring a control unit (CTL) for a technical system (TS), comprising the steps of: a) an operating signal (BS) of the technical system (TS) is supplied to a first machine learning module (NN1), the first machine learning module (NN1) being trained to reproduce a behavior signal specific to the technical system (TS) that occurs without the actual application of a control action based on the operating signal (BS) of the technical system (TS) and to output the reproduced behavior signal as a first output signal (VSR1); b) the first output signal (VSR1) is fed to a second machine learning module (NN2), which is trained to reproduce a resulting behavior signal of the technical system (TS) based on a control action signal (AS) and to output the reproduced behavior signal (VSR2) as a second output signal (VSR2); c) the operating signal (BS) of said technical system (TS) is fed to a third machine learning module (NN3), d) a third output signal (AS) of the third machine learning module (NN3) is provided to the trained second machine learning module (NN2); e) determining a control performance (Q) based on the second output signal (VSR2); f) the third machine learning module (NN3) is trained to optimize the control operation performance (Q) based on the operating signal (BS) of the technical system (TS); g) the control device (CTL) is configured to control the technical system (TS) based on the third machine learning module (NN3) using the third output signal (AS) of the third machine learning module (NN3), the third machine learning module (NN3) is trained based on the first output signal (VSR1); A behavior signal setpoint (OB) is read, and the second output signal (VSR2) is compared with the behavior signal setpoint (OB); The control performance (Q) is determined based on the results of the comparison; and The behavior signal setting value (OB) is provided to the third machine learning module (NN3); The third machine learning module (NN3) is trained to optimize the control operation performance (Q) based on the behavior signal setpoint (OB). A method comprising:

2. 2. The method according to claim 1, characterized in that the control performance (Q) is determined for each point in time in each case based on a single time increment of the behavior signal.

3. a first part (SS1, VS1) and / or a second part (AS2, VS2) of an operating signal (BS) of said technical system (TS) is selected depending on whether they include a control action or not, A first part (SS1, VS1) of the driving signal (BS) not including a control action is used to train the first machine learning module (NN1), and / or a second part (AS2, VS2) of the driving signal (BS) including a control action is used to train the second machine learning module (NN2).

3. The method according to claim 1 or 2.

4. 4. The method according to claim 1, wherein the control performance (Q) is determined based on the first output signal (VSR1).

5. A deviation between the first output signal (VSR1) and the second output signal (VSR2) is determined; The control operation performance (Q) is determined according to the deviation.

5. The method of claim 4.

6. said first machine learning module (NN1) and / or said second machine learning module (NN2) are trained to reproduce separately a plurality of behavioral signals of various processes executed in said technical system (TS); The control performance (Q) is determined in response to the reproduced behavior signal.

6. The method according to claim 1, wherein the first and second electrodes are connected to a first electrode.

7. A specific behavior signal setting (OB) is read for each behavior signal (VS), The control performance (Q) is determined based on a comparison of the reproduced behavior signal (VS) with the specific behavior signal set point (OB).

7. The method of claim 6.

8. 8. The method of claim 7, wherein the third machine learning module (NN3) is trained to optimize the control action performance (Q) based on the specific behavior signal setpoint (OB).

9. A control device (CTL) for controlling a technical system (TS), the control device (CTL) being adapted to carry out a method according to any one of claims 1 to 8.

10. A computer program adapted to carry out the method according to any one of claims 1 to 8.

11. A computer readable storage medium containing the computer program of claim 10.

Images