Method for training an array of neural networks and dynamic system

Abstract

The invention relates to a method for training an array (1) of neural networks (10) for real-time control of a dynamic system (100), the array (1) of neural networks (10) being trained using training data from an input space (2). Each of the neural networks (10) of the array (1) is trained exclusively using training data from an associated subset (20) of the input space (2). The subsets (20) belonging to different neural networks (10) of the array (1) are disjoint and the array (1) of neural networks (10) is trained under at least one secondary condition, according to which an output of the array (1) of neural networks (10) is continuous at at least one transition (21) between two adjoining subsets (20) of the input space (2). A dynamic system is also disclosed.

Description

[0001] P2024, 0761 WO N / INA - 2024P67177 WO 21. August 2025

[0002] 1

[0003] Description

[0004] METHOD FOR TRAINING AN ENSEMBLE OF NEURAL NETWORKS AND DYNAMIC SYSTEM

[0005] A method for training an ensemble of neural networks for the real-time control of a dynamic system and a dynamic system are specified.

[0006] At least one of the various implementation forms is to specify a method for training an ensemble of neural networks that enables real-time control of a dynamic system with improved stability. At least another of the various implementation forms is to specify a dynamic system that is controlled by such an ensemble of neural networks.

[0007] These tasks are solved by a method and an object according to the independent claims. Further advantageous embodiments and developments of the method for training an ensemble of neural networks for the real-time control of a dynamic system and of the dynamic system itself are specified in the dependent claims.

[0008] According to one implementation form, the procedure for training an ensemble of neural networks for the

[0009] Real-time control of a dynamic system: the ensemble of neural networks trained with training data from an input space. The dynamic system is specifically a technical system. For example, the dynamic system has at least one degree of freedom that can change as a function of time. P2024, 0761 WO N / INA - 2024P67177 WO 21 August 2025

[0010] - 2 -

[0011] The method is specifically designed for controlling a dynamic system, for example, for real-time control of a dynamic system. Real-time control is, in particular, a hard-time control.

[0012] Real-time control. In hard real-time control, the ensemble of neural networks must, for example, process input data within a fixed predefined time, such as within a fixed predefined inference time, and provide output values.

[0013] The ensemble of neural networks comprises, for example, at least two, three, four, five, ten, one hundred, or a multitude of neural networks. These neural networks are specifically artificial neural networks. For example, the untrained neural networks of the ensemble are independent of each other. In particular, there are no connections between neurons of different neural networks within the ensemble. The neural networks of the ensemble can have different topologies or the same topology.

[0014] The characteristics of a neural network described below apply in particular to at least one, to several, to a large proportion, or to all neural networks of the ensemble. For example, the neural network is a convolutional neural network (CNN). The neural network is, for example, a feedforward neural network or a recurrent neural network. The neural network can be single-layered or multi-layered. For example, the neural network includes an input layer and an output layer. [The neural network P2024, 0761 WO N / INA - 2024P67177 WO 21. August 2025]

[0015] 3

[0016] A network can also have one or more hidden layers, which are specifically located between the input layer and the output layer. Each layer of the neural network contains at least one neuron, or preferably a plurality of neurons, which are not interconnected within that layer. The number of neurons in different layers can vary. Each neuron has at least one or more inputs and at least one output. An output value at the neuron's output depends, for example, on a weighted average of the input values ​​at that neuron's inputs. For instance, the neuron's output value corresponds to the value of an activation function that calculates the output value as a function of the weighted average of the neuron's input values.In neural networks with forward feedback, outputs of neurons in one layer are only connected to inputs of neurons in a subsequent layer. In recurrent neural networks, outputs of neurons in one layer can also be connected to inputs of neurons in a preceding layer.

[0017] The input values ​​of the neural network's input layer correspond, for example, to state data of the dynamic system and / or state data of its environment. For instance, the state data might be sensor data from sensors of the dynamic system. The output values ​​of the neural network's output layer might include, for example, control data for operating the dynamic system. During normal operation, different neural networks within the ensemble process distinct ranges of state data, with these ranges preferably not overlapping. P2024, 0761 WO N / INA - 2024P67177 WO 21 August 2025

[0018] - 4 -

[0019] The input space corresponds, for example, to a space, in particular a vector space, of all possible input values ​​that are processed by the ensemble of neural networks during intended operation. For example, the input space corresponds to the set of all possible state data. For example, the input space corresponds to the set of all possible sensor data from a given arrangement of sensors in the dynamic system. For example, the dynamic system includes an optical image sensor, and the sensor data corresponds to pixel values ​​of the image sensor. In this case, the input space includes, for example, the vector space of all possible pixel values ​​of the image sensor and can also include sensor data from other sensors.

[0020] The ensemble of neural networks is trained, for example, by at least approximately minimizing a training target function. The training target function parameterizes, for instance, a deviation or error of the actual output values ​​of the ensemble of neural networks from desired target values, averaged over a large number of training data points. This at least approximate minimization is performed particularly with respect to the weights with which the neurons calculate the weighted average of the input values ​​at their respective inputs.

[0021] According to another implementation of the method, each of the ensemble's neural networks is trained exclusively with training data from a corresponding subset of the input space. Specifically, each neural network in the ensemble is assigned exactly one subset of the input space. Input data or training data is, for example, only used by that particular neural network in the ensemble P2024, 0761 WO N / INA - 2024P67177 WO 21. August 2025

[0022] - 5 - processes the input data or training data from whose associated subset of the input space the input data or training data is taken. For example, the subsets correspond to subspaces or subvector spaces of the input space. For example, a sum of all subspaces corresponds to the complete input space. The number of subsets preferably corresponds to the number of neural networks in the ensemble.

[0023] According to another implementation of the method, the subsets belonging to different neural networks of the ensemble are disjunct. In other words, the subsets of the input space do not overlap, so that each element of the input space is assigned to exactly one subset. The subsets preferably border directly on each other. In other words, each element of the input space is assigned to exactly one subset.

[0024] According to another implementation of the method, the training of the ensemble of neural networks is subject to at least one constraint: that the output of the ensemble of neural networks is continuous or continuously differentiable at at least one transition between two adjacent subsets of the input space. In other words, the ensemble of neural networks is trained such that its output for training data from a region that includes at least one transition or boundary between two adjacent subsets of the input space is continuous or continuously differentiable. Specifically, the output of the trained ensemble of neural networks for immediately adjacent training data from immediately adjacent subsets of the input space changes continuously and does not jump. P2024, 0761 WO N / INA - 2024P67177 WO 21 August 2025

[0025] - 6 -

[0026] For example, the training of the ensemble of neural networks is subject to a multitude of constraints, according to which the output of the ensemble of neural networks must be continuous or continuously differentiable at all transitions between all adjacent subsets of the input space. Thus, the output of the trained ensemble of neural networks exhibits, in particular, no discontinuities, jumps, or discontinuities at transitions between subsets of the input space. This allows, for example, instabilities in the real-time control of the dynamic system to be avoided or reduced. Such instabilities include, for example, unwanted high-frequency oscillations, other transient reactions such as chattering, and a general deterioration in the real-time behavior of the dynamic system.Such instabilities can significantly impair the behavior of the dynamic system and / or the predictability of the control of the dynamic system.

[0027] The output of the ensemble of neural networks corresponds, in particular, to the output of that neural network within the ensemble from whose associated subset of the input space the input data or the training data originates. For example, the output of the neural network includes, or consists of, the output values ​​of the output layer of that neural network. Alternatively or additionally, the output of the neural network can also include logits and / or arguments of the activation function of the neurons in the output layer. P2024, 0761 WO N / INA - 2024P67177 WO 21 August 2025

[0028] - 7 -

[0029] According to a preferred implementation form, the procedure for training an ensemble of neural networks for real-time control of a dynamic system includes a step in which

[0030] - the ensemble of neural networks is trained with training data from the input space, whereby

[0031] - each of the ensemble's neural networks is trained exclusively with training data from the corresponding subset of the input space,

[0032] - the subsets belonging to different neural networks of the ensemble are disjunct, and

[0033] - the training of the ensemble of neural networks is carried out under at least one constraint, according to which an output of the ensemble of neural networks is continuous at at least one transition between two adjacent subsets of the input space.

[0034] Artificial neural networks are increasingly used to control dynamic systems, for example in industrial control engineering. For instance, robots can learn to navigate autonomously and in real time, identify and manipulate objects, and understand complex environments with the help of neural networks. However, the storage space required to store complex neural networks and the computing power needed to process data with such networks often conflict with the requirements or system performance of embedded or resource-constrained dynamic systems. Furthermore, the hard real-time constraints of such systems pose a significant challenge. For example, strict cycle times for immediate responses and the safety of such systems are essential, whereas large neural networks, in contrast, [P2024, 0761 WO N / INA - 2024P67177 WO 21. August 2025]

[0035] - 8 - time-consuming inferences may be required, which can exceed these defined cycle times. This is particularly problematic in time-critical applications such as high-frequency motion control, where cycle times can be less than 100 microseconds or less than 250 microseconds. This limits the acceptable computational effort of the inference and thus the complexity of the neural network. At the same time, the accuracy of such control methods is of high relevance.

[0036] The method described here is based, among other things, on the idea of ​​reducing the inference or reaction time of a neural network for the real-time control of a dynamic system, while ensuring accuracy and / or stability of the real-time control of this dynamic system. To this end, instead of a single, complex global neural network, an ensemble of comparatively smaller neural networks is used to control the dynamic system, with the ensemble, for example, mimicking the behavior of the global neural network. Here and in the following, the size of a neural network is determined in particular by the number of neurons and / or by the number of parameters in the respective neural network, for example, by the number of weights of the neurons or the number of connections between the neurons.Each of the ensemble's neural networks processes only data from a corresponding subset of the input space. The smaller size of the ensemble's neural networks reduces their complexity and, consequently, their inference time.

[0037] To ensure the stability of the real-time control of the dynamic system, transitions between P2024, 0761 WO N / INA - 2024P67177 WO 21. August 2025 are to be carried out.

[0038] 9

[0039] Subsets of the input space should not exhibit discontinuities in the ensemble's output. Such discontinuities can lead to unintended instabilities in the control of the dynamic system and are avoided by the constraint described above during ensemble training.

[0040] According to another implementation of the procedure, at least one global neural network is first trained with training data from the entire input space, and during the training of the ensemble of neural networks, the deviation between the output of the at least one global neural network and the output of the ensemble's neural networks is minimized. The global neural network may exhibit one or more of the features described above in connection with the ensemble's neural networks. For example, the global neural network could be a feedforward neural network or a recurrent neural network.

[0041] For example, the training target function for training the ensemble of neural networks parameterizes a deviation or error of the ensemble's output values ​​from the output of the global neural network, averaged over a large number of training data points. In other words, the ensemble of neural networks is trained to mimic the outputs of at least one global neural network.

[0042] In particular, the ensemble's neural networks are smaller than at least one global neural network. For example, the number of neurons in the global neural network is smaller by at least a factor of 10, by at least a factor of 100, by at least a factor of 1000, or by at least a factor of P2024, 0761 WO N / INA - 2024P67177 WO 21 August 2025

[0043] - 10 -

[0044] The number of neurons in each individual neural network of the ensemble is 10,000 times greater than the number of neurons in any one network. Due to the smaller number of neurons in the ensemble's neural networks, these networks exhibit a shorter inference time compared to the global neural network. The inference time corresponds specifically to the time the neural network requires to calculate the output values.

[0045] In the following, at least one global neural network will be referred to as the teacher, while the ensemble's neural networks will be referred to as students. The procedure described above, in which the ensemble of students imitates the teacher, has, for example, the following advantages and can exhibit the features explained below:

[0046] - A larger neural network, such as that of the teacher, has redundant parameters which, while contributing to generalization, are not strictly necessary for the output. Through the distillation process described here, a student can learn the teacher's encapsulated information without inheriting these redundancies;

[0047] - Additionally, this imitation of the teacher by the students can have an intrinsic regularization effect, since each student can be trained with arbitrarily densely sampled outputs of the teacher. This makes overfitting of the students to a non-representative population less likely, especially if the teacher himself has good generalization properties;

[0048] Soft labels can improve student training. Soft labels are specifically a smoothed distribution of the instructor's logits, for example, P2024, 0761 WO N / INA - 2024P67177 WO 21 August 2025

[0049] - 11 - to the outputs before the last activation function of the teacher's neural network. In contrast to hard labels, such as coded one-hot labels, soft labels can contain more information about the teacher's decision boundaries and uncertainties, which can improve the students' learning process. As a counterpart to soft labels in regression problems, for example, the statistical moments of an ensemble can be used, allowing its uncertainty to be incorporated into the students' learning process. Alternatively or additionally, when training students by imitating the teacher, intermediate feature maps, adversarially learned error functions, parameter sharing, and / or attention-like mechanisms can be used, where the learning process is controlled by the teacher.

[0050] Instead of a single global neural network, an ensemble of global neural networks, i.e., an ensemble of teachers, can initially be trained with training data from the entire input space. For example, the global neural networks of the ensemble of teachers have different topologies. The outputs of the ensemble of teachers are then averaged to subsequently train the ensemble of neural networks, i.e., the ensemble of students.

[0051] According to another implementation of the procedure, the output includes at least one logit from an output layer of the respective neural network. The output may also include all logits of the output layer of the respective neural network or consist of all such logits. For example, the logit is a non-normalized prediction of neural network P2024, 0761 WO N / INA - 2024P67177 WO 21 August 2025

[0052] - 12 -

[0053] Network. In particular, the logit is the argument of the activation function of a neuron in the output layer. For example, the logits are inputs of softmax functions in the output layer.

[0054] According to another implementation of the method, the training of the ensemble of neural networks is subject to at least one further constraint, namely that at least a partial derivative of the output of the ensemble of neural networks at the at least one transition between two adjacent subsets of the input space is continuous. In particular, the output of the ensemble of neural networks at the at least one transition between two adjacent subsets of the input space is continuous, simply continuously differentiable, or n-times continuously differentiable. For example, the output of the ensemble of neural networks at the at least one transition between two adjacent subsets of the input space is once, twice, or three times continuously differentiable.For example, a first-order partial derivative, a second-order partial derivative, and / or a third-order partial derivative of the output of the ensemble of neural networks is continuous at at least one transition between two adjacent subsets of the input space.

[0055] For example, the training of the ensemble of neural networks is carried out under a multitude of constraints, such that a multitude of partial derivatives of the output of the ensemble of neural networks are continuous at at least one transition between two adjacent subsets of the input space, or at all transitions between all subsets of the input space. For example, P2024, 0761 WO N / INA - 2024P67177 WO 21 August 2025

[0056] - 13 - the training of the ensemble of neural networks under a multitude of constraints, according to which all partial derivatives of the output of the ensemble of neural networks are continuous up to a given degree of derivatives at at least one transition between two adjacent subsets of the input space or at all transitions between all subsets of the input space.

[0057] According to another implementation of the method, the training of the ensemble of neural networks is subject to at least one further constraint: the expected value of a Frobenius norm, a Jacobian matrix, and / or at least one Hessian matrix of the output of the ensemble of neural networks must be continuous at at least one transition between two adjacent subsets of the input space. For example, the expected value of the Frobenius norm, the Jacobian matrix, and / or the at least one Hessian matrix of the output of the ensemble of neural networks is continuous at all transitions between all subsets of the input space. In particular, the expected values ​​of the Frobenius norm, the Hessian matrices of all components of the output of the ensemble of neural networks are continuous at all transitions between all subsets of the input space. Here, the expected value specifically denotes an average over a large number of training data sets.In contrast to the case of numerous constraints, where all partial derivatives of the output of the ensemble of neural networks are continuous up to the first derivative and / or up to the second derivative at the transitions, the number of constraints can thus be advantageously reduced, thereby simplifying the training. P2024, 0761 WO N / INA - 2024P67177 WO 21 August 2025.

[0058] - 14 -

[0059] According to another implementation of the method, at least one constraint is implemented during the training of the ensemble of neural networks by adding at least one penalty term to a training target function to be minimized. For example, the penalty term has a minimum, preferably a global minimum, if at least one constraint is satisfied. If the constraint can be formulated, for example, in the form g(x) = 0, where x is a data point from the input space and g(x) is a function of x, then the penalty term can, for example, take the form (g(x)). A The two terms can be weighted with an additional constant factor. By adding at least one penalty term, the optimization problem constrained by the constraint can be advantageously transformed, at least approximately, into an unconstrained optimization problem for training the ensemble of neural networks.

[0060] According to another implementation of the method, at least one constraint is taken into account during the training of the neural network ensemble using an augmented Lagrange method. In the augmented Lagrange method, for example, a penalty term described above is combined with another Lagrange multiplier term when minimizing the training target function, in order to at least approximately satisfy the at least one constraint. This can, for example, accelerate convergence and / or avoid numerical instabilities when optimizing the training target function with the at least one constraint.

[0061] According to another implementation form of the procedure, the subsets correspond to directly adjacent ones, P2024 , 0761 WO N / INA - 2024P67177 WO 21 August 2025

[0062] - 15 - connected subvolumes of the input space. For example, the input space is divided equally into the subvolumes. For example, the volumes of all subsets are equal or approximately equal.

[0063] According to another implementation of the procedure, the respective subvolume of a corresponding subset depends on a local complexity measure of the ensemble of neural networks. Alternatively or additionally, the respective subvolume of a corresponding subset depends on a local complexity measure of the at least one global neural network. The local complexity measure corresponds, for example, to a second derivative, in particular a curvature, of the output of the ensemble of neural networks or the output of the at least one global neural network. For example, the local complexity measure corresponds to the local curvature at the center of the respective subvolume, or to a curvature averaged over the respective subvolume. For example, the respective subvolume is smaller the larger the local complexity measure is. For example, the respective subvolume is inversely proportional to the local complexity measure.In particular, the input space is divided into subsets such that the complexity of each neural network in the ensemble is equal or approximately equal. For example, resulting Voronoi cells exhibit at least approximately the same curvature density.

[0064] Tree-based methods that generate a curvature density that is as homogeneous as possible can also be used to divide the input space into subsets. In this case, a lookup (i.e., an assignment of input data to a respective neural network) scales. P2024, 0761 WO N / INA - 2024P67177 WO 21 August 2025

[0065] 16 of the ensemble, advantageously only logarithmic instead of linear with the number of neural networks in the ensemble.

[0066] Furthermore, a dynamic system is specified. This dynamic system is controlled, in particular, using an ensemble of neural networks trained with the procedure described above. All features of the procedure for training an ensemble of neural networks for the real-time control of a dynamic system also apply to the dynamic system of Fenbart, and vice versa.

[0067] According to one implementation, the dynamic system comprises a control unit configured to control, in particular to control in real time, the dynamic system based on sensor data. For example, the control unit comprises an application-specific integrated circuit (ASIC) and / or a processor. The control unit is electrically connected to sensors to receive sensor data during operation. Furthermore, the control unit is electrically connected to control elements, such as actuators, to control the dynamic system in real time based on the sensor data during operation.

[0068] According to another implementation of the dynamic system, real-time control of the dynamic system is achieved using an ensemble of neural networks whose input space corresponds to the possible sensor data. This ensemble of neural networks corresponds specifically to the ensemble of neural networks described in connection with the above procedure. P2024, 0761 WO N / INA - 2024P67177 WO 21 August 2025

[0069] - 17 -

[0070] According to another implementation form of the dynamic system, each of the ensemble's neural networks processes data exclusively from a corresponding subset of the input space.

[0071] According to another implementation form of the dynamic system, the subsets belonging to different neural networks of the ensemble are disj unkt.

[0072] According to another implementation form of the dynamic system, the training of the ensemble of neural networks is subject to at least one constraint, namely that the output of the ensemble of neural networks is continuous at at least one transition between two adjacent subsets of the input space.

[0073] According to a preferred implementation form, the dynamic system comprises:

[0074] - the control unit, which is set up for real-time control of the dynamic system depending on sensor data, wherein

[0075] - the real-time control of the dynamic system is achieved with the ensemble of neural networks, whose input space corresponds to the possible sensor data,

[0076] - each of the ensemble's neural networks processes exclusively data from the corresponding subset of the input space ,

[0077] - the subsets belonging to different neural networks of the ensemble are disjunct, and

[0078] - the training of the ensemble of neural networks was carried out under at least one constraint, according to which the output of the ensemble of neural networks at at least one transition P2024 , 0761 WO N / INA - 2024P67177 WO 21 August 2025

[0079] 18 between two adjacent subsets of the input space is continuous.

[0080] According to another implementation of the dynamic system, the dynamic system is a robot or a part of a robot, and the control unit controls the movement of the robot or the movement of the part of a robot. Specifically, the control unit controls the movement using an ensemble of neural networks. For example, the ensemble of neural networks processes sensor data as input data and provides control data for the robot or the part of a robot as output.

[0081] According to another implementation of the dynamic system, the inference time of the ensemble of neural networks is at most 250 microseconds, or at most 100 microseconds, or at most 10 microseconds. The inference time corresponds specifically to the response time of the ensemble of neural networks. For example, the inference time corresponds to the time between the input of input data to the ensemble of neural networks and the provision of output data by the ensemble of neural networks. For example, the input space is divided into subsets such that the size and / or complexity of each of the neural networks in the ensemble can be reduced to ensure an inference time of at most 250 microseconds, or at most 100 microseconds, or at most 10 microseconds.An inference time of at most 250 microseconds, or at most 100 microseconds, or at most 10 microseconds may be sufficient for the real-time control of a dynamic system, in particular a robot. P2024, 0761 WO N / INA - 2024P67177 WO 21 August 2025.

[0082] 19

[0083] According to another embodiment of the dynamic system, the dynamic system comprises a motor, in particular a reluctance motor, a servo motor, or a linear motor, and / or a magnetic levitation system.

[0084] According to another implementation of the dynamic system, the ensemble of neural networks is configured to model at least one control loop of the dynamic system and / or to provide feedforward control of the dynamic system. For example, the ensemble of neural networks is configured for model predictive control of the dynamic system, where the ensemble of neural networks forms, for instance, a dynamic model of a process to be controlled within the dynamic system. The ensemble of neural networks can also, for example, form an inverse process model that can be used for feedforward control of the dynamic system. In this case, the ensemble of neural networks is used, for example, in parallel with a controller of the dynamic system and provides, for instance, a manipulated variable for a control loop.

[0085] Further advantageous implementation forms and further developments of the method for training an ensemble of neural networks for the real-time control of a dynamic system and the dynamic system itself result from the exemplary implementations described below in conjunction with the figures.

[0086] Figures 1 and 2 show schematic representations of steps in a procedure for training an ensemble of neural networks for real-time control of a dynamic system according to an exemplary implementation. P2024, 0761 WO N / INA - 2024P67177 WO 21 August 2025

[0087] 20

[0088] Figure 3 shows a schematic representation of one step of a procedure for training an ensemble of neural networks according to an example.

[0089] Figure 4 shows a schematic block diagram of a dynamic system according to an exemplary embodiment.

[0090] Identical, similar, or similarly effective elements are marked with the same reference symbols in the figures. The figures and the relative sizes of the elements depicted within them are not to be considered to scale. Rather, individual elements may be exaggerated for clarity and / or better understanding.

[0091] Figure 1 schematically shows a first step of a method for training an ensemble 1 of neural networks 10 for the real-time control of a dynamic system 100 according to an exemplary embodiment. A global neural network 3 is trained with training data from the complete input space 2. The input space 2 comprises all possible state data of the dynamic system 100 and / or its environment, which can be determined, for example, by one, several, or a multitude of sensors 4 (see Figure 4). For the sake of clarity, the input space 2 is two-dimensional in Figure 1 and is spanned by the two input values ​​Xi and X2. In general, the input space 2 can have any dimension and can thus be spanned by any number of independent input values. For example, the input values ​​can be represented as a vector with any dimension. P2024, 0761 WO N / INA - 2024P67177 WO 21.August 2025.

[0092] - 21 -

[0093] For the sake of representation, the output of global neural network 3 in Figure 1 corresponds to a single output value y, which is a unique function of the input values ​​xi and X2. Thus, the output of global neural network 3 is represented in Figure 1 as a two-dimensional surface. In general, the output of global neural network 3 can comprise multiple or a plurality of output values ​​y and, for example, be a vector of arbitrary dimension that is a unique function of the input values ​​xi and X2. The output of global neural network 3 specifically includes control data for the dynamic system 100.

[0094] The global neural network 3, for example, is a convolutional neural network with an input layer 12, an output layer 14, and one, two, several, or a multitude of hidden layers 13 arranged in between (see, for example, Figure 4). Figure 1 schematically depicts a global neural network 3 with two hidden layers 14, each containing five neurons 11. However, the actual number of neurons 11 in the respective layers 12, 13, 14 can be considerably larger. For example, a layer 12, 13, 14 may contain at least 10, at least 100, at least 1000, or more neurons. In particular, the output value of each neuron 11 corresponds to the value of an activation function that depends on a weighted average of all the input values ​​of the respective neuron 11.

[0095] The training of global neural network 3 is achieved, for example, by at least approximately minimizing a training target function that parameterizes the deviation of the actual output values ​​y of global neural network 3 from desired output values ​​y, averaged over a large number of training data points. This includes P2024, 0761 WO N / INA - 2024P67177 WO 21. August 2025

[0096] 22 The training data input values ​​Xi, X2 from the input space 2 with corresponding desired output values ​​y . The training target function is minimized by adjusting the weights of the neurons 11 of the global neural network 3, for example using a gradient descent method.

[0097] Global neural network 3 is, in particular, a teacher used in a subsequent step of the procedure to train ensemble 1 of neural networks 10, i.e., the students. Global neural network 3 is larger and thus has more neurons 11 than any of the neural networks 10 in ensemble 1. Global neural network 3 is particularly robust and thus has high accuracy and high stability. For example, global neural network 3 has redundant parameters that serve for generalization but are not essential for the output. However, due to its complexity, global neural network 3 has a long inference time compared to ensemble 1 of neural networks 10 and is not suitable, or only suitable to a limited extent, for real-time control of the dynamic system 100.For example, the resources, in particular storage and / or computing capacity, of a control unit 101 of the dynamic system 100 (see Figure 4) are not sufficient to control the dynamic system 100 with the global neural network 3 in real time.

[0098] Figure 2 schematically shows a second step of a method for training an ensemble of neural networks 10 for the real-time control of a dynamic system 100 according to an exemplary embodiment. In this step, an ensemble of neural networks 10 is trained to produce the output of the global neural network described in connection with Figure 1. P2024, 0761 WO N / INA - 2024P67177 WO 21. August 2025

[0099] - 23 -

[0100] to imitate network 3 as closely as possible. The neural networks 10 of ensemble 1 are smaller than the global neural network 3 and therefore require fewer resources to generate output values ​​y within a given inference time. Figure 2 schematically depicts neural networks 10 of ensemble 1, each containing a hidden layer 14 with four neurons 11. The actual number of layers 12, 13, 14 and the number of neurons 11 in the respective layers 12, 13, 14 can be considerably larger, but is, in particular, smaller than in the global neural network 3.

[0101] In the example of Figure 2, the input space 2, spanned by the input values ​​Xi and X2, is divided into 5 x 5 equally sized, directly adjacent, disjoint subsets 20. In general, the number of subsets 20 can be arbitrary. The size or volume of each subset 20 can also depend on a complexity measure, such as curvature, of the global neural network 3. Each subset 20 of the input space 2 is assigned exactly one neural network 10 of the ensemble 1, which processes exclusively input values ​​Xi and X2 from that associated subset 20.

[0102] The training of Ensemble 1 neural networks 10 is performed by at least approximately minimizing a training target function that parameterizes the deviation of the output of Ensemble 1 neural networks 10 from the output of the global neural network 3, averaged over a multitude of input data. During the training of Ensemble 1 neural networks 10, the output of the respective neural network 10 can include the output data of output layer 13 and / or logits of output layer 13. The training of Ensemble 1 is performed under at least one P2024,0761 WO N / INA - 2024P67177 WO 21 August 2025

[0103] 24

[0104] A constraint is imposed whereby the output of the ensemble of neural networks 10 is continuous or continuously differentiable at at least one transition 21, preferably at all transitions 21, between adjacent subsets 20 of the input space 2. This constraint is, for example, at least approximately taken into account by an augmented Lagrange method in the approximate minimization of the training objective function.

[0105] The procedure for training the ensemble of neural networks 10 can be mathematically defined, for example, as follows: with the two constraints where the following definitions apply:

[0106] Here, N denotes s The number of neural networks 10 in the ensemble 1, 12, denotes a subset 20 of the input space 2, d is the dimensionality of the input space 2, N c denotes a degree of continuity (e.g. N) c(= 2 for 02- continuity), f is the output of the global neural network 3 and e- denotes the output of the i-th neural network 10 of the ensemble 1. , denotes the parameters, in particular the weights, of the i-th neural network 10 of the ensemble 1, with respect to which the training target function is minimized. E here denotes the expected value or

[0107] XEUj

[0108] Mean value with respect to a multitude of input values ​​x from the respective subset 12, of the input space 2 and ||. ||2 denotes the 2-norm or the Euclidean norm. The first P2024 , 0761 WO N / INA - 2024P67177 WO 21 . August 2025

[0109] - 25 -

[0110] The first constraint requires the pairwise disjunction of the subsets 12i. The second constraint requires that every partial derivative up to degree N be valid. cThe output of each neural network 10 of the ensemble 1 at every point on the surface of its associated disjoint subset 20 matches the corresponding partial derivative of the global neural network 3. This ensures that no discontinuities occur in the output of the ensemble 1 neural networks 10 at transitions between adjacent subsets 20. The multiindex a n It serves as a notation aid for generating every possible partial derivative of degree n.

[0111] Figure 3 shows a schematic example of the output of an ensemble 1 of neural networks 10, where, unlike the ensemble 1 described in conjunction with Figure 2, this ensemble 1 was trained without the constraint that the output of ensemble 1 should be continuous at the transitions between adjacent subsets 20 of the input space 2. Therefore, the output of this ensemble 1 of neural networks 10 exhibits discontinuities 22 at the transitions 21 between the subsets 20. Thus, the output value y of this ensemble 1 can jump around with small changes in the input values ​​Xi, X2 near the transitions 21, which can lead to undesired instabilities in the control of the dynamic system 100, such as high-frequency oscillations or transient responses.

[0112] The embodiment of the dynamic system 100 in Figure 4 comprises sensors 4, a control unit 101, and control elements 5. The sensors 4 provide sensor data that is processed by the control unit 101 during operation. The control unit 101 generates P2024, 0761 WO N / INA - 2024P67177 WO 21. August 2025

[0113] - 26 -

[0114] Control data for real-time control of the dynamic system 100, dependent on sensor data. During operation, this control data is used to control control elements 5, such as actuators, of the dynamic system 100. For example, the dynamic system 100 is a robot that performs predetermined actions based on sensor data.

[0115] The real-time control of the dynamic system 100 is achieved with an ensemble 1 of neural networks 10, which process the sensor data as input values ​​xi, X2 and output the control data as output value y. The ensemble 1 of neural networks 10 corresponds to the ensemble 1 described in conjunction with Figures 1 and 2. In particular, each neural network 10 of the ensemble 1 processes exclusively control data from a predetermined, non-overlapping subset 20 of all possible sensor data. For example, each neural network 10 of the ensemble 1 processes only sensor data that corresponds to an associated predetermined range of values ​​of the sensor data.

[0116] The control unit 101 includes, for example, a memory for storing the ensemble 1 of neural networks 10 and a processing unit for calculating the output of the ensemble 1 of neural networks 10 depending on the sensor data. The inference time of the ensemble 1 of neural networks 10, i.e., the time interval between the provision of sensor data and the output of control data by the ensemble 1, is at most 250 microseconds. This ensures, in particular, real-time control of the dynamic system 100, i.e., with virtually no delay. P2024,0761 WO N / INA - 2024P67177 WO 21 August 2025

[0117] 27

[0118] This patent application claims priority from German patent application DE 102024127762.9, the disclosure of which is hereby incorporated by reference. The invention is not limited to the description provided in the exemplary embodiments. Rather, the invention encompasses every new feature and every combination of features, which in particular includes every combination of features in the claims, even if that feature or combination itself is not explicitly stated in the claims or exemplary embodiments.

[0119] P2024, 0761 WO N / INA - 2024P67177 WO 21. August 2025

[0120] Reference sign

[0121] 1 Ensemble

[0122] 10 neural network 11 neuron

[0123] 12 Input layer

[0124] 13 Output layer

[0125] 14 hidden layers

[0126] 100 dynamic system 101 control unit

[0127] 2 Input room

[0128] 20 subset

[0129] 21 Transition

[0130] 22 Discontinuity 3 global neural network

[0131] 4 Sensor

[0132] 5 Control element xi , X2 Input value y Output value

Claims

P2024,0761 WO N / INA - 2024P67177 WO August 21, 2025 Patent claims 1. Method for training an ensemble (1) of neural networks (10) for real-time control of a dynamic system (100) , wherein: - the ensemble (1) of neural networks (10) is trained with training data from an input space (2), - each of the neural networks (10) of the ensemble (1) is trained exclusively with training data from an associated subset (20) of the input space (2), - the various neural networks (10) of the ensemble (1) belonging subsets (20) are disjoint, and - the training of the ensemble (1) neural networks (10) is subject to at least one constraint, according to which the output of the ensemble (1) neural networks (10) is continuous at at least one transition (21) between two adjacent subsets (20) of the input space (2).

2. Method according to the preceding claim, wherein: - first, at least a global neural network (3) is trained with training data from the entire input space (2), and - when training the ensemble (1) of neural networks (10) a deviation between an output of at least one global neural network (3) and the output of the neural networks (10) of the ensemble (1) is minimized.

3. Method according to one of the preceding claims, wherein the output comprises at least one logit of an output layer of the respective neural network (3, 10).

4. Method according to any of the preceding claims, wherein the training of the ensemble (1) of neural networks (10) under P2024,0761 WO N / INA - 2024P67177 WO August 21, 2025 30 at least one further constraint applies, according to which at least a partial derivative of the output of the ensemble (1) neural networks (10) is continuous at the at least one transition (21) between two adjacent subsets (20) of the input space (2).

5. Method according to one of the preceding claims, wherein the training of the ensemble (1) of neural networks (10) is carried out under at least one further constraint, according to which an expectation value of a Frobenius norm of a Jacobi matrix and / or at least a Hessian matrix of the output of the ensemble (1) of neural networks (10) is continuous at the at least one transition (21) between two adjacent subsets (20) of the input space (2).

6. Method according to one of the preceding claims, wherein the at least one constraint in the training of the ensemble (1) of neural networks (10) is realized by adding at least one penalty term to a training objective function to be minimized.

7. Method according to one of the preceding claims, wherein at least one constraint is taken into account when training the ensemble (1) of neural networks (10) by an augmented Lagrange method.

8. Method according to one of the preceding claims, wherein the subsets (20) correspond to directly adjacent, connected subvolumes of the input space (2).

9. Method according to the preceding claim, wherein the respective partial volume of a corresponding subset (20) of P2024,0761 WO N / INA - 2024P67177 WO August 21, 2025 - 31 - depends on a local complexity measure of the ensemble (1) of neural networks (10) .

10. Dynamic system (100) , exhibiting: - a control unit (101) which is set up for real-time control of the dynamic system (100) depending on sensor data, wherein - the real-time control of the dynamic system (100) is carried out with an ensemble (1) of neural networks (10) whose input space (2) corresponds to the possible sensor data, - each of the neural networks (10) of the ensemble (1) processes exclusively data from an associated subset (20) of the input space (2), - the various neural networks (10) of the ensemble (1) belonging subsets (20) are disjoint, and - the training of the ensemble (1) neural networks (10) is subject to at least one constraint, according to which the output of the ensemble (1) neural networks (10) is continuous at at least one transition (21) between two adjacent subsets (20) of the input space (2).

11. Dynamic system (100) according to the preceding claim, wherein - the dynamic system (100) is a robot or part of a robot, and - the control unit (101) controls a movement of the robot or a movement of part of a robot.

12. Dynamic system (100) according to one of claims 10 or 11, wherein the inference time of the ensemble (1) of neural networks (10) is at most 250 microseconds. P2024,0761 WO N / INA - 2024P67177 WO August 21, 2025 - 32 - 13. Dynamic system (100) according to one of claims 10 to 12, wherein the dynamic system (100) comprises a motor and / or a magnetic levitation system.

14. Dynamic system (100) according to any one of claims 10 to 10. 13, wherein the ensemble (1) of neural networks (10) is set up to model at least one control system of the dynamic system (100) and / or to provide feedforward control of the dynamic system (100).

Images