Computer-implemented method for training a neural network using MTL
Patent Information
- Application Number
- DE102023208154
- Authority / Receiving Office
- DE · DE
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-08-25
- Publication Date
- 2025-02-27
Smart Images
Figure 00000000_0000_ABST
Abstract
Description
[0001] The invention relates to training a neural network that processes multiple tasks simultaneously. The neural network is trained for all its tasks at the same time, a process known as multi-task learning (MTL). The invention lies particularly in the field of MTL.
[0002] MTL aims to learn multiple tasks simultaneously, leveraging their shared underlying characteristics. Unlike traditional single-task learning, where a model performs only one task, MTL enables the sharing of information between different tasks.
[0003] The fundamental idea behind MTL is that tasks in an MTL model are selected in such a way that they can benefit from each other. By training multiple tasks simultaneously, the model can learn a more general representation that goes beyond the specific requirements of a single task. By capturing common features between the tasks, the model can achieve better generalization and improve performance in each individual task.
[0004] Selecting the tasks to be learned is a crucial step in MTL. The selected tasks should share a common feature space or be able to benefit from each other in order to achieve the desired synergy effect.
[0005] An MTL model therefore typically comprises one or more shared layers and separate task-specific layers for each task. The shared layer(s) capture the common features, while the task layers learn specific features for each task.
[0006] During training, inputs flow through the network, and weights are adjusted according to the loss functions of each task. The shared layer(s) enable information exchange between tasks, resulting in improved generalization and learning performance.
[0007] The weighting of individual tasks can be adjusted according to their importance and difficulty. By setting different weightings, certain tasks can be prioritized, while others are considered secondary tasks.
[0008] Since MTL tends to produce overfitted models, the sharing of knowledge between tasks can be considered a form of regularization. The shared layer(s) promote the formation of features that are consistent across different tasks, thus helping to reduce overfitting.
[0009] Overfitting occurs when a model learns the training data very well but struggles to generalize to new, unknown data. Overfitting is particularly likely to happen when a model "memorizes" the training data too closely, losing sight of the underlying patterns and relationships. This can occur especially when the model becomes too complex during the training phase and begins to overlearn random variations and noise in the training data, leading to an overly tight fit to the training data and poor generalization to new data. The features or patterns the model learns during training may then be too specific to the training data and not representative of the entire dataset.
[0010] The advantage of MTL over single-task learning is that it can increase learning efficiency, especially in scenarios with limited data for individual tasks. By sharing information between tasks, MTL models can also capture more robust features and improve performance in individual tasks.
[0011] To train a model with MTL, a loss function is determined for each task, which is then combined into an overall loss function.
[0012] Various approaches for determining the weighting of individual loss functions in multi-task learning are known from the literature. For example, asynchronous backpropagation (AB) with weighted tasks leads to an improvement on the validation data.
[0013] A heuristic is used to determine which tasks are updated every time unit (e.g., epoch) and which tasks are updated only every xth time unit. For example, task 1 could be updated every epoch, task 2 every third epoch, and task 3 every tenth epoch. The idea behind this approach is that some tasks are easy, requiring less frequent weight updates through backpropagation. This allows the network to focus on training the more difficult tasks.
[0014] According to known approaches, the epochs in which tasks are updated must be calculated manually. However, this is not scalable, for example, if many tasks need to be trained.
[0015] The technical problem to be solved by the invention is therefore to find a training method for neural networks with multiple tasks that automatically determines the weighting factors.
[0016] The problem is solved by the subject matter of the independent claims. Disclosure of the invention
[0017] According to a first aspect of the invention, this problem is solved by a computer-implemented method for training a neural network, wherein the network performs several tasks and is trained to solve the tasks, with the following steps: - Collecting data as input values for the network; - Defining the network architecture encompassing multiple subnetworks, with each subnetwork taking on a specific task; - Defining a loss function for each task; - Determining an overall loss function that combines the loss functions of the individual tasks; - Determining an optimization method for the total loss function; - Training the network, where training involves minimizing the overall loss function, wherein the minimization of the total loss function is performed according to the optimization method; wherein the total loss function comprises a trainable weight factor and a regularization term for each loss function, wherein the sum of the weight factors equals 1, wherein the regularization term is minimal for a given weight factor, and wherein the regularization term is larger for weight factors larger than the given weight factor, and wherein the regularization term is larger for weight factors smaller than the given weight factor.
[0018] As a first step, data must be collected for training. This data can be gathered using exemplary processes, created using other methods and simulations, or artificially generated. The collected data forms a complete dataset, which is then divided into a training dataset and a validation dataset.
[0019] The training dataset is used to train the network. This means the training dataset is propagated through the network until the total loss function reaches its minimum. The validation dataset is used to test the network after training. Only with data the network hasn't already seen can it be verified whether the network has correctly adapted to solving the given tasks.
[0020] Before the network can be trained, a network architecture must first be defined. Multi-task networks have several subnetworks for different tasks, with each subnetwork handling one or more tasks.
[0021] The tasks can, for example, be subtasks of a network's overall task. If a network's overall task is to classify an object in a video stream, one subnetwork can be designed to check whether any objects are present in the video stream at all. A second subnetwork can then locate the detected objects, and a third network selects the object of greatest interest from among those detected. A fourth subnetwork can then ultimately classify the object. Furthermore, this embodiment can include additional subnetworks that handle image processing tasks such as color correction, sharpening filters, or contrast adjustments.
[0022] The architecture of a neural network comprises multiple nodes, or neurons, arranged in layers. Subnetworks can share one or more layers to solve their tasks using the same input values. Subsequently, each subnetwork can have its own layers dedicated solely to solving its specific task.
[0023] The architecture of the subnetworks can then be, in particular, a horizontal, vertical, or combined architecture.
[0024] In a horizontal architecture, the subnetworks are arranged in parallel. While the subnetworks access the same parent layer, they process the output of the parent layer independently. The outputs of the subnetworks can then be combined to solve the task of the entire network.
[0025] Vertical architectures comprise subnetworks arranged sequentially. This means that the output values of one network become the input values of the next subnetwork. The final subnetwork in this architecture then performs the task of the entire network.
[0026] Combined architectures combine vertical and horizontal arrangements. This can be illustrated with an example. Consider a network with five subnetworks. The first three networks are arranged vertically, so the second network receives the outputs of the first network as inputs, and the third network receives the outputs of the second network as inputs. A fourth network is parallel to networks one through three, so it uses the same input data as the first network. Networks one through three and network four are therefore arranged horizontally. A fifth network can, in turn, receive the outputs of the third and fourth networks as inputs, so the fifth network is arranged vertically relative to networks one through four.The output values of the fifth network are the last output values and thus represent the solution to the problem of the entire network.
[0027] In this way, subnetworks can be arranged in any architecture. In particular, the architecture of the subnetworks can be adapted to the tasks the network is intended to solve. In one embodiment, the layers of different subnetworks can be interconnected to exchange information between the subnetworks.
[0028] For each task, that is, for each subnetwork, a loss function is defined. The loss function, also called the cost function, is a function used to quantify the deviation or error between a model's predicted values and the actual values of the target variables. It captures the error or discrepancy between the model's predictions and the actual data points. It is a mathematical function that uses the model's input variables to calculate the error value. The larger the error value, the greater the difference between the model's predictions and the actual values.
[0029] The choice of loss function depends on the type of problem to be solved. There are different types of loss functions, each designed for different scenarios and applications.
[0030] The mean squared error (MSE) is frequently used in regression problems. It calculates the average squared error between the predicted and the actual values.
[0031] Binary cross-entropy is used in binary classification problems. It quantifies the error between predictions and the actual binary labels.
[0032] Categorical cross-entropy is used in multiclass classification problems. It measures the error between predictions and actual class labels.
[0033] The log-loss function is frequently used in probabilistic models where predictions are interpreted as probabilities. It calculates the logarithmic error between the probabilities and the actual values.
[0034] The goal during training is to minimize the loss functions of the different tasks. This is typically achieved by adjusting the model's weights and parameters using optimization algorithms such as gradient descent. By minimizing the loss functions, the model can learn to make more accurate predictions and achieve a better fit to the given data.
[0035] Choosing the appropriate loss function is crucial, as it directly influences the nature of each task and the model's outputs. For the network to accomplish its overarching task, the loss functions must be combined into a single overall loss function. Typically, weighting factors are used for this purpose, assigning a weight to each task that reflects its relevance to the network's overall goal. For example, important tasks might have a higher weight than unimportant ones. Alternatively, more difficult tasks might have a higher weight than easier ones, where difficulty and ease can be measured by the number of epochs a subnetwork requires to complete its task satisfactorily.
[0036] According to the invention, the weighting factor for each task is treated as a trainable parameter. The weighting factor can take values between 0 and 1. However, the sum of all weighting factors must equal 1. This ensures that the network does not over-weight any of the tasks.
[0037] The trainable weight factor allows the network to automatically adjust during training to assign a specific weight to each task. In particular, the trainability of the weight factor enables the network to scale with all its tasks. The weight factors adjust themselves automatically during network training, eliminating the need for manual weight allocation. Thus, the invention solves the underlying problem.
[0038] However, it could happen that a network sets the weight factor for the task with the lowest loss function to 1, rendering all other tasks with a weight factor of 0 irrelevant. To prevent this, a regulatory term is provided that becomes very large for the extreme values of the weight factor. This means that in an overall loss function, the large regulatory terms would counteract the particularly low or particularly high weight factors. This prevents one or a few weight factors from being set to 1 or close to 1 while all other weight factors are set to 0.
[0039] Such a regulatory term, or part of a regulatory term, could include, for example: R1(αt)=1−αtlog(αt). R1 becomes infinite for limα→01αt and for limα→01−logαt and thus fulfills the condition of the invention, wherein α tthe weighting factor of the respective task.
[0040] Another example of a regulatory term, or part of a regulatory term, could include a squared deviation: R2(αt)=(αt−1T)2.
[0041] Here, T is the number of tasks in the network, such that with equally distributed weighting factors αt=1T The regulation term R2 is zero and increases for other values the further they deviate from a uniform distribution. In one embodiment, the determined weighting factor is equal to the reciprocal of the number of tasks.
[0042] In one embodiment, the regulatory term can comprise several components, in particular the terms R1 and R2.
[0043] In one embodiment, the regulation term can include a hyperparameter that controls the degree of regulation. In particular, the hyperparameter can have the same value for all tasks.
[0044] Hyperparameters are parameters that control the configuration and behavior of a model, but are not directly learned from the training data. Unlike a model's weights and parameters, which are optimized during training, hyperparameters are set before the training process and influence how the model learns and generalizes.
[0045] Hyperparameters are crucial for the performance and behavior of a learning algorithm and can control various aspects of the model and the training process.
[0046] In one embodiment, the total loss function L can be defined as: L=∑tαtσtlt+μR(αt) with σ t l t as the weighted loss function of the individual tasks and µ as the hyperparameter of the regularization term.
[0047] In one embodiment, the loss functions of the individual tasks are different loss functions.
[0048] This allows the loss function to be advantageously adapted to different tasks.
[0049] In one embodiment, the data are multidimensional data from an imaging sensor. Imaging sensors can be, in particular, LiDAR / LADAR or radar sensors, cameras for the visual, infrared and / or ultraviolet spectrum, ultrasonic or motion sensors, or a combination of the aforementioned sensors.
[0050] Multidimensional data can exhibit higher levels of noise than one-dimensional data. Extracting the actual signal from beneath the noise can be divided into several tasks, each performed by its own subnetwork. Advantageously, any number of such tasks can be incorporated into a single network, as the weighting factors are trained along with the network itself.
[0051] In one embodiment, the network is a network for controlling an autonomous robot or vehicle.
[0052] Navigating autonomous systems in environments with limited control, especially those shared with humans, can place high demands on the controlling network. A network can better meet these demands by dividing navigation into numerous tasks handled by highly specialized subnetworks. In principle, entirely separate networks could also be used to perform the multitude of tasks. However, autonomous navigation must be performed by the vehicle or robot using onboard resources, so duplicating tasks would tie up valuable computing resources. Preparatory tasks, such as image processing, can be shared among the subnetworks. The subsequent navigation tasks, including any necessary system control, can be processed vertically, horizontally, or in a combination of both.
[0053] In one embodiment, the network is a network for directing vehicle traffic.
[0054] A neural network trained with MTL for urban traffic control offers several advantages compared to traditional training methods. Since traffic control encompasses various aspects, such as traffic prediction, vehicle detection, and traffic light control, MTL can efficiently utilize available data and enable a more comprehensive understanding of urban traffic dynamics. By learning multiple tasks simultaneously, the model can develop a more general understanding of the underlying features relevant to traffic control. This improves the model's ability to generalize to new, unknown traffic situations, as it gains broader knowledge of different aspects of traffic. Training tasks together allows the model to benefit from synergistic effects. For example, understanding traffic prediction can help control traffic light sequences more effectively.These synergistic effects can lead to improved overall model performance.
[0055] Furthermore, in many practical scenarios, access to sufficiently labeled data for each individual task can be limited. MTL can help overcome this bottleneck by sharing information between tasks, thereby reducing the need for large amounts of specific training data for each task. This allows the model to learn even with limited datasets and still safely manage traffic flow.
[0056] In one embodiment, the network is a network for controlling a household appliance.
[0057] When training a neural network with MTL to control a household appliance such as a food processor or robotic vacuum cleaner, several advantages arise compared to conventional training methods. Household appliances often have various functions and tasks that are interrelated and based on shared domain knowledge. MTL makes it possible to utilize this domain knowledge more effectively by sharing it between tasks. Through this shared training, the model can develop a more comprehensive understanding of the underlying domain and thus improve its performance in controlling the household appliance.
[0058] Household appliances often require complex tasks that combine various aspects, such as object recognition, motion control, sensor integration, etc. MTL enables the model to learn and integrate multiple aspects simultaneously. This leads to a more holistic approach to controlling household appliances, as the model combines various capabilities and tasks into a single model.
[0059] In one embodiment, the network is a speech processing network.
[0060] A neural network trained with MTL for language processing offers several advantages compared to traditional training methods. MTL allows for the simultaneous training of multiple language-related tasks by sharing common information and patterns between them. Since language processing encompasses various aspects, such as machine translation, speech recognition, and text classification, MTL can efficiently utilize available data and enable a more comprehensive understanding of language.
[0061] By training multiple language-related tasks together, the model can develop a more general understanding of the underlying linguistic features. Learning different linguistic aspects allows it to better generalize to diverse linguistic inputs and tasks. For example, understanding syntax in machine translation can also be beneficial in text classification. These synergistic effects can lead to improved model performance and robustness.
[0062] The aforementioned embodiments have in common that they are scalable through the use of automatically trained weight factors and that no weight factors need to be set manually or by means of a heuristic.
[0063] In one embodiment, the optimization method for the total loss function includes training the weight factors, wherein training the weight factors is a task of the network.
[0064] In these embodiments, the network includes a subnetwork whose task is to train the weight factors. By incorporating a subnetwork dedicated to training the weight factors, the weight factors can be determined more accurately. This subnetwork has only this task, allowing it to focus solely on it.
[0065] In one embodiment, the optimization method includes training the weighting factors using a dedicated neural network.
[0066] In one embodiment, the last layer transforms the initial values of the penultimate layer into a probability distribution to determine the weighting factors. A softmax function can be used for this purpose.
[0067] A softmax is a mathematical function used to generate probability distributions over multiple classes. It can be used as an activation function in the output layer of a classification network. The softmax function takes a vector of real numbers as input and transforms it into a vector of probabilities, where the sum of all probabilities equals 1. This is particularly useful for multi-class classification problems where the network needs to decide which class a given input pattern most likely belongs to.
[0068] The softmax function amplifies large values and attenuates small values. This emphasizes the differences between input values and adjusts the probabilities for the various classes. Larger input values have a larger exponential component and therefore contribute more to the overall result, leading to higher probabilities. The softmax function has the advantage of making the neural network's output interpretable. The resulting probabilities can be used as confidence levels for the different classes, which is helpful in decision-making or evaluating the network's output.
[0069] In one embodiment, the optimization method includes deriving the weighting factors from factors of the network or the output of the network.
[0070] For example, the quotient of the loss function of a single task and the loss function at a previous time unit or the evolution of weights of the network can be formed, which is then optimized when the optimization method is executed.
[0071] The total loss function L could then be defined for two consecutive epochs, for example, as: L=∑i=1KKe(Lit−1Lit−2 / T)∑j=1Ke(Lit−1Lit−2 / T)Li.
[0072] With such a definition, the total loss function can be determined dynamically and without a history, or with a very limited history.
[0073] Furthermore, the hyperparameter T can be taken into account in the softmax function by determining the weighting factor via: α=Softmax([w1,w2]T) with w1=L1t−1L1t−2 and w2=L2t−1L2t−2.
[0074] Advantageously, such a combination maintains the converging behavior of the overall loss function as well as the focus on difficult tasks during training.
[0075] In one embodiment, the network is trained in several epochs, with the influence of the weighting factors decreasing over the epochs.
[0076] A reduction in the influence of the weighting factors has the advantage that the network can concentrate more on the difficult tasks at the beginning, and towards the end of the training the tasks are trained together again on an "equal" basis.
[0077] For example, the total loss function can include a reduction term that reduces the influence of weight factors with increasing training periods.
[0078] In such an embodiment, the total loss function could then look like this: L=∑t(rαtσtlt+(1−r)σtlt)+rμR(αt)
[0079] The total loss function thus consists of two parts. The first term rα t σ t l t decreases over time. The second part (1 - r)σ t l t However, it increases over time. In the illustrated embodiment, the regulation term R is also provided with the reduction term, so that the regulation also decreases over time.
[0080] The decrease can, for example, be linear from 1 to 0 across all or some of the epochs. Alternatively, the decrease can be defined in steps. For example, r could be 1 in epochs 0 to 10, then r = 0.6 in epochs 20 to 100, ..., and r = 0 in epochs 250 and beyond. Another alternative is to let r decrease exponentially.
[0081] In another aspect, the invention relates to a computer program with program code to execute a method for training a neural network as described above when the computer program is executed on a computer.
[0082] In another aspect, the invention relates to a computer-readable data carrier containing the program code of a computer program for carrying out a method for training a neural network as described above when the computer program is executed on a computer.
[0083] In another aspect, the invention relates to a system for training a neural network according to one of the preceding claims, wherein the system is configured to perform a method as described above.
[0084] The described configurations and training programs can be combined in any way desired.
[0085] Further possible embodiments, developments and implementations of the invention also include combinations of features of the invention described previously or subsequently with regard to the exemplary embodiments that are not explicitly mentioned. Brief description of the drawings
[0086] The accompanying drawings are intended to provide a further understanding of the embodiments of the invention. They illustrate embodiments and, in conjunction with the description, serve to explain the principles and concepts of the invention.
[0087] Other embodiments and many of the aforementioned advantages become apparent with reference to the drawings. The elements depicted in the drawings are not necessarily shown to scale.
[0088] They show: Fig. 1. Schematically illustrates the process of training a neural network according to one embodiment; Fig. 2 a neural network with a vertical task structure; and Fig. 3 a neural network with a combined task structure.
[0089] In the figures of the drawings, identical reference symbols denote identical or functionally equivalent elements, parts or components, unless otherwise stated.
[0090] Fig. Figure 1 schematically shows the process for training a neural network with multiple tasks according to one embodiment.
[0091] In step S10, data is collected for training. Data collection can involve recording or creating data. In the next step, S12, the network architecture is defined. The network's tasks are identified, and subnetworks are arranged horizontally, vertically, or in a combination to suit these tasks.
[0092] In step S14, a loss function is defined for each task, which is used to determine the performance of the subnetworks. However, since the network can only be trained as a whole, an overall loss function is determined in step S16, which can be used to quantify and evaluate the work of the network as a whole.
[0093] In step S18, an optimization method is finally defined. For example, the optimization method can determine that the weighting factors decrease over time. Furthermore, it can be determined how strongly the regulation term should influence the overall loss function. For this purpose, the value of a corresponding hyperparameter is determined.
[0094] Finally, the network is trained in step S20, which involves an iterative process in which the overall loss function is minimized, and in step S22 the neural network is deployed.
[0095] Fig. Figure 2 shows an exemplary and schematic representation of the structure of a neural network 10. The network 10 receives two values 12 as input. In other embodiments, a network 10 can receive significantly more input values. For example, the input values 12 for a network 10 can include an image with two spatial dimensions and three color channels, which corresponds to the number of pixels in the image multiplied by three, with one value for each color channel. The networks 10 shown within the scope of this invention are intended only to illustrate the basic structure of the networks and not to limit the invention.
[0096] The input values 12 are processed by nodes 14, which are arranged in several layers 16b and 16c. Each layer is responsible for its own task. Since layers 16a, 16b, and 16c each receive the output values of the preceding layer as input values, layers 16a, 16b, and 16c are arranged vertically. The last layer, 16c, provides the output values 18, which are used to solve the overall task of the network 10.
[0097] Fig. Figure 3 also shows an exemplary and schematic structure of a network 10, which is more complex than the network 10 from Fig. 2.
[0098] The network 10 from Fig. 3 comprises 5 subnetworks. Each subnetwork can contain one or more layers with multiple neurons.
[0099] The input values 12 are fed into a first subnetwork 20. Subnetwork 20 performs general operations on the input values 12 and generates the intermediate values 22. These intermediate values 22 are used as input values for a second subnetwork 24, which in turn generates intermediate values 26. These intermediate values 26 are used as input values for a third subnetwork 28, which in turn generates intermediate values 30.
[0100] Parallel to the operations of the second subnetwork 24 and the third subnetwork 28, a fourth subnetwork 32 operates. Like the second subnetwork 24, the fourth subnetwork receives the intermediate values 22 from the first subnetwork 20 as input values. The fourth subnetwork 32 generates intermediate values 34.
[0101] The fourth subnetwork 32 operates in parallel to the second subnetwork 24 and the third subnetwork 28. Thus, a horizontal network structure exists between the second subnetwork 24 in conjunction with the third subnetwork 28 and the fourth subnetwork 32.
[0102] A fifth subnetwork 36 receives as input the intermediate values 30 from the third subnetwork 28 and the intermediate values 34 from the fourth subnetwork 32. From these input values, the fifth subnetwork 36 generates the output values 38, which solve the task of the entire network 10. The subnetworks 20, 24, 28, and 32, on the other hand, only solve subtasks individually, which are combined with the fifth subnetwork 36 to form a complete solution.
[0103] The overall loss function would be composed of the loss functions of subnetworks 20, 24, 28, 32, 36, each of which would be multiplied by a weighting factor.
Claims
[1] Computer-implemented method for training a neural network (10), wherein the network (10) performs several tasks and is trained to solve the tasks, with the following steps: - collecting data (S10) as input values (12) for the network (10); - defining the network architecture (S12) comprising a plurality of sub-networks (20, 24, 28, 32, 36), each sub-network (20, 24, 28, 32, 36) performing a task; - Defining a loss function (S14) for each task; - Determining an overall loss function (S16) that summarizes the loss functions of the individual tasks; - Determining an optimization method (S18) for the overall loss function; - training the network (S20, 10), wherein the training comprises minimizing the total loss function, wherein the minimization of the total loss function is carried out according to the optimization method; - providing the neural network (S22, 10); wherein the overall loss function comprises a trainable weight factor and a regularization term for each loss function, where the sum of the weight factors is 1, where the regularization term is minimal for a given weight factor, and where the regularization term is larger for weight factors larger than the given weight factor, and where the regularization term is larger for weight factors smaller than the given weight factor. [2] The computer-implemented method of claim 1, wherein the data is multidimensional imaging sensor data. [3] A computer-implemented method according to any one of the preceding claims, wherein the network (10) is a network (10) for controlling an autonomous robot or vehicle. [4] A computer-implemented method according to any one of claims 1 or 2, wherein the network (10) is a network (10) for routing vehicle traffic. [5] Computer-implemented method according to one of claims 1 or 2, wherein the network (10) is a network (10) for controlling a household appliance. [6] The computer-implemented method of claim 1, wherein the network (10) is a network (10) for processing speech. [7] Computer-implemented method according to one of the preceding claims, wherein the optimization method for the overall loss function comprises training the weight factors, wherein the training of the weight factors is a task of the network (10). [8] Computer-implemented method according to one of the preceding claims, wherein the optimization method comprises training the weight factors by means of a stand-alone neural network. [9] Computer-implemented method according to one of claims 7 or 8, wherein the last layer (16c) converts the output values of the penultimate layer into a probability distribution to determine the weighting factors. [10] Computer-implemented method according to one of the preceding claims, wherein the optimization method comprises deriving the weight factors from factors of the network or the output of the network (10). [11] Computer-implemented method according to one of the preceding claims, wherein the training of the network (10) takes place in several epochs, the influence of the weight factors decreasing over the epochs. [12] A computer program comprising program code for carrying out a method for training a neural network (10) according to any one of the preceding claims when the computer program is executed on a computer. [13] Computer-readable data carrier with program code of a computer program for carrying out a method for training a neural network (10) according to one of claims 1 to 11 when the computer program is executed on a computer. [14] System for training a neural network (10) according to one of the preceding claims, wherein the system is designed to carry out a method according to one of claims 1 to 11.