Optimization of mixture expert model (MOE) incorporation into neural network architectures

Abstract

To incorporate a mixed expert model function in a neural network.SOLUTION: The method comprises a step 110 of constructing a candidate version 1A, 1B, of the neural network 1 in which one or more layers have been replaced by a surrogate layer, a step 120 of training the respective candidate version of the neural network using the training examples 2a of the sensor-data 2, using a test sample 5b and / or validation sample 2b of sensor-data for which respective ground truth outputs 2b of the neural network are known, to: Determining 130 the trained candidate version of the neural network 1a *, the accuracy 6 with which 1B * reproduces the ground truth output, and determining 140 the layer replaced by the surrogate layer 1a ' - 1d ' in the candidate version of the neural network with the best accuracy as the optimal layer 1 * for incorporating the mixture of experts model (MoE) functionality.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a neural network that includes a Mixture of Experts (MoE) function in at least one layer and in which processing blocks are selected for use depending on the input. [Background technology]

[0002] background When processing sensor data using a neural network, these sensor data may relate to many different situations. It may be difficult to build and train one single monolithic neural network architecture suitable for all situations. In particular, the monolithic architecture may be very large, so using such a monolithic architecture during inference is computationally expensive.

[0003] This is the purpose of Mixed Expert Model (MoE) technology, which allows deep learning models to have a larger number of parameters without sacrificing inference time. The MoE layer contains a router block, also known as an "expert network," and a collection of individual processing blocks. When an input arrives at the MoE layer, the router block determines which processing block will process this input. In this way, only the processing blocks appropriate for this input are activated. A forward path through each individual processing block will be significantly faster than a forward path through one monolithic network that contains the functionality of all processing blocks. Summary of the Invention [Problem to be solved by the invention]

[0004] In the prior art, there are applications where MoE functions are incorporated into all layers of a neural network, but there are also applications where MoE layers are incorporated only in certain parts of the network. [Means for solving the problem]

[0005] Disclosure of the Invention The present invention provides a method for determining one or more optimal layers for incorporating a model of experts (MoE) function into a neural network that includes N layers and is configured for classification and / or regression of sensor data. The MoE function includes a plurality of M distinct processing blocks and a router block that routes each input to one or more processing blocks for processing. That is, when an input arrives, the router block determines which processing blocks are suitable for processing the input, and the input is processed by these determined processing blocks.

[0006] In particular, the sensor data may include any kind of image (such as still, video, thermal, radar, lidar, or ultrasound images). Classification may involve assigning a classification score for one or more classes to the image based on low-level attributes of the image, such as pixels, voxels, or other components of the image, or based on basic features, such as edges. In particular, the classes may relate to the presence of particular objects, such as people, vehicles, obstacles, traffic signs, or other traffic-related objects, in the image representing a traffic scene.

[0007] The sensor data, in particular the output of a neural network that has been modified by the method, may be used, for example, to automatically determine the operating state of a technical system or to operate the technical system. For example, the sensor data and / or the output may be used to classify the operating state (in a simple example, into normal and abnormal) or to derive a quantity of interest that is related to the operation of the technical system but is not directly measured. That is, in particular, the neural network may be configured to derive from the sensor data a quantity of interest that has a dimensionality different from that of the sensor data. However, in other examples, the output may be a denoised version of the sensor data and therefore may have the same dimensionality as the sensor data.

[0008] The operation of the technical system may be performed in any suitable manner, for example, an operating signal may be calculated based on the output of a neural network and provided as an input to the technical system, for example, the operating signal may be applied to an actuator that directly changes the physical behavior of the technical system, or may be applied as a setpoint to a controller of the technical system configured to maintain a particular characteristic of the technical system at or near a setpoint.

[0009] Examples of technical systems in which the output of a neural network that has been modified by the method according to the invention can be applied as described above when sensor data is supplied to the neural network include vehicles, vehicle assistance systems, robots, quality inspection systems, monitoring systems, medical imaging systems, but also industrial plants that carry out industrial processes.

[0010] In the course of the method, candidate versions of the neural network are constructed. These candidate versions are modified neural network architectures derived from the original architecture with N layers. In each candidate version, one or more layers are replaced by surrogate layers. These surrogate layers can be said to mimic the incorporation of MoE functionality into each layer, with the aim of evaluating whether it is indeed advantageous to incorporate MoE functionality into this layer rather than another layer. Thus, each surrogate layer: It is chosen to be computationally less expensive to train than a full MoE layer, while at the same time The surrogate layer is selected so that the performance of the candidate neural network with the surrogate layer corresponds to the performance that the neural network would have if it had an MoE layer instead of a surrogate layer. That is, when the surrogate layer is present, the performance of the candidate neural network approximates the performance of a neural network with an MoE feature instead of a surrogate layer. Moreover, measuring this performance is much faster than measuring the performance of a neural network with an MoE feature instead of a surrogate layer.

[0011] As a result, each candidate version of the neural network is trained using training examples of sensor data. The training examples may be labeled with the corresponding ground truth to be reproduced by the neural network, or may be unlabeled. In particular, the set of training examples may include a mixture of labeled and unlabeled training examples.

[0012] Then, using test and / or validation samples of sensor data where the respective ground truth outputs of the neural network are known, the accuracy with which the trained candidate version of the neural network reproduces the ground truth outputs is determined according to any suitable metric. For example, the metric may measure how well the ground truth outputs are reproduced on average. However, alternatively or in combination, the metric may measure how well the ground truth outputs are reproduced in the best and / or worst cases in the set of training examples.

[0013] The layer replaced by a surrogate layer in the candidate version of the neural network with the best accuracy is determined as the optimal layer for incorporating the MoE function. In a simple example, the neural network includes five layers. Five candidate versions of the neural network are established with exactly one layer replaced by a surrogate layer. That is, the first candidate version has the first layer replaced by a surrogate layer, the second candidate version has the second layer replaced by a surrogate layer, and so on. After training, these five candidate versions achieve test accuracies of 0.8, 0.82, 0.9, 0.88, and 0.85, respectively. In this case, the third candidate version, with the third layer replaced by a surrogate layer, achieves the best accuracy of 0.9, followed by the fourth candidate version, with the fourth layer replaced by a surrogate layer and achieving the second-best accuracy of 0.88. This indicates that incorporating the MoE function into the third and / or fourth layers of the neural network is most advantageous.

[0014] Figuring out the best location for the MoE function in this way is advantageous because the straightforward approach of placing the MoE function in each and every layer comes with a price: it increases the complexity of the neural network architecture, making training such a neural network architecture more expensive. In particular, in addition to the processing blocks having to learn how to process the inputs assigned to them, the router block also has to learn how to make a good selection of one or more processing blocks for each input. Therefore, training a candidate model with an MoE layer N times is not cheap. Training the MoE requires more computational resources. Furthermore, extensive tuning of the MoE hyperparameters is required to stabilize the training. The proposed method allows this additional effort to be concentrated on layers that offer a reasonable return on this complexity and computational investment. In this case, the complete training of the neural network with the MoE function only needs to be performed once, after the exact location of the MoE function has been determined using the results of the method according to the present invention.

[0015] Figuratively speaking, the rapid determination of optimal layers for incorporating MoE functions corresponds to geological surveys using seismic probes to detect where oil or other sought-after products reside; the optimal layers correspond to optimal locations for mining or drilling; and training a neural network with incorporated MoE functions into these optimal layers corresponds to the actual mining or drilling to ultimately obtain the sought-after product.

[0016] In this way, applications that were previously considered too cumbersome and / or costly can be freed from using MoE functions, thereby enriching these applications with the known benefits of using MoE functions. In particular, each of the fairly small individual processing blocks requires relatively few training examples, and therefore the total number of training examples required to train a neural network is less than the number required to train a monolithic neural network toward comparable performance. Also, because only one or a few of the many processing blocks are active at a time during inference, fewer processing resources and power consumption are required. That is, the hardware platform only needs to have the hardware resources required to run a few processing blocks rather than a large monolithic network. Particularly in embedded applications, such as evaluating sensor data from monitoring the vehicle environment, the available hardware resources and power onboard the vehicle are limited.

[0017] In the technical applications presented above, the main benefit of the method is that it allows MoE to be used in places where it was previously virtually impossible, resulting in a better accuracy of the output of the neural network, which means that the probability that the output and the resulting actions taken on or by the technical system are appropriate given the situation represented by the sensor data is improved.

[0018] In a particularly advantageous embodiment, a surrogate layer is selected so that the performance of a candidate neural network with a surrogate layer represents an upper bound on the performance the neural network would have if it had an MoE layer instead of a surrogate layer. This upper bound on performance represents an ideal case in which all processing blocks ("experts") meet to discuss the optimal output. In a real-world case in which only one or a few processing blocks are active at the same time, it is impossible to have better performance. Figuratively speaking, instead of drawing a complex contour around the performance of a neural network with an MoE function in a particular layer, with many twists and turns along this contour that would be very complex to describe, only a single bounding box is drawn around this performance. To do this, only two corner points need to be defined.

[0019] In particular, the output generated from an input by at least one surrogate layer can be aggregated from the processing results generated from that input by multiple processing blocks, and the aggregation can use more results from more processing blocks than would be used during inference of a real network.

[0020] For example, aggregation may be performed by calculating the mean, median, maximum, or minimum of the results produced by multiple processing blocks, with which form of aggregation is most appropriate depending on the particular application under consideration.

[0021] In a further particularly advantageous embodiment, the output generated by at least one surrogate layer from an input is selected from the processing results generated by multiple processing blocks from that input. That is, only one such processing result is further used. For example, this processing result can be used for a more granular analysis, which indicates which processing block is most suitable for use. Also, the case where only one processing block "expert" is active at a time is much closer to reality during inference than the case where all available processing blocks "expert" are active, resulting in a more accurate estimate of accuracy.

[0022] For example, the processing result that is optimal with respect to a given criterion is selected as the output of at least one surrogate layer, which may include, for example, that the processing result has the highest confidence score or other score, or that the processing result has the lowest uncertainty.

[0023] In a further particularly advantageous embodiment, at least two candidate versions of the neural network are constructed, with different processing results from multiple processing blocks selected as outputs in the same surrogate layer. For example, if there are M different "expert" processing blocks in a first candidate version of the neural network, a particular layer can be replaced by a surrogate layer that uses the output of the first "expert" processing block. A second candidate version of the neural network can have a surrogate layer in the same location that can use the output of the second "expert" processing block, and so on. Thus, testing whether a layer is suitable for incorporating an MoE function can be decomposed into testing M candidate versions of the neural network, and at least M candidate versions of the neural network can be constructed to use the results from processing blocks 1,...,M as the output of a single, identical surrogate layer.

[0024] A candidate version of the neural network may very well include a combination of surrogate layers that use aggregated outputs from multiple processing blocks on the one hand and selected individual outputs from multiple processing blocks on the other hand, where one may be better in some places and the other may be better in other places to approximate the performance of the neural network with the MoE function.

[0025] In a further particularly advantageous embodiment, the MoE function is incorporated into one or more layers determined to be optimal. This generates an MoE-enabled neural network. This MoE-enabled neural network is then trained using training examples of sensor data. The training results in a trained MoE-enabled neural network. As described above, because the determination of the optimal layer into which the MoE function should be incorporated can now be based on an approximation rather than on a full MoE training, the final result, i.e., a trained neural network with the MoE function in the optimal location, can be obtained more quickly.

[0026] In a further particularly advantageous embodiment, training the MoE-enabled neural network includes training a router block to assign training examples to individual processing blocks corresponding to the different groups to which the training examples belong. In this way, each input is handled by the processing block "expert" that is best suited to that input. This is somewhat similar to the patient admission procedure in a hospital emergency room: first, a rough screening is performed by the "router block" to diagnose the type of illness the patient has. The patient is then transferred to the department that corresponds to this particular illness.

[0027] The different groups to which the inputs are assigned are e.g. - different types of objects present in an area monitored by at least one sensor generating sensor data; and / or Different types of disturbances present in the sensor data samples can be represented as:

[0028] For example, in a use case where a neural network is used to analyze sensor data from a vehicle's or robot's environment, a first group may represent traffic signs, a second group may represent other traffic participants, a third group may represent road markings, and a fourth group may represent other obstacles such as vegetation. If different processing blocks are suited to handle these types of objects, the processing can be modularized so that each processing block can be trained specifically for one particular group.

[0029] However, the distribution of inputs to groups, as well as the distribution to different types of objects, does not need to be human-understandable. For example, different types of disturbances (such as noise) present in samples of sensor data may not be perceptible by humans. It is sufficient that the router block can learn how to distinguish between these disturbances.

[0030] Once the MoE-enabled neural network has been trained, samples of sensor data can be provided to the MoE-enabled neural network. Operational signals can then be calculated from outputs generated by the trained MoE-enabled neural network from the sampled sensor data. A vehicle, a driver assistance system, a robot, a quality inspection system, a monitoring system, and / or a medical imaging system can then be operated by the operational signals. Due to the increased accuracy of the output of the trained MoE-enabled neural network, the probability that a reaction implemented by a technical system respectively operated in response to the operational signal will be appropriate in a situation characterized by the sensor data is thus improved.

[0031] The method may be wholly or partly computer-implemented and may be implemented in software. Accordingly, the present invention also relates to a computer program comprising machine-readable instructions that, when executed by one or more computers and / or computing instances, cause the one or more computers and / or computing instances to implement the above-described method. In this specification, control units for vehicles or robots and for other embedded systems capable of executing machine-readable instructions should also be considered as computers. Computing instances include virtual machines, containers, or other execution environments that enable the execution of machine-readable instructions in the cloud.

[0032] The non-transitory storage medium and / or downloadable product may include a computer program. A downloadable product is an electronic product that can be sold online and transferred over a network for immediate fulfillment. One or more computers and / or computing instances may be equipped with such computer program and / or such non-transitory storage medium and / or downloadable product.

[0033] The present invention will now be described with reference to the accompanying drawings, without any intention of limiting the scope of the present invention. [Brief explanation of the drawings]

[0034] [Figure 1] FIG. 1 illustrates an exemplary embodiment of a method 100 for determining one or more optimal layers 1* for incorporating a Model of Mixture of Experts (MoE) function into a given neural network 1. [Figure 2] FIG. 10 is a diagram showing an estimation of the performance of the MoE layer using substitute layers 1a' to 1d'. [Figure 3] FIG. 10 shows the MoE-enabled neural network 1# finally obtained. DETAILED DESCRIPTION OF THE INVENTION

[0035] 1 is a schematic flow chart of one embodiment of a method 100 for determining one or more optimal layers 1* for incorporating a Model of Mixture of Experts (MoE) function into a neural network 1. The neural network 1 includes N layers 1a-1d and is configured for classification and / or regression of sensor data 2. The MoE function includes a plurality of M distinct processing blocks 4a-4d and a router block 3 that routes each input I to one or more processing blocks 4a-4d for processing.

[0036] In step 110, candidate versions 1A, 1B of neural network 1 are constructed. In each candidate version 1A, 1B, one or more layers (1a-1d) are replaced by surrogate layers 1a'-1d'. Each such surrogate layer 1a'-1d' is computationally less expensive to train than a respective MoE layer with full MoE functionality. At the same time, however, the performance of candidate neural network 1A, 1B with surrogate layers 1a'-1d' corresponds to the performance that neural network 1 would have if it had MoE layers instead of surrogate layers 1a'-1d'.

[0037] According to block 111, the surrogate layers 1a'-1d' can be selected so that the performance of the candidate neural network 1A, 1B with the surrogate layers 1a'-1d' is an upper bound on the performance that the neural network would have if it had MoE layers instead of surrogate layers.

[0038] According to the block 112, the output O generated from the input I by at least one of the proxy layers 1a' to 1d' can be aggregated from the processing results generated from the input I by the plurality of processing blocks 4a to 4d.

[0039] In particular, according to block 112a, aggregation may be performed by calculating the mean, median, maximum or minimum of the results produced by the multiple processing blocks 4a-4d.

[0040] According to the block 113, the output O generated from the input I by at least one of the proxy layers 1a' to 1d' can be selected from the processing results generated from the input I by the plurality of processing blocks 4a to 4d.

[0041] According to block 113a, the processing result that is optimal with respect to a given criterion can be selected as output O of at least one surrogate layer 1a'-1d'.

[0042] According to block 113b, at least two candidate versions 1A, 1B of neural network 1 can be constructed 113b, and in each such candidate version 1A, 1B, different processing results from multiple processing blocks 4a-4d can be selected as output O in the same surrogate layer 1a'-1d'. In particular, according to block 113c, at least M candidate versions 1A, 1B of neural network 1 can be constructed, and results from processing blocks 1,...,M, 4a-4c can be used as output O of a single and same surrogate layer 1a'-1d'.

[0043] In step 120, the training example 2a of the sensor data is used to train respective candidate versions 1A, 1B of the neural network 1. The trained states of each candidate version 1A, 1B are labeled by the reference symbols 1A*, 1B*, respectively.

[0044] In step 130, the test samples 2b and / or validation samples 2b of sensor data, for which the respective ground truth outputs 5b of the neural network 1 are known, can be used to determine the accuracy 6 with which the trained candidate versions 1A*, 1B* of the neural network 1 reproduce the ground truth outputs 5b. That is, the test samples 2b and / or validation samples 2b can be fed to the trained candidate versions 1A*, 1B*, and the outputs 5 produced by the trained candidate versions 1A*, 1B* can then be compared to the ground truth outputs 5b to assess the accuracy 6.

[0045] In step 140, the layers 1a-1d replaced by the surrogate layers in the candidate versions 1A, 1B of the neural network 1 with the best accuracy 6 can be determined as the optimal layer 1* for incorporating the MoE function. That is, thanks to the trained candidate versions 1A*, 1B* of the neural network 1 achieving good accuracy 6, the layers that are the surrogate layers 1a'-1d' in the trained candidate versions 1A*, 1B* are deemed to be the optimal layer 1* for incorporating the MoE function.

[0046] In the example shown in Figure 1, this determined optimization is implemented in step 150 by actually incorporating MoE functionality into one or more layers 1* of the determined optimal neural network 1. This extends the original neural network 1 into an MoE-enabled neural network 1#.

[0047] In step 160, the training example 2a of the sensor data is used to train the MoE-enabled neural network 1#, thereby generating a trained MoE-enabled neural network 1**.

[0048] According to block 161, training the MoE-enabled neural network 1# may include training the router block 3 to assign the training examples 2a to the respective processing blocks 4a-4d corresponding to the different groups to which the training examples belong. In particular, according to block 161a, such different groups may be - different types of objects present in an area monitored by at least one sensor generating sensor data (2), and / or Different types of disturbances present in the sensor data (2) samples can be represented as:

[0049] 1 , in step 170, a sample of sensor data 2 is provided to a trained MoE-enabled neural network 1**. An operation signal 180a is calculated from the output 5 generated by the trained MoE-enabled neural network 1** from the sample of sensor data 2. Then, in step 190, the vehicle 50, the driver assistance system 51, the robot 60, the quality inspection system 70, the monitoring system 80, and / or the medical imaging system 90 are operated by the operation signal 180a.

[0050] FIG. 2 shows how the use of surrogate layer 1a' can save time when evaluating whether incorporating MoE functionality into layer 1a is advantageous over incorporating this MoE functionality into other layers 1b-1d.

[0051] When MoE functions are incorporated into layer 1a, the input I to this layer 1a in the course of processing training samples 2a passes through router block 3 and then through any individual processing blocks (here, 4a and 4d) that are used depending on the output of router block 3. This means that the entire MoE processing chain inside the MoE layer needs to be trained, including the decision process made by router block 3. This training is quite complex, as symbolically indicated by the complex contour of the boundary around layer 1a.

[0052] The introduction of the surrogate layer 1a' abstracts away this complexity. In the example shown in FIG. 2, the performance of the surrogate layer 1a' is an upper bound on the performance of the layer 1a with the embedded MoE function. That is, the surrogate layer 1a' serves as a kind of "bounding box" for the performance of the layer 1a with the embedded MoE function. The use of a "bounding box" inevitably overestimates this performance, but the tradeoff is that training the candidate versions 1A, 1B of the neural network 1 with this surrogate layer 1a' and the subsequent accuracy measurement 6 on the test samples 2b and / or validation samples 2b becomes much faster. In particular, it is sufficient to consider only the outputs of the individual processing blocks 4a-4d. The complexity of bundling them together by the router block 3 can be ignored when only considering the upper bound on performance.

[0053] FIG. 3 shows version 1# of neural network 1 augmented with an MoE function in optimal layer 1* according to the method 100 of the present invention. In the example shown in FIG. 3, neural network 1 consists of four layers 1a-1d. The first layer 1a receives input sensor data 2. The output 5 (1a) of the first layer 1a is provided as input 2 (1b) to the second layer 1b. The output 5 (1b) of the second layer 1b is provided as input 2 (1c) to the third layer 1c. The output 5 (1c) of the third layer 1c is provided as input 2 (1d) to the fourth and final layer 1d. The output of the fourth and final layer 1d is the output 5 of the entire neural network 1.

[0054] In the example shown in FIG. 3, during the course of the method 100 described above, layer 1c was found to be the optimal layer 1* for incorporating the MoE function. As a result, the MoE function is now incorporated into this layer 1c. That is, the input I to this layer is now first processed by router block 3. Router block 3 determines to which of the individual processing blocks "experts" 4a-4c the input I should be fed. In the example shown in FIG. 3, these processing blocks are processing blocks 4a and 4c. These blocks 4a and 4c contribute to the output O of layer 1c.

Claims

1. A method (100) for determining one or more optimal layers (1*) for incorporating a Model of Expert Mixture (MoE) function in a neural network (1) including N layers (1a-1d) and configured for classification and / or regression of sensor data (2), the MoE function including a plurality of M distinct processing blocks (4a-4d) and a router block (3) that routes each input (I) to one or more of the processing blocks (4a-4d) for processing; The method (100) comprises: a step (110) of constructing candidate versions (1A, 1B) of said neural network (1), in each of which one or more layers (1a-1d) are replaced by surrogate layers (1a'-1d'); Each surrogate layer (1a'-1d') is computationally less expensive to train than a full MoE layer, while a step (110) in which the performance of the candidate neural network (1A, 1B) with the surrogate layers (1a'-1d') corresponds to the performance that the neural network (1) would have if it had the MoE layers instead of the surrogate layers (1a'-1d'); - training (120) each candidate version (1A, 1B) of said neural network (1) using a training example (2a) of sensor data; - determining (130) the accuracy (6) with which trained candidate versions (1A*, 1B*) of the neural network (1) reproduce the ground truth outputs (5b) using test samples (2b) and / or validation samples (2b) of sensor data for which the respective ground truth outputs (5b) of the neural network (1) are known; a step (140) of determining the layer (1a-1d) replaced by the surrogate layer in the candidate version (1A, 1B) of the neural network (1) with the best accuracy (6) as the optimal layer (1*) for incorporating the MoE function; A method (100) comprising:

2. The surrogate layers (1a'-1d') are selected (111) such that the performance of the candidate neural network (1A, 1B) with the surrogate layers (1a'-1d') is an upper bound on the performance that the neural network would have if it had the MoE layers instead of the surrogate layers. The method (100) of claim 1.

3. An output (O) generated from an input (I) by at least one proxy layer (1a'-1d') is aggregated (112) from processing results generated from the input (I) by a plurality of processing blocks (4a-4d).

3. The method (100) of claim 1 or 2.

4. The aggregation is performed by calculating the mean, median, maximum or minimum of the results produced by the plurality of processing blocks (4a-4d) (112a). The method (100) of claim 3.

5. An output (O) generated from an input (I) by at least one proxy layer (1a'-1d') is selected from processing results generated from the input (I) by a plurality of processing blocks (4a-4d) (113); The method (100) of any one of claims 1 to 4.

6. The processing result that is optimal with respect to a given criterion is selected (113a) as the output (O) of the at least one surrogate layer (1a'-1d'), The method (100) of claim 5.

7. At least two candidate versions (1A, 1B) of the neural network (1) are constructed, and different processing results from a plurality of processing blocks (4a-4d) are selected (113b) as outputs (O) in the same surrogate layer (1a'-1d').

7. The method (100) according to claim 5 or 6.

8. At least M candidate versions (1A, 1B) of the neural network (1) are constructed, and the results from processing blocks 1, ..., M (4a-4c) are used as the output (O) of a single and identical surrogate layer (1a'-1d') (113c); The method (100) of claim 7.

9. The method (100) comprises: Incorporating the MoE functionality into the layer or layers (1*) determined to be optimal, thereby obtaining a MoE-enabled neural network (1#) (150); training (160) said MoE-enabled neural network (1#) using the training examples of sensor data (2a), thereby obtaining a trained MoE-enabled neural network (1**); The method (100) of any one of claims 1 to 8, further comprising:

10. Training the MoE-enabled neural network (1#) includes training (161) the router block (3) to assign the training examples (2a) to individual processing blocks (4a-4d) corresponding to the different groups to which the training examples (2a) belong.

10. The method (100) of claim 9.

11. The different groups are: - different types of objects present in the area monitored by at least one sensor generating said sensor data (2), and / or Different types of disturbances present in the samples of sensor data (2) (161a) The method (100) of claim 10.

12. providing (170) samples of sensor data (2) to said trained MoE-enabled neural network (1**); - calculating (180) an operating signal (180a) from the output (5) generated by the trained MoE-enabled neural network (1**) from the sensor data samples (2); - operating (190) a vehicle (50), a driving assistance system (51), a robot (60), a quality inspection system (70), a monitoring system (80), and / or a medical imaging system (90) by said operating signal (180a); 12. The method of any one of claims 9 to 11, further comprising:

13. 13. A computer program comprising machine-readable instructions that, when executed by one or more computers and / or computing instances, cause the one or more computers and / or computing instances to perform the method (100) of any one of claims 1 to 12.

14. A non-transitory machine-readable storage medium and / or downloadable product comprising the computer program of claim 13.

15. One or more computers and / or computing instances comprising the computer program of claim 13 and / or comprising the non-transitory machine-readable storage medium and / or downloadable product of claim 14.

Images