Method and apparatus for determining a network configuration of a neural network
By using Pareto set and probability distribution to select network configuration variants, the time-consuming problem of finding optimized neural network configuration in complex tasks is solved, and faster network configuration selection and neural network optimization are achieved.
Patent Information
- Application Number
- CN201980027847.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2018-04-24
- Filing Date
- 2019-04-17
- Publication Date
- 2025-03-18
- Estimated Expiration
- 2039-04-17
AI Technical Summary
In complex tasks, the process of finding the right neural network configuration to optimize prediction errors and other optimization goals is time-consuming, especially when the number of possible network configurations is extremely large and requires training to evaluate their working capabilities.
By providing the current set of network configurations corresponding to the Pareto set for prediction errors and other optimization goals, the probability distribution is used to select the network configuration variant, and the network configuration set is updated according to the prediction errors and other optimization goals to iteratively approach the Pareto frontier.
This method significantly reduces the selection time of network configuration, improves the discovery efficiency of suitable network configurations, and enables neural networks to find optimized configurations more quickly in specific applications.
Smart Images
Figure CN112055863B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to neural networks, in particular neural networks for implementing functions of technical systems, in particular robots, vehicles, tools or working machines. The invention also relates to an architecture search of a neural network in order to find a configuration of the neural network that is optimized with respect to prediction errors and with respect to one or more optimization objectives for a specific application. Background Art
[0002] The properties of a neural network are largely determined by its architecture. The architecture of a neural network is defined, for example, by its network configuration, which is given, inter alia, by the number of neuron layers, the type of neuron layer (linear transformation, nonlinear transformation, normalization, connection to other neuron layers, etc.), and the like. In particular, as the complexity of the application or the task to be solved increases, it is expensive to randomly find a suitable network configuration, since each candidate for a network configuration must first be trained in order to be able to assess its working ability.
[0003] To improve the finding of suitable network configurations, expert knowledge is often applied to reduce the number of candidates for possible network configurations before training them. In this way, a search can be conducted among a subset of reasonable network architectures.
[0004] Despite this approach, the number of possible network configurations is still very large. Since the evaluation of the network configuration is only determined after training, for example by analyzing error values, this leads to a very long search time for a suitable network configuration in the case of more complex tasks and correspondingly complex network configurations.
[0005] Furthermore, for most practical applications a neural network is required which is optimized not only with respect to the prediction error but also with respect to at least one further optimization objective which results, for example, from hardware limitations and the like. Summary of the invention
[0006] According to the invention, a method for determining a network configuration of a neural network based on training data for a given application is provided as claimed in claim 1 and a corresponding device as claimed in the independent claim.
[0007] Further refinements are described in the dependent claims.
[0008] According to a first aspect, a method is provided for determining a network configuration of a neural network that is suitable for a predetermined application, the application being in particular for implementing a function of a technical system, in particular a robot, a vehicle, a tool or a working machine, wherein the application is determined in the form of training data; wherein the network configuration describes the architecture of the neural network, the method comprising the following steps:
[0009] a) providing a current network configuration set having network configurations, wherein the current network configuration set corresponds to a Pareto set with respect to prediction error and at least one other optimization objective;
[0010] b) providing a set of network configuration variants according to the probability distribution of these network configurations in the current set of network configurations;
[0011] c) selecting a network configuration from a set of network configuration variants according to a probability distribution of the network configurations of the current set of network configurations with respect to the at least one other optimization objective;
[0012] d) determining a prediction error for each of the selected network configurations;
[0013] e) updating the current set of network configurations according to the prediction error of the selected network configuration and other optimization objectives; and
[0014] f) Select an appropriate network configuration from the current network configuration set.
[0015] The purpose of the above method is to find a set of network configurations with improved selection possibilities; to find a network configuration of a neural network that is optimized with respect to a predetermined application. In particular, it is desirable to select the possible network configurations so that they are optimized with respect to prediction error and one or more other optimization objectives, so that a suitable network configuration can be selected from the found set of network configurations based on judgment. The core of the above method is to determine an optimized Pareto set of suitable network configurations with respect to prediction error and one or more other optimization objectives in a way that saves as much resources as possible.
[0016] The Pareto set of network configurations corresponds to those network configurations in which the associated neural network is not dominated by other network configurations with respect to all optimization objectives. In other words, the Pareto set of network configurations corresponds to those network configurations that are selected so that each of these network configurations is better than each of the remaining network configurations at least with respect to the prediction error or with respect to at least one of the one or more other optimization objectives. In the present invention, in particular, the Pareto set with respect to the prediction error and with respect to at least one other optimization objective should be determined in order to obtain a reduced selection set of network configurations.
[0017] The above method enables: selecting a possible network configuration that is optimized with respect to the prediction error and with respect to one or more other optimization objectives, in order to find a network configuration suitable for the neural network for a given application. Since the determination of the prediction error requires corresponding training of the neural network according to the network configuration candidate data, the above method also provides: selecting the network configuration candidates for the training in a suitable manner according to the preselection. Here, the selection is made according to a probability distribution, so that the network configuration is preferred (präferieren) with respect to the cost value of one or more optimization objectives, for which no network configuration has been evaluated or a better network configuration has not been evaluated with respect to the prediction accuracy. In this way, network configuration variants along a Pareto set are obtained, which provide optimized selection possibilities with respect to the specific application. Here, the expert can make a corresponding selection of the suitable network configuration according to the weight of the optimization objectives.
[0018] Furthermore, steps a) to e) may be performed multiple times in an iterative manner.
[0019] In particular, the method can be terminated when an interruption condition is met, wherein the interruption condition includes the occurrence of at least one of the following events:
[0020] - the predetermined number of iterations has been reached;
[0021] A predetermined prediction error value caused by at least one of the network configuration variants is reached.
[0022] It may be provided that from the set of network configuration variants those network configurations are selected which have the lowest probability according to the probability distribution of the network configurations of the current set of network configurations.
[0023] For example, a network configuration may be selected from the set of network configuration variants based on a density estimate, in particular a kernel density estimate, determined from the current set of network configurations.
[0024] According to one embodiment, training data can be predefined by input variable vectors and output variable vectors assigned to these input variable vectors, wherein the prediction error of the corresponding network configuration is determined as a measure that results from the corresponding deviation between model values obtained by the neural network determined by the corresponding network configuration based on the input variable vectors and the output variable vector assigned to the input variable vector.
[0025] The prediction error of the selected network configuration can also be determined by training with training data under common predetermined training conditions, wherein these common predetermined training conditions predetermine the number of training runs and / or the training time and / or the training method.
[0026] A suitable network configuration may also be selected from the current set of network configurations based on a total cost function that depends on the prediction error and resource costs with respect to at least one optimization objective.
[0027] According to one embodiment, updating the current set of network configurations may be performed such that the updated set of network configurations contains only those network configurations from the current set of network configurations and from the selected network configurations that are better than any of the remaining network configurations with respect to prediction error and at least one of the one or more optimization objectives.
[0028] The updating of the current network configuration set may also be performed in the following manner: adding the selected network configuration to the current network configuration set to obtain an extended network configuration set, and then removing from the extended network configuration set those network configurations that are worse than at least one of the remaining network configurations in terms of prediction error and all of the one or more optimization objectives to obtain an updated network configuration set.
[0029] According to a further aspect, a method is provided for providing a neural network having a network configuration which is determined using the above method, wherein the neural network is designed in particular to implement a function of a technical system, in particular a robot, a vehicle, a tool or a working machine.
[0030] According to a further aspect, the use of a neural network having a network configuration for a predetermined application is provided, the network configuration being created for the predetermined application using the above-described method, wherein the neural network is designed in particular to implement a function of a technical system, in particular a robot, a vehicle, a tool or a working machine.
[0031] According to a further aspect, a device is provided for determining a network configuration of a neural network that is suitable for a predetermined application, in particular a function of a technical system, in particular a robot, a vehicle, a tool or a working machine, wherein the application is determined in the form of training data; wherein the network configuration describes the architecture of the neural network, wherein the device is designed to perform the following steps:
[0032] a) providing a current network configuration set having network configurations, wherein the current network configuration set corresponds to a Pareto set with respect to prediction error and at least one other optimization objective;
[0033] b) provide a collection of network configuration variants;
[0034] c) selecting a network configuration from a set of network configuration variants according to a probability distribution of the network configurations of the current set of network configurations with respect to the at least one other optimization objective;
[0035] d) training a neural network for each of the selected network configurations and determining a corresponding prediction error;
[0036] e) updating the current network configuration set according to the prediction error of the network configuration set and at least one other optimization objective and the selected network configuration; and
[0037] f) Select an appropriate network configuration from the current network configuration set.
[0038] According to a further aspect, a control device, in particular for controlling a function of a technical system, in particular a robot, a vehicle, a tool or a working machine, is provided, having a neural network, which is configured by means of the above-described method. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] The embodiments are further described below with reference to the accompanying drawings.
[0040] Figure 1 A schematic diagram of an exemplary neural network is shown;
[0041] Figure 2 Possible network configurations of neural networks are shown;
[0042] Figure 3 A flow chart for illustrating a method for determining a Pareto-optimal set of network configurations in order to determine suitable network configuration candidates for a predetermined application is shown;
[0043] Figure 4A schematic diagram of the Pareto front of a network configuration as a function of the prediction error and other optimization parameters, in particular resource usage parameters, is shown. DETAILED DESCRIPTION
[0044] Figure 1 Basically, the structure of a neural network 1 is shown, which usually has a plurality of cascaded neuron layers 2, each of which has a plurality of neurons 3. The neuron layer 2 has an input layer 2E for applying input data, a plurality of intermediate layers 2Z, and an output layer 2A for outputting calculation results.
[0045] Here, the neuron 3 of the neuron layer 2 may correspond to a conventional neuron function
[0046] ,
[0047] Among them, j The neuron output corresponding to the neuron, Corresponding to the activation function, x i Corresponding to the corresponding input value of the neuron, w i,j corresponds to the weight parameter of the i-th neuron input in the j-th neuron layer, and θ j Corresponding to the activation threshold. The weight parameters, activation threshold and the choice of activation function can be stored as neuron parameters in the neuron's registers.
[0048] The neuron outputs of neurons 3 can each be forwarded as neuron inputs to other neuron layers, ie neurons 3 of one of the following or preceding neuron layers 2 , or output as calculation results if they are neurons 3 of output layer 2A.
[0049] The neural network 1 formed in this way can be implemented as software or with the aid of computing hardware which maps a part or the entire neural network into an electronic (integrated) circuit. Usually, such computing hardware is selected for the construction of a neural network if the calculations are to be carried out very quickly, which is not possible with a software implementation.
[0050] The structure of the corresponding software or hardware is predetermined by the network configuration, which is determined by a number of configuration parameters. The network configuration determines the calculation rules of the neural network. Figure 1 In the case of a conventional network configuration schematically shown in , the configuration parameters include: the number of neuron layers; the corresponding number of neurons in each neuron layer; network parameters, which are given by weights, activation thresholds and activation functions; specifications about the coupling of neurons to input and output neurons; and the like.
[0051] In addition to the network configuration described above, other configurations of the neural network are possible, in which neurons are provided which are coupled on the input side to neurons from different neuron layers and on the output side to neurons from different neuron layers. In addition, in particular neuron layers can also be provided which are provided with feedback, that is to say which are coupled on the input side to neuron layers provided on the output side of the relevant neuron layer with respect to the data flow. In this regard, Figure 2 A possible configuration of a neural network with a plurality of layers L1 to L6 is schematically shown, which are firstly described in a manner known per se, as in Figure 1 , that is to say the neuron input is connected to the neuron output of the previous neuron layer. Neuron layer L3 also has a region which is coupled on the input side to the neuron output of neuron layer L5. A neuron layer L4 can also be provided which is connected on the input side to the output of neuron layer L2.
[0052] In the following, a method for determining an optimized network configuration for a neural network based on a predetermined application is to be described. The application is determined essentially by the size of input variable vectors and output variable vectors assigned to them, which are training data that define the desired network properties or a specific task.
[0053] exist Figure 3 A method for determining a set of suitable network configurations for a neural network with respect to a desired application is described in . The set obtained in this way should be used to facilitate the selection of a network configuration for the desired application. Thus, the network configuration should describe a neural network that can be used and is suitable for a specific application and is optimized with respect to the prediction error and additionally with respect to at least one other optimization objective. In particular, the set of network configurations should correspond to a Pareto front or a Pareto set of network configurations that are optimized with respect to the prediction error and with respect to at least one other optimization objective.
[0054] The purpose of the present invention is to approximate the function
[0055]
[0056] Pareto front of . Here, N is a neural network, error(N) is the prediction error of the neural network with respect to the validation data describing the application, and f(N) is an arbitrary N-dimensional function that describes the required resources of the neural network N in the form of the cost of the corresponding resources (resource cost), that is, the resource cost with respect to one or more optimization objectives additional to the prediction error. These additional optimization objectives may relate to the characteristics of resources for computing hardware, etc., such as: memory size, analysis speed, compatibility with respect to dedicated hardware, analysis energy consumption and the like. The method takes into account that the analysis of the prediction error is very expensive, because this requires the training of the neural networks of these network configurations. The cost of analyzing f(N) with respect to at least one additional optimization objective is significantly lower, because training of the neural network N is not required for this.
[0057] Then combine Figure 3 The method described below approximates the Pareto front of the above function in an iterative manner.
[0058] In step S1, a network configuration set is provided, which is a pre-given initial Pareto front P of the network configuration. 1 , that is, the current set of network configurations. These network configurations correspond to neural networks with specific network architectures. The relevant neural networks may include conventional neural networks, convolutional neural networks, or any other networks with learning capabilities, such as recursive neural networks.
[0059] In step S2, starting from the current set of network configurations, other / new network configurations, that is to say network configuration variants, can be determined together. These network configuration variants can be determined, for example, by applying different network morphisms to one or more of the network configuration variants or can be randomly selected. In principle, the network configuration variants can be generated in a substantially arbitrary manner.
[0060] A network morphism corresponds to predetermined rules that can be determined with the help of operators. In general, a network morphism is an operator T that maps a neural network N to a network TN, where:
[0061] in ,
[0062] Here, w is the network parameter (weight) of the neural network N, and are the network parameters of the neural network TN that are changed. X corresponds to the space on which the neural network is applied.
[0063] By changing the network configuration of the current network configuration set in step S2, k network configuration variants are obtained. These network configuration variants can also be generated in other ways, in particular, also independently of the respective current network configuration set.
[0064] In step S3, a subset of j network configurations is selected from the number k network configuration variants. The selection may be based on a density estimation, in particular a kernel density estimation (Kernel Density Estimator) p kde According to the current Pareto frontier P i against Alternative density estimation methods include parametric density models (such as Gaussian mixture models) or histograms.
[0065] Kernel density estimation is a statistical method for estimating the probability distribution of a random variable. Here, kernel density estimation corresponds to the function p kde , using this function to describe a probability measure of the occurrence of a specific network configuration based on a probability distribution, which is determined by the network configuration of the current network configuration set.
[0066] Next, the subset of network configurations is selected based on the p kde The inversely proportional probability distribution p is randomly selected, that is, the probability that the neural network N belongs to the k network configurations corresponds to:
[0067]
[0068] Where c is just a constant used to normalize the probability distribution. Instead of the above relationship, other relationships can also be used, which satisfy the following conditions:
[0069] .
[0070] If the kernel density estimate The kernel density estimate of is large for the selected network configuration variant among the network configuration variants N* (this is the case when there are already multiple network configurations in the current set of network configurations with approximately the same value of f(N)), then the associated network configuration variant N* will probably not improve this value further. If the kernel density estimate of is small for the selected network configuration variant among the network configuration variants N* (this is the case when few network configurations of the current network configuration set have approximately the same value of f(N)), then the probability that the relevant network configuration variant N* improves this value is large. In other words, the following network configurations are selected from these network configuration variants, which have a higher probability of belonging to the Pareto front of the current network configuration set or improving the approximation.
[0071] The analysis of f(N) can be performed very simply and computationally cheaply, so that the respectively current set of network configuration variants can be selected to be large. The choice of the network configurations to be selected therefrom determines the computational effort to a large extent, since these network configurations must be trained in order to determine the corresponding prediction errors.
[0072] In the following step S4 , the selected network configuration is trained with the same training data under predetermined training conditions and the corresponding prediction error is determined.
[0073] When training a neural network that is predetermined by means of network configuration variants, the same evaluation measure should be obtained. The neural network of all network configurations is therefore trained for a predetermined number of training cycles and a predetermined training algorithm.
[0074] In step S5, an update of the Pareto front is performed according to the determined prediction error and the resource cost with respect to one or more other optimization objectives. i An update is performed so that the selected network configuration is added to the current set of network configurations to obtain an extended set of network configurations, and then those network configurations are removed from the extended set of network configurations that are worse than at least one of the remaining network configurations in terms of prediction error and all of the one or more optimization objectives.
[0075] In the subsequent step S6, it is checked whether the interruption condition is met. If so (either one: yes), the method continues with step S7, otherwise (either one: no) the method jumps back to step S2. The interruption condition may include:
[0076] - the predetermined number of iterations has been reached;
[0077] A predetermined prediction error value caused by at least one of the network configuration variants is reached.
[0078] In this way, a respectively current set of network configurations suitable for the relevant application can be iteratively brought close to the Pareto front of the optimized network configurations.
[0079] exist Figure 4 , for example, shows the change process of the Pareto front of the set of resource costs with respect to the network configuration with respect to the prediction error error(N) and with respect to at least one other optimization target f(N).
[0080] Now, the network configuration of the current network configuration set determined after the last iteration cycle is the basis for selecting a network configuration suitable for the application determined by the training data. This can be achieved, for example, by pre-setting a total cost function, which takes into account the prediction error and resource costs. In practice, it will be determined based on the relevant application which network configuration in the current network configuration set (current Pareto set) is most suitable for the selected application. This can be achieved based on restrictive pre-regulations. As an example, a network configuration that does not exceed a network size of, for example, 1 GB of memory can be selected from the Pareto frontier.
[0081] The above-described method makes it possible to speed up the architecture search for a network configuration in an improved manner, since the evaluation of the operating capabilities / prediction errors of network configuration variants can be performed significantly more quickly.
[0082] The network configuration determined in this way can be used to select a suitable configuration of the neural network for a pre-given task. The optimization of the network configuration is closely related to the proposed task. The task is obtained by pre-determining the training data, so that before the actual training, the training data must first be defined, and the optimized / suitable network configuration for the proposed task is determined based on these training data. For example, an image recognition or classification method can be defined by the following training data, which contains input images, object allocation, and object classification. In principle, in this way, the network configuration can be determined for all tasks defined by the training data.
[0083] Thus, a neural network configured in this way can be used in a control device of a technical system, in particular in a robot, a vehicle, a tool or a working machine, in order to determine an output variable as a function of an input variable. The output variable can include, for example, a classification of the input variable (that is, for example, an assignment of the input variable to one of a plurality of predeterminable categories) and, if the input data include image data, a semantic segmentation of these image data, in particular pixel-by-pixel (that is, for example, an assignment of a segment of the image data to one of a plurality of predeterminable categories, region-by-region or pixel-by-pixel). In particular, sensor data or variables determined from sensor data can be considered as input variables of the neural network. The sensor data can come from sensors of the technical system or be received from the outside by the technical system. The sensor can include, in particular, at least one video sensor and / or at least one radar sensor and / or at least one LiDAR sensor and / or at least one ultrasonic sensor. Based on the output variable of the neural network, at least one actuator of the technical system can be controlled by a computing unit of the control device of the technical system using a control signal. For example, this can control the movement of a robot or a vehicle or control a drive unit or a driver assistance system of the vehicle.
Claims
1. A computer-implemented method for determining a suitable network configuration for computing hardware and for a neural network for a predetermined application, wherein the predetermined application is determined in the form of training data; wherein the network configuration describes the architecture of the neural network, the method comprising the following steps: a) providing a current set of network configurations having network configurations, wherein the current set of network configurations corresponds to a Pareto set in terms of prediction error and at least one other optimization objective, wherein the other optimization objective characterizes a resource cost of the network configurations on computing hardware, wherein the resource cost is memory size, analysis speed, compatibility with respect to dedicated hardware or analysis energy consumption; b) providing a set of network configuration variants according to a change in the network configuration of the current network configuration set; c) selecting a subset of network configurations from the provided set of network configuration variants according to a probability distribution (p), wherein the probability distribution (p) characterizes the distribution of the current set of network configurations with respect to the at least one other optimization objective; d) training the neural network of each network configuration in the selected network configuration and determining the corresponding prediction error error(N); e) updating the current network configuration set according to the prediction error of the network configuration set and the at least one other optimization objective and the selected network configuration; and f) selecting a suitable network configuration from the current set of network configurations, wherein the suitable network configuration is selected from the current set of network configurations based on a total cost function, wherein the total cost function depends on the prediction error and the resource cost. 2 . The method according to claim 1 , wherein the predetermined application is used to implement a function of a technical system.
3. The method according to claim 2, wherein the technical system is a robot, a vehicle, a tool or a working machine. The method according to claim 1 , wherein the current network configuration set is a Pareto set.
5. The method according to any one of claims 1 to 4, wherein the subset of network configurations is selected from a set of network configuration variants according to a probability distribution (p), where the probability distribution (p) and the density estimate (p kde ), The density estimate (p kde ) is calculated based on the current network configuration set. The method according to claim 5 , wherein the current network configuration set is a Pareto set.
7. The method according to claim 5, wherein the density estimate (p kde ) characterizes the density of the current set of network configurations with respect to at least one other optimization objective.
8. The method according to any one of claims 1 to 4, wherein steps a) to e) are performed multiple times iteratively, When an interruption condition is met, the method is terminated, wherein the interruption condition includes the occurrence of at least one of the following events: - the predetermined number of iterations has been reached; - a predetermined value of the prediction error error(N) caused by at least one of the network configuration variants is reached.
9. The method according to any one of claims 1 to 4, wherein those network configurations are selected from the set of network configuration variants, which have the lowest probability according to the probability distribution of network configurations of the current set of network configurations.
10. The method of claim 9, wherein a network configuration is selected from a set of network configuration variants based on a density estimate determined from the current set of configurations. The method of claim 10 , wherein the density estimate is a kernel density estimate.
12. A method according to any one of claims 1 to 4, wherein the training data are predefined by an input variable vector and an output variable vector assigned to the input variable vector, wherein a prediction error of a corresponding network configuration is determined as a measure resulting from a corresponding deviation between a model value obtained by a neural network determined by the corresponding network configuration based on the input variable vector and an output variable vector assigned to the input variable vector.
13. A method according to any one of claims 1 to 4, wherein the prediction error of the selected network configuration is determined by training with the training data under common predetermined training conditions, wherein the training conditions predetermine the number of training processes and / or the training time and / or the training method.
14. A method according to any one of claims 1 to 4, wherein the updating of the current set of network configurations is performed so that the updated set of network configurations contains only those network configurations from the current set of network configurations and from the selected network configurations, and those network configurations are better than any of the remaining network configurations with respect to the prediction error and at least one of the one or more optimization objectives.
15. A method according to any one of claims 1 to 4, wherein the updating of the current set of network configurations is performed in the following manner: adding the selected network configuration to the current set of network configurations to obtain an extended set of network configurations, and then removing from the extended set of network configurations those network configurations that are worse than at least one of the remaining network configurations in terms of the prediction error and all of the one or more optimization objectives to obtain an updated set of network configurations.
16. Method for providing a neural network having a network configuration, the network configuration being created using a method according to any one of claims 1 to 15, wherein the neural network is designed for controlling a technical system, wherein the network configuration is used.
17. The method according to claim 16, wherein the technical system is a robot, a vehicle, a tool or a working machine.
18. Use of a neural network for controlling a technical system, wherein the network configuration of the neural network is created using a method according to any one of claims 1 to 15.
19. The use according to claim 18, wherein the technical system is a robot, a vehicle, a tool or a working machine.
20. A device for determining a suitable network configuration of a neural network for computing hardware and for a predetermined application, wherein the application is determined in the form of training data; wherein the network configuration describes the architecture of the neural network, wherein the device is designed to perform the following steps: a) providing a current set of network configurations having network configurations, wherein the current set of network configurations corresponds to a Pareto set in terms of prediction error and at least one other optimization objective, wherein the other optimization objective characterizes a resource cost of the network configurations on computing hardware, wherein the resource cost is memory size, analysis speed, compatibility with respect to dedicated hardware or analysis energy consumption; b) provide a collection of network configuration variants; c) selecting a network configuration from a set of network configuration variants according to a probability distribution of the network configurations of the current set of network configurations with respect to the at least one other optimization objective; d) training the neural network of each network configuration in the selected network configuration and determining the corresponding prediction error error(N); e) updating the current network configuration set according to the prediction error of the network configuration set and the at least one other optimization objective and the selected network configuration; and f) selecting a suitable network configuration from the current set of network configurations, wherein the suitable network configuration is selected from the current set of network configurations based on a total cost function, wherein the total cost function depends on the prediction error and the resource cost.
21. The device according to claim 20, wherein the predetermined application is used to implement a function of a technical system.
22. The device according to claim 21, wherein the technical system is a robot, a vehicle, a tool or a working machine. 23 . A control device having a neural network configured by means of a method according to claim 1 .
24. The control device according to claim 23, wherein the control device is used to control a function of a technical system. 25 . The control device according to claim 24 , wherein the technical system is a robot, a vehicle, a tool or a working machine.
26. Computer program product comprising a computer program, the computer program comprising instructions, the instructions being set up such that, when the instructions are executed on a computer, the computer carries out the method according to any one of claims 1 to 15.
27. An electronic storage medium having a computer program stored thereon, the computer program comprising instructions which are configured such that, when the instructions are executed on a computer, the computer carries out the method according to any one of claims 1 to 15.