Method for determining optimal architecture of artificial neural network

Determining the optimal architecture of artificial neural networks through directed graph representation and stream allocation methods solves the problems of high resource consumption and multi-criteria requirements, improves accuracy and saves resources.

CN118940811APending Publication Date: 2024-11-12ROBERT BOSCH GMBH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410575330.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2023-05-10
Filing Date
2024-05-10
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

The prior art has high resource consumption and is difficult to meet the needs of multiple criteria when determining the optimal architecture of artificial neural networks, especially when changing presets or considering other criteria, the architecture needs to be completely redetermined.

Method used

By providing a set of possible architectures for artificial neural networks, representing potential architectures using directed graphs, and determining the optimal architecture through the allocation and update of flows based on reward and cost functions until the abortion criteria are met, combined with a weighted summing method to accommodate multiple pre-pre- The best architecture is set.

Benefits of technology

Improves optimal architecture determination accuracy under multiple criteria, reduces resource consumption, especially memory and processor capacity requirements, and avoids resource-intensive re-adaptation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118940811A_ABST
    Figure CN118940811A_ABST
Patent Text Reader

Abstract

A method for determining an optimal architecture of an artificial neural network. The invention relates to a method for determining an optimal architecture of an artificial neural network, the method (1) having the following steps: for each of at least two presets for determining the architecture, determining the optimal architecture of the artificial neural network; respectively determining a corresponding preset optimal architecture by assigning a stream to each edge of the directed graph and respectively updating the streams assigned to the edges along the trajectory based on a cost function, with these steps being repeated until the determined trajectory satisfies an abortion criterion for the architecture search, and wherein the trajectory satisfying the abortion criterion represents the optimal architecture (4); and determining an optimal architecture (5) of the artificial neural network on the basis of the respective optimal architecture with respect to each of the at least two presets.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a method for determining an optimal architecture of an artificial neural network, and in particular to a method for determining an optimal architecture of an artificial neural network in view of multiple criteria. The method can save resources when determining the optimal architecture, and the method can also improve the accuracy when determining the optimal architecture. Background Art

[0002] Machine learning algorithms are based on the use of statistical methods to train data processing facilities so that they can perform specific tasks without having to be explicitly programmed for this purpose. The goal of machine learning is to construct algorithms that can learn from data and make predictions. These algorithms create mathematical models with which data can be classified, for example.

[0003] An example of such a machine learning algorithm is an artificial neural network. Such an artificial neural network is oriented to biological neurons and allows unknown system behaviors to be trained from existing training data and then the trained system behaviors are also applied to unknown input variables. Here, the neural network consists of layers with idealized neurons, which are interconnected in different ways depending on the topology of the network. The first layer, also referred to as the input layer, collects and transmits input values ​​here, wherein the number of neurons in the input layer corresponds to the number of input signals that should be processed. The last layer is also referred to as the output layer and has exactly the same number of neurons as the output values ​​that should be provided. In addition, at least one intermediate layer is located between the input layer and the output layer, which intermediate layer is often also referred to as a hidden layer, wherein the number of intermediate layers and the number and / or type of neurons in these layers are related to the specific task that should be solved by the neural network.

[0004] However, the development of the architecture of the artificial neural network, that is, the appearance of the network or the determination of the number of layers in the network and the determination of the number and / or type of neurons in the individual layers is usually very expensive, especially in terms of resource consumption. In order to optimize the development of the architecture, a neural architecture search (NAS) has been developed, which automatically develops the best architecture for a specific problem. The NAS algorithm first assembles an architecture for the artificial neural network from various modules and configurations, then trains the architecture with a set of training data, and then evaluates the results obtained in terms of performance. Based on this evaluation, a new architecture that is probably more optimized in terms of performance can then be determined, which is then trained again based on the training data, and the results obtained are then evaluated in terms of performance. These steps can be repeated so frequently here until no improvement can be achieved by changing the architecture, wherein a gradient-based method is usually used to determine a more optimized architecture.

[0005] In particular, the performance of an artificial neural network depends in particular on the selected architecture. In this context, however, it has proven disadvantageous that it is often difficult to determine the actual optimal architecture for the artificial neural network, wherein the determination of the optimal architecture is nevertheless often associated with a high expenditure of resources. If a plurality of preconditions or conditions or criteria are to be taken into account when determining the optimal architecture, then it is often necessary to completely redefine the optimal architecture if the preconditions change or if further criteria are to be taken into account.

[0006] A method for creating an artificial neural network is known from the publication DE 10 2019214625 A1. The method comprises providing a plurality of different data sets, initializing a plurality of hyperparameters, training the artificial neural network, evaluating the trained artificial neural network, optimizing the hyperparameters based on the evaluation, and retraining the artificial neural network using the optimized hyperparameters.

[0007] The object of the present invention is therefore to specify an improved method for determining an optimal architecture for an artificial neural network.

[0008] This object is achieved by a method for determining an optimal architecture of an artificial neural network according to the features of patent claim 1 .

[0009] This object is also achieved by a system for determining an optimal architecture of an artificial neural network according to the features of patent claim 8 . Summary of the invention

[0010] According to one embodiment of the present invention, the task is solved by a method for determining an optimal architecture of an artificial neural network, wherein the method comprises: providing a set of possible architectures of the artificial neural network; representing the set of possible architectures of the artificial neural network in a directed graph, wherein the nodes of the directed graph respectively symbolize a subset of one of the possible architectures, wherein the initial node symbolizes the input layer, wherein the end nodes of the directed graph respectively symbolize the subset with the output layer, and wherein the edges of the directed graph symbolize the possible links between the subsets; for each of at least two presets for determining the architecture, by respectively assigning a flow to each edge of the directed graph, based on the directed graph The graph specifies a strategy for determining a corresponding optimal architecture and determines the corresponding optimal architecture based on the specified strategy by repeatedly determining a trajectory from an initial node to an end node, determining a reward for the determined trajectory, determining a cost function for the determined trajectory based on the determined reward for the trajectory and the flow assigned to the edge along the trajectory, and updating the flow assigned to the edge along the trajectory based on the cost function to respectively determine the optimal architecture for the corresponding preset, wherein the steps of determining the trajectory, determining the reward, determining the cost function and updating the flow are repeated until the determined trajectory satisfies a termination criterion for architecture search, and wherein the trajectory that satisfies the termination criterion represents the optimal architecture. In addition, the method includes determining an optimal architecture of the artificial neural network based on the corresponding optimal architecture for each of at least two presets or the previously determined optimal architecture for all of the at least two presets, and providing the optimal architecture of the artificial neural network.

[0011] In this case, a set of possible architectures is understood to be a plurality of possible architectures of the artificial neural network or a corresponding search space.

[0012] A directed graph is also a graph having nodes and edges connecting the individual nodes, wherein the edges are directed edges, ie edges which can only be traversed in one direction.

[0013] Each node of the directed graph symbolizes a subset of one of the possible architectures, which means that each node symbolizes a subset of at least one of the possible architectures of the artificial neural network, wherein each node can symbolize a different subset, and wherein the subsets can be distributed to the nodes of the directed graph, so that all possible architectures of the artificial neural network are collectively contained or represented in the directed graph. The subsets here each include or characterize at least one layer of the corresponding possible architecture.

[0014] A strategy for determining an optimal architecture based on a directed graph is also understood to be a plan based on which or based on corresponding specifications or corresponding criteria, individual nodes of the directed graph are selected in order to obtain a trajectory.

[0015] In this context, in particular a continuous path between an initial node and one of the end nodes is referred to as a trajectory.

[0016] A reward or bonus is also understood to be a value of an improvement achievable by a corresponding architecture which is determinable by evaluating the architecture represented by the corresponding trajectory.

[0017] A cost function or loss is also understood to be the loss or error between the reward expected for a determined trajectory based on the flow assigned to the edges along the trajectory and the determined actual reward for the trajectory.

[0018] Under the termination criterion for the architecture search, a previously defined criterion is also specified, wherein the determination of the corresponding optimal architecture is terminated if the determined architecture or the architecture represented by the determined trajectory meets the termination criterion with respect to the corresponding predefined criteria or the corresponding criterion.

[0019] In this case, representing the architecture by the determined trajectory means that the architecture is formed by correspondingly linking the subsets symbolized by the nodes along the determined trajectory.

[0020] The method according to the invention differs from the known common methods for determining the optimal architecture of an artificial neural network in that the reward itself is not optimized, but the potential architectures are tested or queried based on the rewards assigned to these architectures, respectively. In addition, the method according to the invention differs from the known common methods for determining the optimal architecture of an artificial neural network in that, for example, the gradient is not estimated for determining the optimal architecture, but the flow or value of the distribution between the individual edges of the directed graph or the subset of possible architectures of the artificial neural network is optimized and adapted to the actual situation.

[0021] Rather than optimizing the reward itself, testing or querying potential architectures based on the rewards assigned to them in each case has the advantage that the accuracy in determining the best architecture and in particular the probability that the actually optimal architecture can be found can be increased.

[0022] The gradients are not estimated, but the flows or values ​​assigned to the individual edges of a directed graph or to a subset of possible architectures of an artificial neural network are optimized and adapted to the actual situation, which has the advantage, inter alia, that this is, for example, less susceptible to noise and that overall fewer iterations are required to determine the optimal architecture, thereby saving resources required for determining the optimal architecture, such as memory and / or processor capacity.

[0023] Furthermore, determining an optimal architecture of the artificial neural network based on a previously determined architecture, which is optimal in each case with respect to the presettings to be taken into account when determining the optimal presettings, has the advantage that the optimal presettings of the artificial neural network do not have to be completely re-determined in a complex and resource-intensive manner if the presettings are to be weighted differently or other criteria are to be taken into account.

[0024] In general, an improved method for determining an optimal architecture for an artificial neural network is thus described.

[0025] In one embodiment, the step of determining the optimal architecture of the artificial neural network based on the corresponding optimal architecture for each of at least two presets comprises summing the optimal architectures for the at least two presets in a weighted manner.

[0026] Summing the best architectures means here that the rewards respectively corresponding to the corresponding best architectures and the best flows respectively corresponding to the corresponding best architectures are summed, wherein the best architecture of the artificial neural network is determined based on the sum of the individual best flows.

[0027] Weighting is also understood to mean an assessment of the individual influencing variables of the mathematical model, for example in terms of their value. Summing in a weighted manner means that the individual addends, ie the individual optimal flows, are weighted based on their value or importance.

[0028] The optimal architecture of the artificial neural network can therefore be determined in a simple manner based on the respective optimal architecture with respect to the at least two predefined specifications, without complex and resource-intensive adaptation being necessary.

[0029] The corresponding weighting can be based on the current hardware situation of at least one target component.

[0030] The hardware conditions of the at least one target component are also understood to be further information about resources available to the at least one target component, in particular for the use of the artificial neural network, such as memory and / or processor capacity.

[0031] Therefore, when determining the optimal architecture of the artificial neural network, the conditions of the data processing facility to which the respectively trained artificial neural network is subsequently applied are taken into account.

[0032] In one specific embodiment, the reward for the trajectory is also determined based on the hardware conditions of the at least one target component.

[0033] When determining the optimal architecture of the artificial neural network, the conditions of the data processing system in which the correspondingly trained artificial neural network is subsequently used are therefore also taken into account.

[0034] Another embodiment of the present invention also describes a method for training an artificial neural network, wherein the method includes providing training data for training the artificial neural network; providing an optimal architecture for the artificial neural network, wherein the optimal architecture of the artificial neural network has been determined by the above-mentioned method for determining the optimal architecture of the artificial neural network; and training the artificial neural network based on the training data and the optimal architecture.

[0035] Therefore, a method for training an artificial neural network is described, which is based on an optimal architecture determined by an improved method for determining an optimal architecture for an artificial neural network. Instead of optimizing the reward itself, the potential architectures are tested or queried based on the rewards assigned to these architectures respectively, which has the advantage that the accuracy in determining the optimal architecture and in particular the probability that the actually optimal architecture can be found can be increased. The gradient is not estimated, but the flow or value of the distribution between the individual edges of the directed graph or the subset of possible architectures of the artificial neural network is optimized and adapted to the actual situation. This also has the advantage that this is, for example, less susceptible to noise and that fewer iterations are required overall to determine the optimal architecture, thereby saving resources required for determining the optimal architecture, such as memory and / or processor capacity. In addition, determining the optimal architecture of the artificial neural network based on a previously determined architecture, which is optimal in each case in view of the presets to be considered when determining the optimal preset, also has the advantage that if the presets are to be weighted differently or other criteria are to be considered, the optimal architecture of the artificial neural network does not have to be completely re-determined in a complex and resource-intensive manner.

[0036] The training data may include sensor data.

[0037] A sensor, also called detector, (measured variable or measurement) sensitive element or (measurement) detection element, is a technical component which can detect specific physical or chemical properties and / or material properties of its environment qualitatively or quantitatively as a measured variable.

[0038] It is thus possible in a simple manner to detect situations outside the actual data processing facility on which the method is carried out and to take them into account when training the artificial neural network.

[0039] Utilizing another embodiment of the present invention, a method for controlling a controllable system based on an artificial neural network is further described, wherein the method comprises providing an artificial neural network that is trained to control a controllable system, wherein the artificial neural network has been trained by the above-mentioned method for training an artificial neural network; and controlling the controllable system based on the provided artificial neural network.

[0040] In this case, the controllable system may be, in particular, a robotic system, wherein the robotic system may be, for example, an embedded system of a motor vehicle and / or a motor vehicle function.

[0041] A method for controlling a controllable system based on an artificial neural network is therefore described, wherein the artificial neural network is based on an optimal architecture determined by an improved method for determining an optimal architecture of an artificial neural network. The fact that potential architectures are tested or queried based on the rewards assigned to these architectures instead of optimizing the rewards themselves has the advantage that the accuracy in determining the optimal architecture and in particular the probability that the actually optimal architecture can be found can be increased. The fact that the flows or values ​​assigned to the individual edges of a directed graph or a subset of possible architectures of an artificial neural network are optimized and adapted to the actual situation instead of estimating the gradients has the advantage that this is, for example, less susceptible to noise and that fewer iterations are required overall to determine the optimal architecture, thereby saving resources required for determining the optimal architecture, such as memory and / or processor capacity. The determination of the optimal architecture of an artificial neural network based on a previously determined architecture, which is optimal in each case with respect to the presets to be considered when determining the optimal preset, has the advantage that the optimal preset of the artificial neural network does not have to be completely re-determined in a complex and resource-intensive manner if the presets are to be weighted differently or other criteria are to be considered.

[0042] Another embodiment of the present invention further describes a system for determining an optimal architecture of an artificial neural network, wherein the system comprises a first providing unit, which is configured to provide a set of possible architectures of the artificial neural network; a mapping unit, which is configured to map the set of possible architectures of the artificial neural network onto a directed graph, wherein the nodes of the directed graph respectively symbolize subsets of one of the possible architectures, wherein the initial node symbolizes the input layer, wherein the end nodes of the directed graph respectively symbolize subsets with output layers, and wherein the edges of the directed graph respectively symbolize possible links between the subsets; and at least one first determining unit, which is configured to map the set of possible architectures of the artificial neural network to a directed graph, wherein the nodes of the directed graph respectively symbolize subsets of one of the possible architectures, wherein the initial node symbolizes the input layer, wherein the end nodes of the directed graph respectively symbolize the subsets with the output layer, and wherein the edges of the directed graph respectively symbolize possible links between the subsets; and For each preset of the directed graph, a flow is assigned to each edge of the directed graph, a strategy for determining the corresponding optimal architecture is specified based on the directed graph, and the corresponding optimal architecture is determined by repeatedly determining a trajectory from an initial node to an end node based on the specified strategy, a reward for the determined trajectory is determined, a cost function for the determined trajectory is determined based on the determined reward for the trajectory and the flow assigned to the edge along the trajectory, and the flow assigned to the edge along the trajectory is respectively updated based on the cost function to respectively determine the optimal architecture for the corresponding preset, wherein the steps of determining the trajectory, determining the reward, determining the cost function and updating the flow are repeated until the determined trajectory satisfies a termination criterion for the architecture search, and wherein the trajectory that satisfies the termination criterion represents the optimal architecture. In addition, the system has: a second determination unit, which is configured to determine the optimal architecture of the artificial neural network based on the corresponding determined optimal architecture for each of the at least two presets; and a second providing unit, which is configured to provide the optimal architecture of the artificial neural network.

[0043] Therefore, an improved system for determining an optimal architecture for an artificial neural network is described. The fact that potential architectures are tested or queried based on the rewards assigned to these architectures instead of optimizing the rewards themselves has the advantage that the accuracy in determining the optimal architecture and in particular the probability that the optimal architecture can be found can be increased. The fact that the flows or values ​​assigned to the individual edges of a directed graph or a subset of possible architectures of an artificial neural network are optimized and adapted to the actual situation instead of estimating the gradients has the advantage that this is, for example, less susceptible to noise and that fewer iterations are required overall to determine the optimal architecture, thereby saving resources required for determining the optimal architecture, such as memory and / or processor capacity. Furthermore, determining the optimal architecture of an artificial neural network based on previously determined architectures, which are optimal in each case in view of the presets to be considered when determining the optimal presets, has the advantage that the optimal presets of the artificial neural network do not have to be completely re-determined in a laborious and resource-intensive manner if the presets are to be weighted differently or other criteria are to be considered.

[0044] In one embodiment, the second determination unit is designed to determine the optimal architecture of the artificial neural network by summing the respective optimal architectures for each of the at least two presets in a weighted manner. The optimal architecture of the artificial neural network can thus be determined in a simple manner based on the respective optimal architectures for the at least two presets without requiring complex and resource-intensive adaptation.

[0045] The respective weighting can be based on the current hardware conditions of the at least one target component. Thus, when determining the optimal architecture of the artificial neural network, the conditions of the data processing system in which the respectively trained artificial neural network is subsequently used are taken into account.

[0046] In one embodiment, the first determination unit is further designed to determine the reward for the trajectory based on the hardware conditions of the at least one target component. Thus, when determining the optimal architecture of the artificial neural network, the conditions of the data processing system in which the correspondingly trained artificial neural network is subsequently used are also taken into account.

[0047] According to another embodiment of the present invention, a system for training an artificial neural network is further described, wherein the system comprises a first providing unit configured to provide training data for training the artificial neural network; a second providing unit configured to provide an optimal architecture for the artificial neural network, wherein the optimal architecture has been determined by the above-mentioned system for determining an optimal architecture for an artificial neural network; and a training unit configured to train the artificial neural network based on the training data and the optimal architecture.

[0048] Therefore, a system for training an artificial neural network is described, which is based on an optimal architecture determined by an improved system for determining an optimal architecture for an artificial neural network. The fact that potential architectures are tested or queried based on the rewards assigned to these architectures instead of optimizing the rewards themselves has the advantage that the accuracy in determining the optimal architecture and in particular the probability that the actually optimal architecture can be found can be increased. The fact that the flows or values ​​assigned to the individual edges of a directed graph or to a subset of possible architectures of an artificial neural network are optimized and adapted to the actual situation instead of estimating the gradients has the advantage that this is, for example, less susceptible to noise and that fewer iterations are required overall to determine the optimal architecture, thereby saving resources required for determining the optimal architecture, such as memory and / or processor capacity. The determination of the optimal architecture of an artificial neural network based on previously determined architectures, which are each optimal in view of the preconditions to be considered when determining the optimal preconditions, has the advantage that the optimal preconditions of the artificial neural network do not have to be completely re-determined in a laborious and resource-intensive manner if the preconditions are to be weighted differently or other criteria are to be considered.

[0049] The training data can again comprise sensor data. This makes it possible to detect situations outside the actual data processing facility on which the method is carried out in a simple manner and to take them into account when training the artificial neural network.

[0050] Using another embodiment of the present invention, a system for controlling a controllable system based on an artificial neural network is also described, wherein the system has a providing unit, which is constructed to provide an artificial neural network, which is trained to control the controllable system, wherein the artificial neural network has been trained by the above-mentioned system for training an artificial neural network; and a control unit, which is constructed to control the controllable system based on the provided artificial neural network.

[0051] Therefore, a system for controlling a controllable system based on an artificial neural network is described, wherein the artificial neural network is based on an optimal architecture determined by an improved system for determining an optimal architecture for an artificial neural network. The fact that potential architectures are tested or queried based on the rewards assigned to these architectures instead of optimizing the rewards themselves has the advantage that the accuracy in determining the optimal architecture and in particular the probability that the actually optimal architecture can be found can be increased. The fact that the flows or values ​​assigned to the individual edges of a directed graph or to a subset of possible architectures of an artificial neural network are optimized and adapted to the actual situation instead of estimating the gradients has the advantage that this is, for example, less susceptible to noise and that fewer iterations are required overall to determine the optimal architecture, thereby saving resources required for determining the optimal architecture, such as memory and / or processor capacity. The determination of the optimal architecture of an artificial neural network based on a previously determined architecture, which is optimal in each case in view of the presets to be considered when determining the optimal preset, has the advantage that the optimal preset of the artificial neural network does not have to be completely re-determined in a complex and resource-intensive manner if the presets are to be weighted differently or other criteria are to be considered.

[0052] With a further specific embodiment of the invention, a computer program is also specified with a program code for carrying out the above-described method for determining an optimal architecture of an artificial neural network when the computer program is executed on a computer.

[0053] The computer program has the advantage that it is designed to carry out an improved method for determining an optimal architecture for an artificial neural network. The fact that potential architectures are tested or queried based on the rewards assigned to these architectures instead of optimizing the rewards themselves has the advantage that the accuracy in determining the optimal architecture and in particular the probability that the optimal architecture can be found can be increased. The fact that the gradients are not estimated but the flows or values ​​assigned to the individual edges of the directed graph or the distribution between the subsets of possible architectures of the artificial neural network are optimized and adapted to the actual situation has the advantage that this is, for example, less susceptible to noise and that fewer iterations are required overall to determine the optimal architecture, thereby saving resources required for determining the optimal architecture, such as memory and / or processor capacity. The determination of the optimal architecture of the artificial neural network based on previously determined architectures, which are each optimal in view of the presets to be considered when determining the optimal presets, has the advantage that the optimal presets of the artificial neural network do not have to be completely re-determined in a laborious and resource-intensive manner if the presets are to be weighted differently or other criteria are to be considered.

[0054] In summary, it can be determined that the method described in the present invention for determining the optimal architecture of an artificial neural network, in particular, the method for determining the optimal architecture of an artificial neural network with respect to multiple criteria can save resources when determining the optimal architecture and can also improve the accuracy when determining the optimal architecture.

[0055] The described embodiments and developments can be combined with one another as desired.

[0056] Further possible embodiments, refinements and implementations of the invention also include combinations of features of the invention which have been described above or below with reference to the exemplary embodiments, which have not been explicitly mentioned. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The accompanying drawings should facilitate a further understanding of the embodiments of the present invention. The drawings illustrate the embodiments and, together with the description, serve to explain the principles and concepts of the present invention.

[0058] Further embodiments and many of the advantages mentioned are apparent from the drawings. The elements shown in the drawings are not necessarily shown in true proportion to one another.

[0059] Figure 1 A flowchart showing a method for determining an optimal architecture of an artificial neural network according to an embodiment of the present invention; and

[0060] Figure 2 A schematic block diagram of a system for determining an optimal architecture of an artificial neural network according to an embodiment of the present invention is shown.

[0061] In the figures of the drawings, identical reference signs denote identical or functionally identical elements, components or assemblies unless otherwise indicated. DETAILED DESCRIPTION

[0062] Figure 1 A flow chart of a method for determining an optimal architecture of an artificial neural network 1 according to an embodiment of the present invention is shown.

[0063] Neural Architecture Search (NAS) is generally understood to be a method for automatically developing an optimal architecture for an artificial neural network for a given problem. The optimal architecture makes the complex manual design of the artificial neural network superfluous and is a subfield of automated machine learning.

[0064] The scalable neural architecture search method is a gradient-based method. Here, a hypergraph is formed by all possible architectures contained in the search space for an artificial neural network, wherein each possible architecture is a subgraph of the hypergraph. The nodes of the hypergraph are respectively used to symbolize a subset of one of the possible architectures, wherein one node is respectively able to symbolize exactly one possible layer of the artificial neural network, wherein the initial node is used to symbolize the input layer of the artificial neural network, wherein the end nodes of the directed graph are respectively used to symbolize a subset of one of the possible architectures with an output layer, and wherein the edges are used to symbolize possible links between subsets, wherein parameters are assigned to each edge based on the strategy for selecting nodes. In addition, an attempt is made to find an architecture based on the hypergraph for which the reward or benefit is the largest, wherein a gradient descent method is used to determine the optimal architecture for the artificial neural network. In the case of multi-criteria tasks or neural networks that should perform multiple tasks, it is also generally attempted to determine the optimal architecture based on a general strategy that should cover all criteria.

[0065] Here, Figure 1 Method 1 is shown, the method comprising step 2: providing a set of possible architectures of an artificial neural network, or a corresponding search space, step 3: representing the set of possible architectures of the artificial neural network in a directed graph, wherein the nodes of the directed graph respectively represent a subset of one of the possible architectures with symbols, wherein the start node represents an input layer with symbols, wherein the end nodes of the directed graph respectively represent a subset with an output layer with symbols, and wherein the edges of the directed graph represent possible links between the subsets with symbols, step 4: for each of at least two presets for determining the architecture, by assigning a stream to each edge of the directed graph, based on the directed graph provisions for determining the corresponding The method 1 further comprises a step 5 of determining an optimal architecture of an artificial neural network based on the corresponding optimal architecture of each of at least two presets; and a step 6 of providing an optimal architecture of the artificial neural network.

[0066] The advantage here is that the accuracy in determining the best architecture and in particular the probability that the actually best architecture can be found can be increased by not optimizing the reward itself but rather testing or querying potential architectures based on the rewards respectively assigned to these architectures.

[0067] Instead of estimating the gradients, the flows or values ​​assigned to the individual edges of the directed graph or the distribution between a subset of the possible architectures of the artificial neural network are optimized and adapted to the actual situation. This has the advantage, among other things, that it is less susceptible to noise, for example, and overall requires fewer iterations to determine the optimal architecture, thereby saving resources required for determining the optimal architecture, such as memory and / or processor capacity.

[0068] Furthermore, determining an optimal architecture of the artificial neural network based on a previously determined architecture, which is optimal in each case with respect to the presettings to be taken into account when determining the optimal presettings, has the advantage that the optimal presettings of the artificial neural network do not have to be completely re-determined in a complex and resource-intensive manner if the presettings are to be weighted differently or other criteria are to be taken into account.

[0069] Overall, an improved method for determining an optimal architecture for an artificial neural network 1 is thus described.

[0070] In particular, Figure 1 Here, method 1 is shown, which is based on the use of flow methods instead of gradient-based approaches in order to determine, in particular, the optimal architecture of a neural network for simultaneously executing multiple tasks. In this case, the desired architecture can be determined with regard to its performance with respect to the multiple tasks to be executed by the corresponding neural network.

[0071] According to the invention, at least two pre-settings have hardware metrics, in particular expected waiting time, energy consumption and / or currently available resources. In addition, at least two pre-settings can also have acceptance tasks, such as detection accuracy or detection type.

[0072] The set of possible architectures and thus the directed graph or hypergraph may here be based on labeled training data, for example labeled sensor data for training an artificial neural network.

[0073] according to Figure 1 In one embodiment, each node in the directed graph furthermore symbolizes exactly one possible layer of the artificial neural network. Based on the method 1 shown, for each of the at least two predefined settings, in particular a corresponding architecture can be built sequentially, i.e. each layer can be selected individually, or it can be determined individually at which layer should be inserted at which time. For this purpose, in particular, the links of the directed graph can be predefined at a predefined set of actions involving the selection of the individual edges of the directed graph.

[0074] The step of determining a reward for the determined trajectory can also be performed, for example, in each case again by training the architecture represented by the determined trajectory based on labeled training data, wherein the obtained results are subsequently verified or assessed with respect to performance with respect to corresponding specifications.

[0075] For example, the cost function can be determined by determining flow matching objectives, but it is also possible to determine the cost function by determining a detailed balance objective and a backward policy or a trajectory balance objective.

[0076] The step of respectively updating the flows assigned to the edges along the trajectories based on the cost function can furthermore comprise applying a backtracking algorithm.

[0077] The termination criteria can also be selected in each case such that the determination of the corresponding optimal architecture is terminated as soon as the reward determined for the determined trajectory lies within a respectively predefined target range of the reward, wherein the target range can be different for each of the at least two presets, for example depending on the importance of the corresponding preset.

[0078] The initial flow value can also be selected randomly here.

[0079] Furthermore, a corresponding strategy for determining an optimal architecture based on a directed graph may be based on flow values.

[0080] according to Figure 1 An embodiment of the invention, for each of at least two presets, a strategy for determining a corresponding optimal architecture for each node of a directed graph based on a directed graph, in particular describes the probability that a trajectory to be determined passes through a corresponding node of the directed graph, wherein the probability is respectively proportional to the flow of edges assigned to the directed graph from a previously selected node to the corresponding node, and wherein the trajectory is determined with the highest probability and / or proportional to the probability by respectively selecting the edge.

[0081] The strategy here also specifies that at certain points in time it is also possible to deviate from the specified probability and follow other paths.

[0082] according to Figure 1 In an embodiment, step 5 of determining the optimal architecture of the artificial neural network based on the corresponding optimal architecture for each of at least two presets includes summing the determined optimal architectures for each of the at least two presets or for all of the at least two presets in a weighted manner.

[0083] In particular, the optimal architecture of the artificial neural network is determined here by summing the corresponding individual rewards in a weighted manner and summing the corresponding individual flows in a weighted manner.

[0084] according to Figure 1 In one embodiment of the invention, the weighting is here also based on the current hardware situation of the at least one target component.

[0085] For example, the weighting can be selected in this case such that the energy consumption is minimized as far as possible as soon as the level of the energy store of the at least one target component is below a threshold level.

[0086] In particular, the predefined importance or the desired influence can be varied by correspondingly selecting the weighting.

[0087] according to Figure 1 In an embodiment of the invention, the reward for a trajectory is also determined based on the hardware conditions of at least one target component. For example, the hardware requirements can be taken into account in determining the performance of an artificial neural network trained based on the architecture and training data represented by the trajectory, wherein a weighting factor can be assigned to the hardware characteristics, and wherein the larger the weighting factor is selected, the more the focus is on the hardware requirements.

[0088] The optimal architecture determined by method 1 can then be used to train a corresponding artificial neural network based on the corresponding labeled training data.

[0089] In particular, an artificial neural network can be trained to control a controllable system and subsequently used to control the controllable system, wherein the controllable system can be, for example, an embedded system of a motor vehicle or a function of an autonomous motor vehicle. For example, the method can be used to determine an optimal architecture of a neural network for performing a plurality of tasks within the scope of partially automated driving.

[0090] However, in addition, artificial neural networks can also be trained to classify image data, in particular digital image data based on low-level features, such as edges or pixel properties. In this case, image processing algorithms can also be used to analyze classification results that focus on corresponding low-level features.

[0091] Figure 2 A schematic block diagram of a system for determining an optimal architecture of an artificial neural network 10 according to an embodiment of the present invention is shown.

[0092] according to Figure 2In an embodiment of the present invention, the system 20 has a providing unit 11, which is configured to provide a set of possible architectures of the artificial neural network; a mapping unit 12, which is configured to map the set of possible architectures of the artificial neural network onto a directed graph, wherein the nodes of the directed graph respectively represent subsets of one of the possible architectures with symbols, wherein the initial node represents the input layer with symbols, wherein the end nodes of the directed graph respectively represent subsets with output layers with symbols, and wherein the edges of the directed graph represent possible links between the subsets with symbols; a first determining unit 13, which is configured to, for each of at least two presets for determining the architecture, respectively assign a flow to each edge of the directed graph, prescribe a strategy for determining the corresponding optimal architecture based on the directed graph, and repeatedly determine the optimal architecture from the initial node based on the prescribed strategy. The invention relates to a method for determining an optimal architecture according to a point-to-end node trajectory, determining a reward for the determined trajectory, determining a cost function for the determined trajectory based on the determined reward for the trajectory and the flow assigned to the edge along the trajectory, and updating the flow assigned to the edge along the trajectory based on the cost function to respectively determine the optimal architecture with respect to the corresponding preset, wherein the steps of determining the trajectory, determining the reward, determining the cost function and updating the flow are repeated until the determined trajectory satisfies the termination criterion for architecture search, and wherein the trajectory that satisfies the termination criterion represents the optimal architecture; a second determining unit 14, which is constructed to determine the optimal architecture of the artificial neural network based on the corresponding optimal architecture with respect to each of at least two presets; and a second providing unit 15, which is constructed to provide the optimal architecture of the artificial neural network.

[0093] The first providing unit can be a receiver, which is designed to receive corresponding data. The second providing unit can be a transmitter, which is designed to transmit corresponding data or information. The first providing unit and the second providing unit can also be integrated into a common transceiver.

[0094] Furthermore, the imaging unit, the first determination unit and the second determination unit may each be implemented, for example, based on a code stored in a memory and executable by a processor.

[0095] according to Figure 2 In an embodiment of the present invention, the second determination unit 14 is again designed to determine the optimal architecture of the artificial neural network by summing the at least two predetermined optimal architectures in a weighted manner.

[0096] The weighting can in turn be based on the current hardware situation of the at least one target component.

[0097] according to Figure 2In an embodiment of the present invention, the first determination unit 13 is furthermore designed to determine a reward for a trajectory based on the hardware conditions of at least one target component.

[0098] Furthermore, the system 10 may be designed, in particular, to carry out the above-described method for determining an optimal architecture of an artificial neural network.

Claims

1. A method for determining an optimal architecture of an artificial neural network, wherein the method (1) comprises the following steps: - providing a set of possible architectures of the artificial neural network (2); - mapping the set of possible architectures of the artificial neural network onto a directed graph, wherein the nodes of the directed graph respectively symbolize a subset of one of the possible architectures, wherein the initial node symbolizes the input layer, wherein the terminal nodes of the directed graph respectively symbolize the subset having the output layer, and wherein the edges of the directed graph respectively symbolize possible links between the subsets (3); - for each of at least two presets for determining an architecture, respectively, assigning a flow to each edge of the directed graph, specifying a strategy for determining the corresponding optimal architecture based on the directed graph, and determining the corresponding optimal architecture based on the specified strategy by repeatedly determining a trajectory from an initial node to an end node, determining a reward for the determined trajectory, determining a cost function for the determined trajectory based on the determined reward for the trajectory and the flow assigned to the edge along the trajectory, and respectively updating the flow assigned to the edge along the trajectory based on the cost function to respectively determine the architecture that is optimal with respect to the corresponding preset, wherein the steps of determining the trajectory, determining the reward, determining the cost function, and updating the flow are repeated until the determined trajectory satisfies a termination criterion for architecture search, and wherein the trajectory that satisfies the termination criterion represents the optimal architecture (4); - determining an optimal architecture of the artificial neural network based on the corresponding optimal architecture for each of at least two presets (5); and - Providing an optimal architecture of the artificial neural network (6).

2. The method (1) according to claim 1, wherein the step (5) of determining the optimal architecture of the artificial neural network based on the corresponding optimal architecture for each of the at least two presets comprises summing the corresponding optimal architecture for each of the at least two presets in a weighted manner.

3. The method (1) according to claim 2, wherein the corresponding weighting is based on the current hardware situation of at least one target component. 4 . The method ( 1 ) according to claim 1 , wherein the reward for the determined trajectory is determined based on the hardware conditions of at least one target component.

5. A method for training an artificial neural network, wherein the method comprises the following steps: -Provide training data for training artificial neural networks; - providing an optimal architecture for an artificial neural network, wherein the optimal architecture has been determined by a method for determining an optimal architecture for an artificial neural network according to any one of claims 1 to 4; and -Training the artificial neural network based on the training data and the optimal architecture. The method of claim 4 , wherein the training data comprises sensor data.

7. A method for controlling a controllable system based on an artificial neural network, wherein the method comprises the following steps: - providing an artificial neural network, which is trained to control a controllable system, wherein the artificial neural network has been trained by a method for training an artificial neural network according to claim 5 or 6; and - Controlling the controllable system based on the provided artificial neural network.

8. A system for determining an optimal architecture of an artificial neural network, wherein the system (10) comprises: a first providing unit (11), the first providing unit being configured to provide a set of possible architectures of the artificial neural network; a mapping unit (12), the mapping unit being configured to map the set of possible architectures of the artificial neural network onto a directed graph, wherein the nodes of the directed graph respectively symbolize subsets of one of the possible architectures, wherein the initial node symbolizes an input layer, wherein the end nodes of the directed graph respectively symbolize subsets having an output layer, and wherein the edges of the directed graph respectively symbolize possible links between the subsets; At least one first determination unit (13), the first determination unit being configured to, for each of at least two presets for determining an architecture, respectively assign a flow to each edge of the directed graph, specify a strategy for determining the corresponding optimal architecture based on the directed graph, and determine the corresponding optimal architecture based on the specified strategy by repeatedly determining a trajectory from an initial node to an end node, determining a reward for the determined trajectory, determining a cost function for the determined trajectory based on the determined reward for the trajectory and the flow assigned to the edge along the trajectory, and respectively updating the flow assigned to the edge along the trajectory based on the cost function to respectively determine the architecture that is optimal with respect to the corresponding preset, wherein the steps of determining the trajectory, determining the reward, determining the cost function, and updating the flow are repeated until the determined trajectory satisfies a termination criterion for architecture search, and wherein the trajectory that satisfies the termination criterion represents the optimal architecture; a second determination unit (14) configured to determine an optimal architecture of the artificial neural network based on a corresponding optimal architecture for each of at least two presets; and a second providing unit (15), which is configured to provide an optimal architecture of the artificial neural network.

9. The system (10) according to claim 8, wherein the second determination unit (14) is configured to determine the optimal architecture of the artificial neural network by summing the corresponding optimal architectures for each of the at least two presets in a weighted manner.

10. The system (10) of claim 9, wherein the respective weighting is based on a current hardware condition of at least one target component.

11. The system (10) according to any one of claims 8 to 10, wherein the at least one first determination unit (13) is respectively designed to determine a reward for the trajectory based on hardware conditions of at least one target component.

12. A system for training an artificial neural network, wherein the system comprises: a first providing unit, which is configured to provide training data for training the artificial neural network; a second providing unit, which is configured to provide an optimal architecture for the artificial neural network, wherein the optimal architecture has been determined by a system for determining an optimal architecture for an artificial neural network according to any one of claims 8 to 11; and a training unit, which is configured to train the artificial neural network based on the training data and the optimal architecture. The system of claim 12 , wherein the training data comprises sensor data.

14. A system for controlling a controllable system based on an artificial neural network, wherein the system comprises: a providing unit, which is constructed to provide an artificial neural network, which is trained to control the controllable system, wherein the artificial neural network has been trained by the system for training an artificial neural network according to claim 13 or 14; and a control unit, which is constructed to control the controllable system based on the provided artificial neural network.

15. A computer program having a program code for executing the method for determining an optimal architecture of an artificial neural network according to any one of claims 1 to 4 when the computer program is executed on a computer.