Method and / or apparatus for architecture search

Abstract

The invention relates to a method for searching for an architecture of a one-shot neural network for solving a multi-task problem depending on at least one target hardware.

Description

[0001] The invention relates to a method and / or a device for architecture search of a neural network architecture. State of the art

[0002] Neural networks are increasingly being used to evaluate measured variables in vehicle control units and other embedded systems. Compared to other evaluation techniques, neural networks are characterized by their high generalization power. This allows them, after sufficient training, to accurately evaluate even previously unseen situations and generate control signals that lead to a situation-appropriate response from the vehicle or other system.

[0003] The price for this performance is that neural networks place comparatively high demands on the hardware platform on which they are implemented. Typically, graphics processing units (GPUs) with large memory configurations are required. Aside from the fact that the price of the hardware platform increases with its configuration, space is often limited in vehicle control units and other embedded systems. The maximum power consumption is also limited by the energy source used, such as a vehicle's electrical system or a battery, and / or by the maximum permissible heat dissipation. Therefore, DE 10 2019 202 816 A1 discloses a method for training a neural network in which less relevant neurons and connections between neurons are completely deactivated.

[0004] The applicant also discloses a method for multi-criteria architecture search. This method deals with the search for a neural one-shot architecture for multi-objective optimization. A technique is proposed that allows a multi-objective optimization problem to be optimized without having to resort to scalarizing the various objectives. Furthermore, this approach offers not just a single architecture for a specific weighting of the optimization objectives, but rather a series of optimal architectures for different weightings of the objectives. To achieve this, the method comprises the following steps: The objectives are sampled, weighted from the Dirichlet distribution. Furthermore, a sample of an architecture is drawn from a distribution dependent on an objective weight. Weights are updated to determine a loss function.To minimize the cost function, the parameters of the one-shot network are adjusted. Furthermore, architectural parameters are adjusted to minimize the cost function.

[0005] It is an object of the invention to provide an improved method and / or an improved device for training a one-shot neural network.

[0006] The object is achieved by the method according to the features of patent claim 1. The object is achieved by the device according to the features of patent claim 10. Disclosure of the invention

[0007] According to a first aspect, a method for searching for an architecture of a one-shot neural network f(w, α) for solving a multi-task problem depending on at least one target hardware D is provided. The method comprises the steps of: providing a hypernetwork g with network parameters θ, task weights or partial target weights λ and at least one hardware embedding e hw for the at least one target hardware D; providing the one-shot neural network f(w, α) with network weights w and architecture parameters α; initializing the task weights λ; in particular, randomly selecting, in particular by means of the hypernetwork g, the architecture parameters α depending on the initialized task weights λ and the at least one hardware embedding e hw for each target hardware d of the at least one target hardware D; calculating a gradient gθd with respect to the network parameters θ by a loss function gθd=∇θ∑iλiLi(α,w); Calculating a multiple gradient descent g θ for each target hardware d of the at least one target hardware D; updating the network parameters θ based on the multiple gradient descent g θ and / or based on the update of the task weights λ; calculating a gradient gwd with respect to the network weights w by a loss function gwd=∇w∑iλiLi(α,w); Aggregating the calculated gradients gwd; and updating the network weights w based on the aggregated gradients.

[0008] According to a second aspect, a device for searching the architecture of a one-shot neural network f(w, α) for solving a multi-task problem depending on at least one target hardware D is specified. The device comprises an evaluation and computing device which is designed to carry out the following steps: providing a hypernetwork g with network parameters θ, task weights λ and at least one hardware embedding e hw for the at least one target hardware D; providing the one-shot neural network f(w, α) with network weights w and architecture parameters α; initializing the task weights λ; in particular, randomly selecting the architecture parameters α depending on the initialized task weights λ and the at least one hardware embedding e hw for each target hardware d of the at least one target hardware D; calculating a gradient gθd with respect to the network parameters θ by a loss function gθd=∇θ∑iλiLi(α,w);

[0009] Calculating a multiple gradient descent g θ for each target hardware d of the at least one target hardware D; updating the network parameters θ based on the multiple gradient descent g θ and / or based on the update of the task weights λ; calculating a gradient gwd with respect to the network weights w by a loss function gwd=∇w∑iλiLi(α,w); Aggregating the calculated gradients gwd; and updating the network weights w based on the aggregated gradients.

[0010] The statements made for the method apply accordingly to the device and vice versa.

[0011] It is understood that the steps according to the invention, as well as other optional steps, do not necessarily have to be performed in the order shown, but can also be performed in a different order. Furthermore, additional intermediate steps can be provided. The individual steps can also comprise one or more substeps without thereby departing from the scope of the method according to the invention.

[0012] The training of the one-shot neural network f(w, α) is done by updating the network parameters θ and the network weights w, i.e., based on the updated network weights w and the updated network parameters θ.

[0013] Particularly preferably, one or more meta-learned predictors for hardware objective functions are provided. The hardware objective functions are preferably optimized with respect to latency. The predictors are preferably trained before the architecture search and then used to calculate objective functions for hardware. The predictors, like the function g, preferably use a hardware embedding in order to be able to generalize to different hardware. The predictors are preferably trained using a simple procedure, in particular by sampling a hardware and architecture in each training step. The task weights are preferably non-trainable and are drawn from a probability distribution in each training iteration.The method and / or device deals with the architecture search for a neural network architecture for multi-task approaches and / or for multi-criteria approaches and / or for hardware-sensitive approaches.

[0014] In the context of neural architecture search, a one-shot model preferably refers to a technique in which a single neural network is trained to simultaneously represent and evaluate multiple candidate architectures. Instead of training and evaluating each architecture individually, a one-shot model enables efficient exploration of a large search space by sharing weights and parameters across different architectures. The one-shot model consists of a supernetwork, which represents a larger network encompassing all possible architectures. It includes various architectural components such as convolutional layers, pooling layers, and skip connections, which can be selectively activated or deactivated for each candidate architecture. By using different activation patterns, the supernetwork can simulate different architectures within a single model.During the training process, the one-shot model is trained on a proxy task such as image classification using a combination of architectural parameters and weight distribution. The model learns to adjust its weights and parameters based on the performance of different architectures within the supernetwork. This enables efficient exploration of the search space, as the model can evaluate multiple architectures simultaneously and update its parameters accordingly. Once training is complete, the one-shot model can select the best architecture based on its learned weights and parameters. This selected architecture can then be trained from scratch or fine-tuned to achieve optimal performance in the target application.Overall, the one-shot model approach in neural architecture search enables efficient exploration of the search space by training a single model to simultaneously represent and evaluate multiple candidate architectures.

[0015] A hypernetwork is a type of neural network designed to generate or influence the parameters of another neural network. A hypernetwork typically comprises a main network, referred to as the "base network," and a secondary or "hypernetwork." The base network can be, for example, a CNN (convolutional neural network) or an RNN (recurrent neural network), or, as in this case, a one-shot network trained for multi-tasks such as image classification or language processing. The hypernetwork is used to generate or manipulate the weights or other parameters of the base network or one-shot model. This provides a flexible way to adjust the structure and / or parameters of the one-shot network without having to retrain the entire network.This can be particularly useful when adapting or fine-tuning a network for tasks or data without requiring all the training time and resources to retrain.

[0016] It is understood that the at least one image file may also be a video file. The provisions of this application apply accordingly to video files to be generated. In this case, a text-in-video generation algorithm is preferably used.

[0017] The method and / or the device can be used, for example, in the technical context of generic facial recognition and / or in the technical context of vehicle assistance systems and / or in the technical context of autonomous driving and / or in the technical context of computer vision and / or in the technical context of quality monitoring of production components in automatic optical inspection and / or in the technical context of other technical fields in which image data is evaluated and / or categorized and / or classified.

[0018] Particularly preferably, the method and / or the device can be used in the analysis of data acquired by at least one (image) sensor. The at least one sensor can, for example, determine measured values ​​of an environment in the form of sensor signals. Such sensor signals can, for example, be present in particular as digital images and / or videos. The sensor can, for example, be a camera and / or a lidar sensor and / or an ultrasound sensor. The invention can therefore, for example, be used for image and / or video and / or audio analysis downstream of the acquisition and there for classifying or segmenting the acquired image data, in particular in order to solve multi-task problems. In general, this can involve any type of sensor data (including radar, lidar, ultrasound, etc.) and / or data from multiple sensors (e.g., multiple cameras) and / or data from combinations of sensors (e.g., camera, radar & lidar).

[0019] The invention can be used in particular to classify the sensor data and / or to detect the presence of objects in the sensor data and / or to perform a semantic segmentation of the sensor data, e.g. with regard to traffic signs, road surfaces, pedestrians and / or vehicles. In this case, anomalies in the classification and / or segmentation of input data, e.g. the sensor data, can also be detected. The method and / or the device can also be used to determine one or more continuous values, i.e. to perform a regression analysis, e.g. with regard to a distance, a speed and / or an acceleration. The method and / or the device can also be used to track an element, e.g. an object, in the input data. This is done, for example, on the basis of low-level features (e.g.edges or pixel attributes in images).

[0020] The method and / or device can also be used to calculate a control signal for controlling a technical system, such as a computer-controlled machine, e.g. a robotic system, a vehicle, a household appliance, a power tool, a manufacturing machine, a personal assistant or an access control system. It can also be a system for transmitting information, such as a monitoring system or a medical (imaging) system. The method and / or device can analyze input data (e.g. scalar time series), in particular from a sensor, in fact any sensor, and operate the technical system accordingly on the basis of the analysis.

[0021] In this example, the method, specifically the hypernetwork, is extended by adding a hardware embedding as an additional input to the network, thus enabling the sampling of one-shot network architectures for different hardware. Furthermore, an optional memory bank and an attention mechanism can be added to the neural network.

[0022] Furthermore, a training strategy is adapted here that thus considers multiple target hardware devices. First, the updates for θ are preferably calculated for multiple target hardware platforms or target hardware devices using multiple gradient descent. Furthermore, the updates for the one-shot model weights w are preferably calculated based on multiple architectures, namely based on the different hardware embeddings considered by the hypernetwork g.

[0023] A meta-learned latency predictor is provided, which is trained by sampling an architecture and a target device during each training step. The predictors are preferably used in addition to the hypernetwork. These are preferably trained separately before the actual architecture search, specifically before the described training procedure. During the architecture search, the predictors are fixed and remain unchanged.

[0024] The present method allows to train and / or test architectures within the one-shot network for different hardware, which can be used both during training and for unknown hardware embeddings.

[0025] The training procedure preferably optimizes the hypernetwork so that it can predict architectures for multiple target hardware and also allows for predictions for unknown hardware with zero shots. The proposed training procedure leads to improved predictor performance and potentially also to lower resource consumption.

[0026] The presented method optimizes the architecture of neural networks with respect to multiple objectives (also called a multi-objective optimization problem). These can include both objectives related to the performance of a neural network on a specific task (e.g., object recognition from images) and objectives related to the efficiency of an architecture on a target hardware device. The method can be applied to any system that uses neural networks.

[0027] The method and / or apparatus may also utilize multi-objective search with respect to model performance (e.g., accuracy, etc.) and the model's hardware performance (e.g., latency, FLOPS, power consumption, memory usage). For example, an additional loss term may be added to the loss function, where the additional loss term predicts hardware performance depending on architectural parameters of the one-shot model and / or the hypermodel.

[0028] Here, the one-shot model f(w, α) is used, which represents the superposition of several neural network architectures in a single network, where w are the weights of the individual architectures of the network and α are the architecture parameters that determine which of the architectures embedded in the one-shot model is active. α can be an output of a function or a probability distribution g with the parameters θ: α = g(θ). g is preferably represented by a neural network. The goal of the one-shot architecture search is preferably to optimize θ to minimize a loss. minθ L(α,w(λ))+λLhw(α), subject to minw(λ) Ldata*(α,w|λ),

[0029] Here, w are the optimal layer parameters, which may depend on λ. The layer parameters w can preferably be optimized by optimizing another data-dependent loss. The loss L(α, w) can, for example, be the loss for validation data. Ldata*(α,w) can be the loss in the training data. The following can be preferred: L(α,w)=Ldata*(α,w). Furthermore, there can in principle be more than two loss functions, ie more generally minθ ∑iλiLi(α,w)subject to minw Ldata*(α,w|λ).

[0030] In a preferred aspect, the initialization comprises a sample weighting of the task weights λ from a Dirichlet distribution Dir(β) based on a concentration hyperparameter β.

[0031] The Dirichlet distribution is a family of multivariate probability distributions and is used in statistics for modeling proportions or probability distributions across categories. The distribution is characterized by a vector of positive real numbers that serve as concentration parameters. These parameters influence how strongly the weights are concentrated towards the vertices or edges of the simplex represented by the Dirichlet distribution (a multidimensional space where the sum of the coordinates is 1). In the context of sampling weighting, initializing task weights from a Dirichlet distribution means that the initial distribution of weights between different tasks is determined by the characteristics of the Dirichlet distribution, which are controlled by .A higher value of would result in a more even distribution of weights, meaning a less biased initialization, while a lower value favors a greater focus on fewer tasks, which can be potentially useful when some tasks are considered more important than others or when it is desired to optimize model performance with respect to specific tasks.

[0032] In a preferred aspect, the method comprises providing a hardware predictor with architectural parameters α and at least one hardware embedding e hw . In a preferred aspect, the hardware predictor is trained by drawing an architecture and target hardware in each gradient update step, in particular randomly, for example from a probability distribution or a Dirichlet distribution.

[0033] In a preferred aspect, the concentration hyperparameter β is initialized as a vector of ones.

[0034] Initializing β as a vector of ones results in a uniform Dirichlet distribution. In a uniform Dirichlet distribution, all possible distributions of the categories (or tasks) are equally likely. This approach implies no a priori assumptions about the importance or distribution of the categories and thus represents an "uninformed" starting condition. Using a vector of ones as the initialization value is easy to implement and understand. It provides a clear and neutral starting point for modeling, which can be particularly useful when no specific prior information about the data to be modeled is available. Although initialization begins with a vector of ones, the parameters β and the associated task weights λ can be adjusted during the training process.This allows the model to learn from the data and optimize the weights accordingly, based on observed performance or data properties. In multi-task learning scenarios or other contexts where the Dirichlet distribution is used to guide the initialization or tuning of model parameters, choosing a vector of ones as the initialization of β can influence the learning process by providing a neutral, unbiased basis for exploring different weights.

[0035] In a preferred aspect, the network parameters θ are updated by applying a Frank-Wolfe routine or by averaging the gradients of the multiple gradient descent.

[0036] Updating task weights is a step to optimize the weights of the different tasks and improve the overall performance of the model. Two specific methods are highlighted for this update: the Frank-Wolfe routine and the gradient averaging of multiple gradient descent. The Frank-Wolfe routine, also known as the conditioned gradient method, is an optimization algorithm used for convex optimization problems. Unlike other gradient descent methods that take steps toward the negative gradient of the objective function, the Frank-Wolfe routine searches for a solution within a convex solution space by solving a linear sub-problem in each iteration step. This approach is particularly useful for optimization problems where the solution must remain within a certain region or space, such asin the optimization of task weights that must satisfy certain constraints. Gradient averaging of multiple gradient descent is an approach typically applied in multi-task learning scenarios where several tasks are learned simultaneously. This method considers the gradients of all tasks and averages them to find a common direction for updating the weights. This technique aims to find a compromise between the different tasks and determine an update direction that is beneficial for all tasks overall. The Frank-Wolfe routine refers to how the gradients are aggregated with respect to the θ parameters. The θ parameters are then updated based on the descent direction calculated using the Frank-Wolfe routine and using any optimizer (Gradient Descent + Momentum, Adam, ...).

[0037] In a preferred aspect, the aggregation is carried out by averaging gw=1|D|∑d∈Ddwd.

[0038] Other aggregation methods may also be beneficial.

[0039] In a preferred aspect, the selection of the architecture parameters α comprises an output of a function or a probability distribution and / or the selection by the hypernetwork. In a preferred aspect, the provision of the architecture parameters α is carried out using the function g (or the hypernetwork) in a differentiable manner. For the use of discrete architecture parameters α, this can be done using differentiable approximations such as the straight-through gumbel softmax.

[0040] If the selection of the architecture parameters α is based on the output of a function, this means that a specific mathematical or algorithmic function is used to determine the optimal or appropriate values ​​for α. This function can be based on heuristics, optimization techniques, or on performance metrics of the model on a validation dataset. The function evaluates potential architectures or configurations and selects the one that performs best according to a specific criterion. If the selection is based on a probability distribution, this implies a probabilistic approach to determining α. Instead of directly determining fixed values, the architecture parameters are treated as random variables that follow certain distributions.This method can be applied, for example, in Bayesian optimization strategies, where the probability distribution represents the uncertainty about model performance over the space of possible architectures. The selection is then performed in a way that iteratively reduces this uncertainty by selecting new data points (model architectures) that maximize the expected information gain.

[0041] In a preferred aspect, an inference method is proposed, which comprises: applying a one-shot neural network f(w, α) trained according to the present method in one of its aspects to solve a multi-task problem, in particular a classification task and / or a segmentation task.

[0042] Multi-task problems refer to scenarios where a model is required to solve multiple tasks simultaneously. This can increase the efficiency of the learning process and improve the model's generalization ability, as it learns to extract features that are useful across different tasks.

[0043] Classification tasks are those where the goal is to classify inputs into predefined categories. Segmentation tasks are those that aim to identify and classify specific regions within the input data, often in the context of images, where the goal is, for example, to detect different objects within an image and determine their boundaries. The proposed inference method leverages the trained one-shot network to effectively solve these multi-task problems. By applying the one-shot network, the learned weights and architectural parameters are used to tackle new instances of the tasks with minimal training data.

[0044] In a preferred aspect, a computer program with program code is also claimed for executing at least parts of the method according to the invention in one of its embodiments when the computer program is executed on a computer. In other words, a computer program (product) comprising instructions that, when executed by a computer, cause the computer to execute the method / steps of the method according to the invention in one of its embodiments.

[0045] In a preferred aspect, a computer-readable data carrier with program code of a computer program is also proposed for executing at least parts of the method according to the invention in one of its embodiments when the computer program is executed on a computer. In other words, the invention relates to a computer-readable (storage) medium comprising instructions which, when executed by a computer, cause the computer to execute the method / steps of the method according to the invention in one of its embodiments.

[0046] The described designs and further training courses can be combined as desired.

[0047] Further possible embodiments, developments and implementations of the invention also include combinations of features of the invention described previously or below with regard to the embodiments that are not explicitly mentioned. Short description of the drawings

[0048] The accompanying drawings are intended to provide a further understanding of embodiments of the invention. They illustrate embodiments and, in conjunction with the description, serve to explain principles and concepts of the invention.

[0049] Other embodiments and many of the aforementioned advantages will become apparent upon review of the drawings. The elements illustrated in the drawings are not necessarily drawn to scale.

[0050] They show: Fig. 1 is a schematic flow diagram of the present method; and Fig. 2 a schematic block diagram of the method according to the invention.

[0051] In the figures of the drawings, the same reference symbols designate the same or functionally equivalent elements, parts or components, unless otherwise stated.

[0052] Fig. 1 shows a schematic flow diagram of a method for training a one-shot neural network f(w, α) to solve a multi-task problem depending on at least one target hardware D.

[0053] In any embodiment, the method can be carried out at least partially by a device 100, which for this purpose can comprise several components not shown in detail, for example, one or more provision devices and / or at least one evaluation and computing device. It is understood that the provision device can be designed jointly with the evaluation and computing device or can be different from it. Furthermore, the device can comprise a storage device and / or an output device and / or a display device and / or an input device.

[0054] The computer-implemented method comprises at least the following steps: In a step S1, a hypernetwork g is provided with network parameters θ, task weights λ and at least one hardware embedding e hw for at least one target hardware D.

[0055] In a step S2, the one-shot neural network f(w, α) with network weights w and architecture parameters α is provided.

[0056] In step S3, the task weights λ are initialized. This initialization involves a sample weighting of the task weights λ from a Dirichlet distribution Dir(β) based on a concentration hyperparameter β. The concentration hyperparameter β is initialized as a vector of ones.

[0057] In a step S4, the architecture parameters α are selected, in particular randomly, depending on the initialized task weights λ and the at least one hardware embedding e hw for each target hardware d of the at least one target hardware D. Selecting the architecture parameters α comprises an output of a function or a probability distribution. The selected architecture parameters α are preferably differentiable.

[0058] In a step S5, a gradient is calculated gθd with respect to the network parameters θ by a loss function gθd=∇θ∑iλiLi(α,w).

[0059] In a step S6, a multiple gradient descent g is calculated θfor each target hardware d of the at least one target hardware D. The calculation is performed by applying a Frank-Wolfe routine or by averaging the gradients of the multiple gradient descent.

[0060] In a step S7, the network parameters θ are updated based on the multiple gradient descent g θ and / or based on the task weights λ.

[0061] In a step S8, a gradient is calculated gwd with respect to the network weights w by a loss function gwd=∇w∑iλiLi(α,w).

[0062] In a step S9, the calculated gradients are aggregated gwd. Aggregation is done by averaging gw=1|D|∑d∈Ddwd.

[0063] In a step S10, the network weights w are updated based on the aggregated gradients.

[0064] Fig. 2 shows a block diagram of an embodiment of the present method.

[0065] A one-shot neural network 200 is initialized by parameters of a hypernetwork 202. The hypernetwork 202 can also be a metanetwork. The hypernetwork 202 has network parameters θ. The hypernetwork 202 receives task weights 204, λ, and at least one hardware embedding 206, e hw as inputs. The architecture parameters α can be randomly determined depending on the initialized task weights 204, λ and the at least one hardware embedding 206, e hw for each target hardware d of the at least one target hardware D can be selected, which is Fig. 2 is marked with the reference number 208.

[0066] The outputs of the hypernetwork 202 can also serve as input for a predictor 210, which also receives the hardware embedding(s) 206 as input. The predictor 210 can be an MLP (Multilayer Perceptron) or a GCN (Graph Convolutional Network). The predictor 210 can be pre-trained and frozen in this pre-trained state. The output of the predictor 210 can serve as input for a loss function 212, which also receives an output from the one-shot network 200, which was formed based on the at least one architectural parameter α. The results of the loss function are plotted in a graph that maps an error 214 (abscissa) against a latency 216 (ordinate). Latency is only one example of a hardware metric here. Energy consumption can also be mentioned as an example of a further hardware metric.

[0067] The procedure can be illustrated again by the following mathematical formulation: QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature

[0000] DE 10 2019 202 816 A1

[0003]

Claims

[1] Method for searching for an architecture of a one-shot neural network f(w, α) for solving a multi-task problem depending on at least one target hardware D; the method comprising the steps: - Providing (S1) a hypernetwork g with network parameters θ, task weights λ and at least one hardware embedding e hw for at least one target hardware D; - Providing (S2) the one-shot neural network f(w, α) with network weights w and architecture parameters α; - Initializing (S3) the task weights λ; - in particular, randomly selecting (S4) the architecture parameters α depending on the initialized task weights λ and the at least one hardware embedding e hw for each target hardware d of the at least one target hardware D; - Calculate (S5) a gradient gθd with respect to the network parameters θ by a loss function gθd=∇θ∑iλiLi(α,w); - Calculating a multiple gradient descent g θ for each target hardware d of the at least one target hardware D; - Updating (S7) the network parameters θ based on the multiple gradient descent g θ and / or based on the update of the task weights λ; - Calculate (S8) a gradient gwd with respect to the network weights w by a loss function gwd=∇w∑iλiLi(α,w); - Aggregating (S9) the calculated gradients gwd; and - Updating (S10) the network weights w based on the aggregated gradients. [2] The method (100) of claim 1, wherein the initializing comprises sampling the task weights λ from a Dirichlet distribution Dir(β) based on a concentration hyperparameter β. [3] The method (100) of claim 2, wherein the concentration hyperparameter β is initialized as a vector of ones. [4] A method according to any one of the preceding claims, wherein the updating of the network parameters θ is performed by applying a Frank-Wolfe routine or by averaging the gradients of the multiple gradient descent. [5] Method according to one of the preceding claims, wherein the aggregation is carried out by averaging gw=1|D|∑d∈Dgwd occurs. [6] Method according to one of the preceding claims, wherein selecting the architecture parameters α comprises an output of a function or a probability distribution and / or selecting by the hypernetwork. [7] Inference method that has: - Applying a one-shot neural network f(w, α) trained according to the method according to one of the preceding claims to solve a multi-task problem, in particular a classification task and / or a segmentation task. [8] Computer program with program code to carry out at least parts of a method according to one of claims 1 to 7 when the computer program is executed on a computer. [9] Computer-readable data carrier with program code of a computer program for carrying out at least parts of a method according to one of claims 1 to 7 when the computer program is executed on a computer. [10] Device (100) for searching for an architecture of a one-shot neural network f(w, α) for solving a multi-task problem depending on at least one target hardware D; the device (100) comprising an evaluation and computing device designed to carry out the following steps: - Providing a hypernetwork g with network parameters θ, task weights λ and at least one hardware embedding e hw for at least one target hardware D; - Providing the one-shot neural network f(w, α) with network weights w and architecture parameters α; - Initialize the task weights λ; - in particular, randomly selecting the architecture parameters α depending on the initialized task weights λ and the at least one hardware embedding e hw for each target hardware d of the at least one target hardware D; - Calculating a gradient gθd with respect to the network parameters θ by a loss function gθd=∇θ∑iλiLi(α,w); - Calculating a multiple gradient descent g θ for each target hardware d of the at least one target hardware D; - Updating the network parameters θ based on the multiple gradient descent g θ and / or based on the update of the task weights λ; - Calculating a gradient gwd with respect to the network weights w by a loss function gwd=∇w∑iλiLi(α,w); - Aggregating the calculated gradients gwd; and - Update the network weights w based on the aggregated gradients.

Images