Method for generating an optimized neural network architecture that takes into account the hardware constraints of a hardware target

Abstract

Method for generating an optimized neural network architecture taking into account the hardware constraints of a hardware target. The present invention relates to a method for generating an optimized neural network architecture taking into account the hardware constraints of a hardware target (20), the method comprising: obtaining an algorithmic task (100); defining a neural search space (104) as a function of the algorithmic task (100); defining a hardware configuration space (102) of the hardware target; searching for a neural network architecture for the hardware target using a genetic algorithm (108) to optimize evaluation criteria (106), the evaluation criteria (106) being at least one hardware performance and at least one algorithmic performance;and evaluate the hardware performance(s) by using a performance predictor if the performance predictor has a reliability above a reliability threshold or the hardware performance(s) being determined by a performance evaluation by running the neural network on the hardware target (20) if the performance predictor has a reliability below the reliability threshold. Figure for the abstract: 2;

Description

Title of the invention: Method for generating an optimized neural network architecture taking into account the hardware constraints of a hardware target

[0001] The present invention relates to a method for generating an optimized neural network architecture taking into account the hardware constraints of a hardware target, the hardware target comprising at least one accelerator and at least one general-purpose processor.

[0002] US2022108054A1 relates to an architecture search system that evaluates A search space for neural networks and hardware architecture configurations with a single multi-objective optimization algorithm. The optimization algorithm attempts to jointly search for the most optimized neural architecture and its execution configuration.

[0003] Neural networks (NNs) are used as an alternative to traditional algorithms due to their superior algorithmic performance on certain tasks, for example, in image processing. However, in the context of embedded artificial intelligence, their overly complex structure is unsuitable for execution on a target with latency or energy consumption constraints. It is therefore necessary to modify the weights, operations, or overall topology of the neural networks to ensure better compatibility with the final hardware target (Algorithm-Architecture Fit).

[0004] This disclosure falls within the general context of automated neural architecture design with consideration of deployment on hardware target, or "HardWare-aware Neural Architecture Search" (HW-NAS).

[0005] HW-NAS, also known as Hardware-aware NAS, refers to an automatic optimization process for neural network architectures that takes into account deployment constraints on a hardware accelerator. The idea is to find the optimal neural architecture for a specific task using machine learning algorithms.

[0006] To evaluate performance in the optimization loop, HW-NAS methods propose either to deploy the neural network entirely and directly on the target hardware ("Hardware-In-The-Loop") for a very accurate but slow evaluation, or to use embedded performance prediction models (truth tables, theoretical models, or regression models) for a faster but often inaccurate evaluation. In some cases, such predictors are difficult to be implemented due to a lack of clear information on the target hardware and the deployment flow, if it is proprietary.

[0007] A HW-NAS process can present an extremely complex optimization problem because the joint search space can be immense (e.g., >1020 possible combinations) and indistinguishable. In particular, performance evaluation is very time-consuming.

[0008] The aim of the invention is then to propose a method and a system for finding an optimal neural network architecture in a reduced time.

[0009] To this end, the invention relates to a method for generating an optimized neural network architecture taking into account the hardware constraints of a hardware target, the hardware target comprising at least one accelerator and at least one general-purpose processor, the method comprising: - Obtain an algorithmic task; - Define a neural search space based on the algorithmic task; - Define a hardware configuration space for the target hardware, the hardware configuration space being dependent on the target hardware; - Research into a neural network architecture for the target hardware using a genetic algorithm to optimize evaluation criteria, the evaluation criteria being at least one hardware performance and at least one algorithmic performance, the hardware performance(s) and the algorithmic performance(s) being evaluated in parallel; and - Evaluate the hardware performance(s) by using a performance predictor if the performance predictor has a reliability above a reliability threshold or the hardware performance(s) being determined by a performance evaluation by running the neural network on the hardware target if the performance predictor has a reliability below the reliability threshold.

[0010] According to other advantageous aspects of the invention, the method comprises one or more of the following features, taken individually or in all technically possible combinations: • the neural network is a convolutional neural network, particularly compatible with dense predictions requiring a multi-scale representation of information; • The architecture of the neural network is the structure and parameters of the neural network, in particular the number of layers, the type of layers, the weights, the operations that make up the neural network and / or their arrangement within the neural network; • The hardware accelerator is a graphics processing unit (GPU), a tensor processing unit (TPU), and / or a deep learning accelerator (DLA). • the algorithmic task is an image processing application; • Hardware performance includes latency and average power consumption. and / or the algorithmic performance is the average of the intersection over the union; • the performance predictor is a look-up table; • The performance predictor is a high-level model of the target material; • The performance predictor is an analytical model based on a neural network; and / or • the process further includes deploying the neural network having the optimal evaluation criteria on the target hardware.

[0011] The invention also relates to a use of the hardware target configured according to the method as defined above for image processing, in particular image recognition, object detection or semantic image segmentation.

[0012] The invention also relates to a system enabling the configuration of the architecture of a neural network for a hardware target, the hardware target comprising at least one accelerator and at least one general-purpose processor, the system comprising at least one computer, the computer being configured to be connected to the hardware target, the computer being suitable for implementing a method as defined above.

[0013] The invention also relates to a computer program comprising software instructions which, when executed by a computer, implement a method for configuring a neural network architecture for a hardware target, as defined above.

[0014] The invention will become clearer upon reading the following description, given solely by way of non-limiting example, and made with reference to the drawings in which:

[0015] [Fig-1] [Fig.1] schematically shows a system according to one embodiment;

[0016] [Fig.2] [Fig.2] is a schematic flowchart of a process according to an embodiment;

[0017] [Fig.3] [Fig.3] is a schematic flowchart of the communication between a computer managing the HW-NAS flow and the hardware target;

[0018] [Fig.4] [Fig.4] is a flowchart of a part of a process according to a mode of realization; and

[0019] [Fig.5] [Fig.5] shows the performance of a neural network on a hardware target.

[0020] [Fig. 1] Figure 1 schematically shows a system according to one embodiment. The system includes a computer 10 responsible for managing the HW-NAS process and communicating with the hardware target 20 to evaluate hardware performance. The computer 10 includes at least one processor 12 and at least one memory 14.

[0021] Computer 10 is suitable for implementing a method to generate an optimized neural network architecture taking into account the hardware constraints of a hardware target which will be described later.

[0022] The processor(s) 12 are general-purpose processors. Each processor 12 is an electronic circuit designed to manipulate and / or transform data represented by electronic or physical quantities in registers of the computer and / or memories 14 into other similar data corresponding to physical data in register memories or other types of display devices, transmission devices or storage devices.

[0023] Alternatively, when the method is implemented in the form of one or more software programs, i.e., in the form of a computer program, also called a computer program product, it is further capable of being stored on a computer-readable medium (not shown). The computer-readable medium is, for example, a medium capable of storing electronic instructions and being connected to a bus of a computer system. By way of example, the readable medium is an optical disc, a magneto-optical disc, ROM, RAM, any type of non-volatile memory (e.g., FLASH or NVRAM), or a magnetic card. A computer program comprising software instructions is then stored on the readable medium.

[0024] In addition, the computer 10 includes an input and output module 16, for example to obtain an algorithmic task and / or for a human-machine interface, in particular to define the general parameters of the HW-NAS flow.

[0025] The computer 10 is suitable for being connected to a hardware target 20.

[0026] The hardware target 20 comprises at least one general-purpose processor 22 and at less an accelerator 24. For example, the general-purpose processor 22 is a central processing unit (CPU). In one embodiment, the hardware target 20 further includes a memory 26. In other embodiments, the hardware target 20 includes multiplication circuits and accumulation (MAC) (programmable or hard-coded), registers and data storage.

[0027] There are several types of accelerators or hardware accelerators 24, for example a Graphics Processing Unit (GPU), Tensor Processing Unit (TPU), Neural Processing Unit (NPU) and / or a Deep Learning Accelerator (DLA). A DLA is a fixed-function processor that accelerates deep learning workloads on these platforms.

[0028] The accelerator(s) 24 are specifically designed to handle common operations in neural networks, such as: matrix multiplications, additions and / or type conversions.

[0029] Possible hardware configurations of the hardware target 20 include at least one accelerator and at least one general-purpose processor 22.

[0030] In one embodiment, it is not possible to access detailed information on the operation of the hardware target 20, for example the cycle-accurate behavior of the hardware target 20 and / or the internal operation of the hardware target 20 to predict the behavior, or the compatible deployment flow of the hardware target 20.

[0031] A neural network comprises an ordered succession of layers of neurons, each of which takes its inputs from the outputs of the previous layer.

[0032] More precisely, each layer includes neurons taking their inputs from the outputs of the neurons of the previous layer, or from the input variables for the first layer.

[0033] Alternatively, more complex neural network structures can be envisaged with a layer that can be linked to a layer further away than the immediately preceding layer.

[0034] Each neuron is also associated with an operation, that is to say a type of processing, to be carried out by said neuron within the corresponding processing layer.

[0035] Each layer is connected to the other layers by a plurality of synapses. A synaptic weight is associated with each synapse, and each synapse forms a link between two neurons. This is often a real number, which takes on both positive and negative values.

[0036] Each neuron is capable of performing a weighted sum of the value(s) received from the neurons of the preceding layer, each value being multiplied by the respective synaptic weight of each synapse, or connection, between said neuron and the neurons of the preceding layer, then applying an activation function, typically a non-linear function, to said weighted sum, and delivering the output The value resulting from the application of the activation function is transmitted to the neuron, and in particular to the neurons in the next layer connected to it. The activation function introduces non-linearity into the processing performed by each neuron. The sigmoid function, the hyperbolic tangent function, and the Heaviside function are examples of activation functions.

[0037] As an optional complement, each neuron is also capable of applying, in addition, a multiplicative factor, also called bias, to the output of the activation function, and the value delivered at the output of said neuron is then the product of the bias value and the value from the activation function.

[0038] A convolutional neural network is also sometimes called a convolutional neural network or by the acronym CNN, which refers to the English term "Convolutional Neural Networks".

[0039] In a convolutional neural network, each neuron in the same layer has exactly the same connection pattern as its neighboring neurons, but at different input positions. The connection pattern is called the convolutional kernel or, more often, the "kernel" in reference to the corresponding English term.

[0040] A fully connected layer of neurons is a layer in which the neurons of said layer are each connected to all the neurons of the preceding layer.

[0041] Such a type of layer is more often referred to by the English term "fully connected", and sometimes designated by the name "dense layer".

[0042] A neural network architecture or neural network topology is the structure and parameters of the neural network, including the number of layers, the type of layers, the weights, the operations that make up the neural network, and / or their arrangement within the neural network. For example, the architecture of a neural network includes a description of the operations that make up the neural network and their arrangement within that network.

[0043] [Fig.2] [Fig.2] is a schematic flowchart of a process according to a mode of Implementation, notably a HW-NAS process. In a first step, an algorithmic task is obtained. The task is, for example, an image processing task, such as image recognition. In one embodiment, the algorithmic task is the semantic segmentation of aerial images. Therefore, in one embodiment, the neural network is suitable for an image processing application.

[0044] During the process, a hardware configuration space 102 of the target hardware is defined or obtained. The hardware configuration space 102 depends on the target hardware 20 and, in some cases, also on the neural space 104.

[0045] In one embodiment, the hardware search space describes the various hardware resources that can be used for the physical execution of the neural network on the hardware target 20. By way of further examples, the hardware search space may include accelerator design choices such as the size of the activation buffer, tiling choices, the distribution of neural network tasks over the general-purpose processor(s) 22 and the accelerator(s) 24, dynamic voltage and frequency scaling (DVFS), etc.

[0046] During the process, a neural search space 104 is defined or obtained according to the algorithmic task. The neural search space 104 is defined according to the algorithmic task.

[0047] The neural search space 104 describes the possible architectures for a neural network. The neural search space 104 includes, for example, the number of layers in the network, the dimensions of the activations for each layer, the connectivity of the layers, the types of network layers at particular locations in the network (for example, convolutional layers, pooling layers, fully connected layers, recurrent layers, etc.), and the representation of the inputs and / or outputs for the neural network.

[0048] In one embodiment, the neural network is a complex neural network, for example, a neural network of the Fasterseg, FastNAS, Yolo, or BiSeNet v2 type. A complex neural network has a more complex architecture than neural networks for image classification. For example, a complex neural network allows for a multi-scale representation of information with more layers of operations or with parallel arrangements of the layers, particularly for dense predictions. In one embodiment, a dense prediction includes at least one prediction for each pixel.

[0049] During the process, evaluation criteria 106 to be optimized by an algorithm 108, in particular a multi-objective genetic algorithm, are also defined. The evaluation criteria 106 include at least one hardware performance and at least one algorithmic performance.

[0050] Hardware performance includes, for example, latency, energy efficiency, energy, memory footprint, hardware resource utilization rate (including the utilization rate of the general-purpose processor(s) and accelerator(s), and / or average energy power.

[0051] Algorithmic performance is, for example, the average of the intersection over the union. The intersection over the union (loU) is the area of ​​overlap between the predicted segmentation and the ground truth divided by the area of ​​union between the predicted segmentation and the ground truth. field. Alternatively, this metric is also called the "Jaccard index." For a neural network, the mean loU (mloU or "mean intersection over union") can be calculated. Other examples are Fl score, precision, recall, loU per class, mAP ("mean average precision"), etc. The algorithm for determining algorithmic performance depends on the algorithmic task, particularly in image processing. Precision is the proportion of correctly labeled elements (among the elements present or absent in that class). Recall is the proportion of correctly labeled elements in a class among all the elements actually present in that class.

[0052] In one embodiment, the genetic algorithm tries to maximize the mloU, minimize the latency and / or minimize the average electrical power.

[0053] The genetic algorithm 108, in particular the multi-objective genetic algorithm, is an optimization algorithm for exploring different possible neural network architectures on different hardware configurations of the target hardware and for selecting the neural network architecture with a hardware configuration that produces the best results on the target hardware according to defined criteria 106.

[0054] In one embodiment, the genetic algorithm 108 is a multi-objective genetic algorithm, for example, an NSGA II (Non-Dominated Sorting Genetic Algorithm II) algorithm. Other examples of a multi-objective genetic algorithm are NSGA-III, R-NSGA-III, C-TAEA (two-archive evolutionary algorithm), and SMS-EMOA (S-metric selection evolutionary multiobjective optimization algorithm).

[0055] During each iteration of the genetic algorithm 108, a specific neural network architecture and a specific hardware configuration are selected for a performance evaluation 110 according to the evaluation criteria 106. The errors mentioned in [Fig.2] are the algorithmic performance and the hardware performance to be optimized.

[0056] According to one embodiment, the hardware performance and the algorithmic performance are evaluated in parallel, in particular by an evaluation of the hardware performance 110a and by an evaluation of the algorithmic performance 110b.

[0057] [Fig. 3] [Fig. 3] is a schematic flowchart of the communication between a computer managing the HW-NAS stream and the hardware target, particularly during the execution of the process according to one embodiment. Steps 102, 104, 106, and 108 of [Fig. 2] are managed in the computer by processor 12 and in block 101.

[0058] [Fig.4] [Fig.4] is a flowchart of part of a process according to an embodiment and shows the evaluation of the material performance 110a and the evaluation of the algorithmic performance 110b in parallel.

[0059] For the hardware performance evaluation 110a, the neural architecture and hardware configuration are provided. In one example for the hardware performance evaluation, an untrained neural network is used.

[0060] During the hardware performance evaluation 110a, it is determined, in step 112, whether a predetermined performance predictor has an accuracy above a reliability threshold for a hardware configuration to be evaluated.

[0061] For example, the reliability threshold is an error of less than 30%, in particular less than 25%, and / or a correlation, for example measured with a Kendall's ratio, greater than 0.6, for example greater than or equal to 0.7. For example, the reliability of a performance predictor for a hardware configuration is predefined. For example, the error and correlation are determined during tests with several complex neural networks by comparing their performance as determined by the performance predictor with their performance measured during an actual deployment on a hardware target. In addition, the predictor must be faster than a direct deployment on the hardware target. For example, a prediction with a LUT table may take 2 seconds, compared to 10 minutes or more for a direct deployment on the target.The results are, for example, stored in a database, such as a table, to be consulted during step 110a.

[0062] The predetermined performance predictor is, for example, a lookup table. For example, a lookup table includes, for each operation of the neural network or part of a neural network, a measure of hardware performance. When a lookup table makes a prediction, the operations and parts of the neural network and their performances are consulted to calculate hardware performance.

[0063] Other performance predictors can also be used, for example, a regression predictor, a high-level model, such as a transactional model, a Register Transfer Level (RTL) model, or an analytical model. In one embodiment, a high-level model is a model with a significant but sufficient abstraction from the hardware target to predict its performance. For example, a high-level model only considers memory transfers, particularly of DRAM (Dynamic Random Access Memory), and calculations with the most operations, such as convolutions. A high-level model is, in one embodiment, for example, a behavioral or abstract model.

[0064] In one embodiment, the look-up-table is particularly suited for complex neural network models, for example models of the FasterSeg type.

[0065] [Fig. 5] [Fig. 5] shows the performance of a performance predictor, here a Look-up table, for a hardware configuration. In this case, the hardware target is 20. two accelerators, namely a DLA and a GPU. When the neural network is fully executed on the DLA or GPU, the predictor has minimal error compared to an actual execution on the target hardware.

[0066] We return to [Fig.3] [Fig.3] and [Fig.4] [Fig.4]. If the predetermined performance predictor has an accuracy above the reliability threshold, the hardware performance evaluation is carried out with the predetermined performance predictor (step 114), in particular on computer 10.

[0067] If the predetermined performance predictor has an accuracy below the reliability threshold, the neural network is deployed on the target hardware and the performance is measured on the target hardware (step 116), see arrow in [Fig. 3] towards the target hardware 20. The neural network is then deployed on the target hardware 20 and executed. The performance is measured and returned to the computer 10, specifically in step 110a.

[0068] For example, in one embodiment, if a look-up table is used as a performance predictor, and if the neural network is placed entirely on one of the hardware target accelerators, the look-up table is used, and if the neural network is placed on both a first accelerator 22 and a second accelerator 24, direct evaluation on the hardware target is used (Hardware-in-the-Loop).

[0069] In step 118 we then obtain the hardware performance, in particular the latency and / or the average energy power.

[0070] For the algorithmic performance evaluation 110b, the neural network architecture is provided. In step 120, a complete training of the neural network is performed, for example on an accelerator cluster or on the computer 10. A complete training of the neural network is training until convergence towards a satisfactory or stable training error. In one embodiment, this error corresponds to the algorithmic performance.

[0071] Next, in step 122, the algorithmic performance is determined, for example the average of the intersection over the union is calculated.

[0072] We return to [Fig.2] [Fig.2]. The result of the evaluations is provided to the genetic algorithm 108 (error arrow).

[0073] The genetic algorithm performs a plurality of iterations to converge and find the best neural architecture 124 and the best hardware configuration 126. For example, at least several thousand iterations are performed. The number of iterations is specifically set by a user.

[0074] Depending on the complexity of the neural space and the possible hardware configurations, the method evaluates and compares a plurality of neural architectures and different candidate hardware configurations in order to converge towards a set of satisfactory solutions (Pareto front).

[0075] Next, in step 128, the best neural architecture 124 and the best hardware configuration 126, which corresponds to the neural network with the optimal evaluation criteria, is deployed on a hardware target for use (inference). In the field of artificial intelligence (AI), inference corresponds to the process that a trained neural network model uses to draw conclusions from new data after training.

[0076] In one embodiment, the material target is used in a vehicle, for example an aircraft vehicle.

[0077] The hardware performance predictor accelerates performance evaluation compared to execution on target hardware 20, while introducing a measurement error that can impact the convergence of the automated process for generating an optimized neural network architecture that takes hardware constraints into account. Depending on the embodiment, the parallel use of a performance predictor and deployment on the target hardware for evaluation significantly accelerates the NAS process. For example, performance evaluation is reduced by 33% with the parallel use of a performance predictor and performance determination on the target hardware.

Claims

Demands

1. A method for generating an optimized neural network architecture taking into account the hardware constraints of a hardware target (20), the hardware target (20) comprising at least one accelerator (24) and at least one general-purpose processor (22), the method comprising: - Obtaining an algorithmic task (100); - Defining a neural search space (104) as a function of the algorithmic task (100); - Defining a hardware configuration space (102) of the hardware target, the hardware configuration space being dependent on the hardware target (20);- Searching for a neural network architecture for the hardware target using a genetic algorithm (108) to optimize evaluation criteria (106), the evaluation criteria (106) being at least one hardware performance and at least one algorithmic performance, the hardware performance(s) (110a) and the algorithmic performance(s) (110b) being evaluated in parallel; and - Evaluating the hardware performance(s) by using a performance predictor if the performance predictor has a reliability above a reliability threshold or the hardware performance(s) being determined by a performance evaluation by running the neural network on the hardware target (20) if the performance predictor has a reliability below the reliability threshold.

2. A method according to claim 1, wherein the neural network is a convolutional neural network, in particular compatible with dense predictions requiring a multi-scale representation of information.

3. A method according to claim 1 or 2, wherein the neural network architecture is the structure and parameters of the neural network, in particular the number of layers, the type of layers, the weights, the operations that make up the neural network and / or their arrangement within the neural network.

4. A method according to any one of the preceding claims, wherein the hardware accelerator is a graphics processing unit (GPU), a tensor processing unit (TPU), and / or a deep learning accelerator (DLA).

5. A method according to any one of the preceding claims, wherein the algorithmic task is an image processing application.

6. A method according to any one of the preceding claims, wherein the hardware performance is the latency, the average power energy, and / or the algorithmic performance is the average of the intersection over the union.

7. A method according to any one of the preceding claims, wherein the performance predictor is a look-up table.

8. A method according to any one of the preceding claims 1 to 6, wherein the performance predictor is a high-level model of the hardware target (20).

9. A method according to any one of the preceding claims 1 to 6, wherein the performance predictor is an analytical model based on a neural network.

10. A method according to any one of the preceding claims, the method further comprising deploying the neural network having the optimal evaluation criteria on the target material (20).

11. Use of the hardware target configured according to the method according to any one of the preceding claims for image processing, in particular image recognition, object detection or semantic image segmentation.

12. A system enabling the configuration of a neural network architecture for a hardware target, the hardware target comprising at least one accelerator and at least one general-purpose processor, the system comprising at least one computer, the computer being configured to be connected to the hardware target, the computer being suitable for implementing a method according to any one of claims 1 to 10.

Images