Method for optimising a neural network for use on a microcontroller

Abstract

A method for optimising a neural network for use on a microcontroller is disclosed. In the method, parameter data are optimised, which data are used when compressing an initial neural network, when generating, and preferably optimising, a program code for the compressed neural network, and when compiling the generated program code in order to obtain a compiled neural network. The invention further relates to a computing unit configured to carry out the method.

Description

[0001] Description

[0002] Method for optimizing a neural network for use on a microcontroller

[0003] The present invention relates to a method for optimizing a neural network for use on a microcontroller, in particular a vehicle control unit. Preferably, the neural network is used as a virtual sensor. The invention also relates to a control unit configured and programmed to execute the method.

[0004] Modern mechatronic systems require the execution of complex control algorithms. Examples of such systems include an electric vehicle axle with integrated components such as an electric motor, transmission, power electronics, and thermal management system; a steer-by-wire system; and an intelligent robot gearbox. Numerous sensor data points are necessary for the reliable control of these systems. The use of so-called virtual sensors, which determine a target variable from other measured variables (i.e., measured sensor data) through simulation, can reduce costs and increase reliability. In the application of thermal management in an electric vehicle axle, it has been shown that classical simulation methods, when used as virtual sensors, do not meet the runtime requirements necessary for effective control.

[0005] Therefore, highly optimized neural networks should be used as "Reduced Order Models" as a replacement. Deep neural networks, in particular, typically have memory and computing requirements that are too high for vehicle ECUs. This is due to the limited hardware specifications for the vehicle ECU, imposed for cost reasons, and the resulting high utilization of the microcontroller. Furthermore, neural network models are usually developed in programming languages ​​(primarily Python) that are not directly executable on production hardware for vehicle applications. Therefore, in addition to optimization, the models must be translated into C / C++ code for subsequent application to the target architecture. It is thus necessary to develop the neural networks using programming languages ​​and software libraries commonly used in data science (e.g., Python ...to transform TensorFlow, pytorch) with high memory and computing requirements into an optimized form suitable for execution on the highly resource-constrained serial hardware (e.g. Infineon AUTRIX ThCore microcontroller) commonly used in the automotive context.

[0006] The transfer of a neural network to a vehicle electronic control unit (ECU) comprises three phases: In the first phase, the network architecture is typically adapted. In the second phase, C / C++ code is generated. In the third phase, the generated C / C++ code is compiled for the target architecture of the vehicle ECU and applied to it.

[0007] In principle, optimizations to reduce the memory requirements and / or the runtime for the execution of the neural network are possible in all three phases:

[0008] In the first phase, methods for so-called network compression, such as pruning, quantization, and weight clustering, can be applied. These methods are discussed, for example, in Mishra et al. 2020: “A Survey on Deep Neural Network Compression: Challenges, Overview, and Solutions”, https: / / arxiv.org / abs / 2010.03954.

[0009] In the second phase, general code structure optimization techniques (e.g., loop tiling, loop fusion) can be applied; furthermore, specific adaptations are possible to exploit particular hardware characteristics (e.g., distributing parallel operators of a neural network across different processor cores; utilizing any available tensor processing unit (e.g., the parallel processing unit in the Aurix TC4x); exploiting specific memory hierarchies) and / or specific adaptations to exploit neural network evaluation features (e.g., operator fusion). In the third phase, further optimizations can be performed within the compiler.

[0010] For the rapid and efficient implementation of a neural network on a vehicle control unit, an integrated and, as far as possible, automated approach is required. Various methods for optimizing and compiling neural networks have therefore already been proposed in the literature. Regarding the second and third phases, a distinction must be made between methods with and without the use of so-called inference libraries:

[0011] An inference library provides an efficient implementation of individual operators for neural networks. Using such libraries on a microcontroller means additional memory overhead, which must be compensated for through optimizations.

[0012] Methods using hardware-independent inference libraries generate generally usable ANSI C or C++ code. This means that more in-depth, hardware-specific optimization possibilities are disregarded. Examples of this are revealed in Ma: “A higher-level Neural Network library on Microcontrollers”, https: / / github.com / majianjia / nnom, David et al. 2020: “Tensorflow lite micro: Embedded machine learning on tinyml systems”, CoRR, abs / 2010.08678 and Deutel et al. 2022: “Deployment of energy-efficient deep learning models on Cortex-M based microcontrollers using deep compression”, CoRR, abs / 2205.10369.

[0013] Currently available methods using hardware-specific inference libraries that exploit such optimization possibilities are primarily geared towards ARM- and RISC-V-based platforms (see, for example, Lin et al. 2020: “Mcunet: Tiny deep learning on IoT devices”, Annual Conference on Neural Information Processing Systems (NeurlPS), virtual, 2020, and Chen et al. 2018: “TVM: an automated end-to-end optimizing compiler for deep learning”, 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI), pp. 578-594, Carlsbad, USA, 2018). In a method without an inference library, code generators avoid the additional memory requirements incurred by including an inference library and allow for greater adaptation to the specific target architecture. They are currently available for RISC-V platforms (see, for example, Burrello et al. 2021: “DORY: automatic end-to-end deployment of real-world DNNs on low-cost IoT mcus”. IEEE Trans.Computers, 70(8):1253-1268).

[0014] Some of the methods described above for the second and third phases also support optimization in the first phase (e.g., quantization and weight clustering in David et al. 2020: “Tensorflow lite micro: Embedded machine learning on tinyml systems”, CoRR, abs / 2010.08678, pruning and quantization in Deutel et al. 2022: “Deployment of energy-efficient deep learning models on Cortex-M based microcontrollers using deep compression”, CoRR, abs / 2205.10369).

[0015] The aforementioned methods have in common that they can largely automate the optimization of neural networks, ensuring that the memory requirements and runtime specifications of the target hardware are met. The sequential, automated execution of the first, second, and third phases is known, for example, from DE 102022 002 695 A1.

[0016] Both the methods from non-patented literature and the method from patent literature overlook the fact that the methods particularly suitable for the first phase remove information from the neural network (e.g., through rounding during quantization or by removing neurons or entire layers from the network). This increases the error rate, which in turn leads to a decrease in the quality of the neural network's predictions.

[0017] Applying the known methods thus leads to a compiled neural network that, while meeting the requirements of an electronic control unit (ECU) in terms of memory usage and runtime, may no longer provide the necessary predictive accuracy. In this case, manual modifications to the method must first be made (e.g., deactivating various optimization stages; adjusting parameters such as the pruning rate, which defines the extent to which neurons should be removed from the network, etc.). The neural network can then be reprocessed with the newly parameterized method, and the quality of the result can be checked. This process may need to be repeated several times and is therefore time-consuming and costly.Furthermore, this "trial and error" approach does not guarantee that the optimal solution (maximum quality with minimal or predetermined maximum storage space and runtime requirements) will be found.

[0018] Based on this, the object of the present invention is to provide a method for optimizing a neural network for use on a microcomputer, in particular a vehicle control unit, which is improved compared to the prior art.

[0019] This problem is solved by the articles with the features according to the independent claims. Advantageous embodiments are the subject of the dependent claims.

[0020] Revealbart is a method for optimizing a neural network for use on a microcontroller, in particular a vehicle control unit.The procedure comprises the following steps: providing an initial neural network; providing initial parameter data; compressing the initial neural network taking the parameter data into account; generating, and preferably optimizing, program code for the compressed neural network taking the parameter data into account; compiling the generated program code taking the parameter data into account to obtain a compiled neural network; running the compiled neural network with test data on the microcontroller and / or a runtime environment simulating the microcontroller to determine performance data; inputting the performance data into an objective function; inputting an output value of the objective function into a search algorithm; and providing further parameter data.The procedure is executed taking into account the additional parameter data instead of the initial parameter data until convergence is achieved in the search algorithm.

[0021] By executing this procedure, the parameter data considered during the compression of the initial neural network, the generation of the program code for the compressed neural network, and the compilation of the generated program code are optimized. Consequently, the configuration of the compression, generation, and compilation steps can be fully automated, resulting in a neural network optimized for the microcontroller.

[0022] The performance data can include an error rate value, a memory requirement value, and a runtime value. Consequently, the network obtained through the procedure is optimized with respect to these parameters.

[0023] Accordingly, the procedure is run with specific parameter data, and the error rate value, the memory requirement value, and the runtime value are automatically determined for the test data on the microcontroller or the runtime environment and evaluated in the search algorithm to determine improved parameter data.

[0024] The objective function can consider a subset of the performance data values ​​as sub-objectives. The other values ​​can be considered as constraints. For example, the error rate can be used as the target value in the objective function. Memory requirements and runtime are defined as constraints that must not be exceeded. Alternatively, two of the performance data values, such as error rate and runtime, can be considered as sub-functions in the objective function. In this case, the memory requirements are used to define the constraint.

[0025] The defined constraints can be taken into account by the search algorithm when determining the additional parameter data. This limits the specification of the further parameter data to an appropriate range, ensuring that the constraints are reliably met and avoiding unnecessary iterations of the process steps.

[0026] The objective function can consider the values ​​of the performance data as sub-objectives. Consequently, the values ​​of the other parameter data are determined using multi-objective optimization or Pareto optimization.

[0027] This allows for consideration of potentially strict requirements of the individual application case, compliance with which could not be reliably guaranteed when searching for the Pareto-optimal solution across all three values ​​of the performance data (e.g. minimizing / balancing error rate and runtime while simultaneously adhering to a fixed, predetermined memory quota still available on the microcontroller).

[0028] Furthermore, the search algorithm can be a gradient-free optimization method. Examples include mixed-integer optimization approaches, evolutionary optimization, or Bayesian optimization.

[0029] The process can further include outputting the remaining parameter data upon reaching convergence. This output parameter data can then be applied to other neural networks to be implemented on a suitable microcontroller.

[0030] Additionally or alternatively, the procedure can also include providing the compiled neural network upon reaching convergence. The compiled neural network is then ready for implementation on the microcontroller, i.e., the vehicle's control unit.

[0031] Also disclosed is a computing unit that is trained and programmed to execute the disclosed procedure.

[0032] The present invention is described in detail below with reference to the figure. Figure 1 shows a schematic flowchart of a method for optimizing a neural network for use on a microcontroller according to an embodiment of the present invention.

[0033] The present invention is described below with reference to preferred embodiments and the figure. However, this description of embodiments should not be considered exhaustive.

[0034] Figure 1 shows a schematic flowchart of a method for optimizing a neural network for use on a microcontroller. The microcontroller can be a vehicle control unit. The neural network can be used, in particular, as a virtual sensor. That is, acquired or measured sensor data is fed into the neural network, and the neural network provides calculated or simulated sensor data.

[0035] In step 110, an initial neural network is provided, which is to be optimized for use on a microcontroller. The neural network has a specific network architecture, and the weights of the neural network have already been trained.

[0036] In step 120, initial parameter data is provided. This parameter data includes values ​​for (de)activating individual compression methods, as well as for the parameters of each method (e.g., pruning rate, bit depth of quantization, etc.). The initial parameter data can be set manually beforehand. Alternatively, the parameter data can be initialized by disabling all grain priming methods.

[0037] In step 130 (corresponding to the first phase described at the beginning), the neural network is compressed at the network level. Various compression methods (e.g., pruning, quantization, weight clustering) are applied to the neural network sequentially in a modular fashion. This approach allows for future expansion to include newly developed compression methods. The current selection of compression methods to be applied, as well as the parameters for each method (e.g., pruning rate, quantization bit depth, etc.), are contained in the parameter data.

[0038] In step 140 (corresponding to the second phase described at the beginning), code generation and, preferably, code optimization take place. In this step of the process, the neural network is translated into C or C++ code. For this purpose, the neural network can first be converted into the standard exchange format Open Neural Network Exchange (ONNX). For example, the translation from this format can be performed using a code generator specifically designed for microcontrollers of the Infineon AURIX TriCore architecture. During the translation, further optimizations, which are enabled or disabled in the parameter data, can be applied. In particular, the code structure can be adapted to the specific requirements of subsequent AUTOSAR embedding. The intermediate step via the standard exchange format allows the neural network, compressed in step 130, to be extracted and further processed using other methods (e.g.,...).for model-based development using MATLAB / Simulink).

[0039] In step 150 (corresponding to the third phase described at the beginning), the generated program code for the neural network is compiled. The generated C or C++ code is thus translated into a binary file, the so-called compiled code, for the microcontroller. This compilation can be performed using proprietary tools from the microcontroller manufacturer (e.g., HighTec compilers). Alternatively, the code can be integrated into other applications, such as MATLAB / Simulink, and processed there automatically (e.g., with MATLAB Embedded Coder).

[0040] In step 160, the binary file, i.e., the compiled neural network, is executed or evaluated on the microcontroller. For this purpose, test data is input in step 161. The test data includes data that is input into the neural network and labels that, ideally, are output by the neural network. The test data thus corresponds to ground truth data.

[0041] In step 162, performance data is determined during the execution of the neural network. This performance data can include an error rate value, a memory requirement value, and a runtime value. The neural network can be run directly on the microcontroller to determine the performance data. Alternatively or additionally, execution can also take place on a runtime environment that simulates the microcontroller, i.e., an emulator.

[0042] In step 163, the specified performance data is entered into an objective function. A subset of the performance data values, e.g., the error rate, can be optimized as sub-objectives of the objective function, specifically by minimizing them. Furthermore, compliance with the other performance data values, e.g., the memory requirement and runtime, can be ensured by establishing constraints. Alternatively, all performance data values ​​can be minimized as sub-objectives within the objective function. The objective function thus corresponds to a multi-objective optimization or Pareto optimization.

[0043] The constraints can be entered into a search algorithm described below in step 180.

[0044] In step 170, the output value of the objective function is fed into a search algorithm. The search algorithm controls the iterative execution of the procedure. Based on the output value of the objective function and taking into account the constraints (see step 180), the search algorithm determines further parameter data and triggers the re-execution of the subsequent steps of the procedure from step 130 onwards, considering this additional parameter data. The parameter data can have binary values ​​(e.g., pruning enabled / disabled) or continuous values ​​(e.g., pruning rate 12.5%). Each determination of the performance data on the target architecture evaluates the corresponding parameter data for the search procedure.

[0045] A gradient-free optimization method can be used as the search procedure.

[0046] Mixed-integer optimization approaches that support "black-box objective functions" are suitable for this purpose. Instead of using an explicitly formulated functional relationship between the parameter data to be optimized and the objective function, "black-box objective functions" work with several specific performance data points.

[0047] Evolutionary optimization can also be performed.

[0048] This also includes Bayesian optimization with coding of the parameter data analogous to evolutionary optimization and consideration of the objective function as a Gaussian process, which is modeled based on several specific performance data.

[0049] If convergence is not achieved, the additional parameter data for steps 130, 140, and 150 are provided in step 171. The procedure is then repeated with the provided parameter data.

[0050] Upon convergence of the search method, instead of iterating the process again, the optimized neural network and / or the additional parameter data are output in step 190. This additional parameter data can then be applied to other neural networks intended for implementation on a corresponding microcontroller. The optimized neural network is then ready for implementation on the microcontroller. The method described above can be used to optimize a neural network used for the control of one of the following systems: an electric vehicle axle with integrated components, a steer-by-wire system, or an intelligent robot gearbox. In particular, the optimized neural network can be used as a "virtual sensor" that provides or predicts unacquired sensor data based on acquired sensor data.

[0051] Numerous applications are conceivable in the automotive sector: thermal regression analysis, replacement of the rotational position sensor, general condition monitoring, lifetime estimation, classification, fault detection, predictive maintenance, control and strategy algorithms with reinforcement learning, thermal management, and BLDC motors. Furthermore, applications in the industrial sector are also possible.

[0052] Robotic torque sensor and sensorized robot gearbox with neural networks for torque estimation from raw signals.

[0053] Reference symbol list

[0054] 100 methods for optimizing a neural network

[0055] 110 Provisioning an initial neural network

[0056] 120 Providing initial parameter data

[0057] 130 Processing the initial neural network

[0058] 140 Generating program code for the compressed neural

[0059] network

[0060] 150 Compiling the generated program code

[0061] 160 Running the compiled neural network with test data

[0062] 161 Providing test data

[0063] 162 Determining performance data

[0064] 163 Entering performance data into an objective function

[0065] 170 Entering an output value of the objective function into a

[0066] search algorithm

[0067] 171 Providing additional parameter data

[0068] 180 Providing constraints

[0069] 190 Provision of the compiled neural network and / or the additional parameter data

Claims

Claims 1. Method for optimizing a neural network (100) for use on a microcontroller, comprising the following steps: Providing an initial neural network (110); Providing initial parameter data (120); Compressing the initial neural network (130) taking into account the parameter data; Generating, and preferably optimizing, a program code for the compressed neural network (140) taking into account the parameter data; Compiling the generated program code (150) taking into account the parameter data to obtain a compiled neural network; Running the compiled neural network with test data (160) on the microcontroller and / or a runtime environment simulating the microcontroller to determine performance data (162); Inputting the performance data into an objective function (163); Inputting an output value of the objective function into a search algorithm (170) and determining further parameter data; and Providing the additional parameter data (171) wherein the procedure is executed taking into account the additional parameter data instead of the initial parameter data until convergence is achieved in the search algorithm (170).

2. Method according to claim 1, characterized in that the performance data includes an error rate value, a memory requirement value and a runtime value.

3. Method according to claim 2, characterized in that the objective function comprises a subset of the values ​​of the performance data as sub-objective functions and the other values ​​are considered as constraints (180).

4. Method according to claim 3, characterized in that the constraints are taken into account by the search algorithm when determining the further parameter data (180).

5. Method according to claim 2, characterized in that the objective function takes into account the values ​​of the performance data as sub-objective functions.

6. Method according to one of claims 1 to 5, characterized in that the search algorithm is a gradient-free optimization method.

7. Method according to one of claims 1 to 6, characterized in that the method further comprises outputting the additional parameter data (190) upon reaching convergence.

8. Method according to one of claims 1 to 7, characterized in that the method further comprises providing the compiled neural network (190) upon reaching convergence.

9. Computing unit that is trained and programmed to execute the method according to any one of claims 1 to 8.

Images