A method and apparatus for memory planning for code generation to create program code for computing artificial neural networks in a hardware environment.

Abstract

We will perform memory planning for the code used to compute the neural network. [Solution] The method includes step S1 of providing a series of computation steps of multiple layers of a neural network. For each computation step, the size of one or more model parameter blocks is determined, and one or more model parameter blocks have model parameters for each computation step. The method also includes a step of determining rules for memory planning for each specific computation step that requires the use of model parameters. The rules indicate that the model parameters are loaded into the relevant memory area in the main memory of the hardware environment. The method further includes step S2 of performing memory planning in which the memory areas of the respective input data blocks, output data blocks, and model parameter blocks for each computation step are determined taking into account the determined rules.

Description

Technical Field

[0001] The present invention relates to the implementation of program code on a hardware environment that exists, for example, as a microcontroller-controlled control device. The present invention further relates to a method for memory planning for handling input data, output data, and model parameters.

Background Art

[0002] Background Art A specific hardware environment, for example, a microcontroller in a control device, requires the creation of adapted executable program code in order to take into account the characteristics and limitations of the specific hardware environment. Therefore, in particular, the amount of memory available in the main memory that can be directly accessed by the microcontroller or the acceleration hardware is limited, or, for example, the memory shift operation or copy operation from the data memory such as flash memory or external memory to the main memory may be particularly time-consuming due to hardware.

[0003] The calculation steps for calculating the corresponding network layer of a neural network may require a large amount of memory. This is because for each calculation step, one input data block, one model parameter block, and one output data block need to be stored in the main memory so that they can be called and are available to the microcontroller.

[0004] Existing code generators determine, during the memory planning period, in which area of the main memory the data blocks in memory required for each calculation step are stored. During the memory planning period, in addition to the input data block, the output data block, and, optionally, the model parameters being allocated to the memory area of the main memory, a corresponding memory area is also allocated to the data generated during the calculation period in the calculation step.

[0005] Conventional neural network code generators generally do not assume limited main memory, and typically allocate distributed memory regions for storing input data blocks, network parameter blocks, and output data blocks for each computation step performed sequentially. Therefore, until now, it has been common practice to freely distribute model parameters within available memory in order to minimize overall memory demand. However, this procedure can lead to the need to copy model parameters in segments to various memory regions of main memory before executing computation steps.

[0006] In particular, copying memory regions from data memory to main memory is typically a time-consuming memory operation, even between memory regions within main memory. Therefore, memory planning must aim to reduce the total computation time caused by the execution time of memory operations. [Overview of the Initiative] [Problems that the invention aims to solve]

[0007] The object of the present invention is to provide improved memory management for computation of artificial neural networks that can reduce the number of memory operations. [Means for solving the problem]

[0008] Disclosure of the invention This problem is solved by the method for performing memory planning for code generation of code for computing a neural network, as described in claim 1, and by the apparatus according to the claim for parallel relations.

[0009] Further configurations are described in the dependent claims.

[0010] According to the first aspect, a computer-implemented method for performing memory planning for code generation to determine code for computing a neural network in a hardware environment includes the following steps: - A step providing sequential computation steps of multiple layers of a neural network, wherein for each computation step, the size of one input data block, the size of one output data block, and the size of one or more model parameter blocks are determined depending on the type of computation step, and one or more model parameter blocks have model parameters for each computation step, and - A step of determining a rule for memory planning for each specific computation step that requires the use of model parameters, wherein the rule indicates that the model parameters are loaded into the relevant memory region in the main memory of the hardware environment. - For each calculation step, a memory planning step is performed in which the memory areas of the respective input data blocks, output data blocks, and model parameter blocks are determined in main memory, taking into account the determined rules. Includes.

[0011] In particular, performing memory planning may involve using an optimization method whose objective function aims to minimize the number of memory operations. Furthermore, performing memory planning may involve using an optimization method whose objective function aims to minimize the total memory demand in main memory.

[0012] The individual computation steps for computing the computational layers of a neural network are typically computed serially in the hardware environment. This means that the code for the generated hardware environment determines the order in which the input data is processed and the output data for each computation step is generated. Each computation step extracts input data from an input data block and stores the resulting output data in one or more output data blocks. The input and output data blocks represent memory regions in main memory, and these memory regions are located within the relevant address space.

[0013] The hardware environment includes a computing unit, main memory, and data memory. The generated code for computing the neural network is executed in the form of computation steps in the computing unit, in which case input and output data blocks in main memory are used to provide input data and store output data. Access to data from main memory has a short access time, while access to data memory requires a relatively long access time. Memory planning aims to reduce the access time or overall computation duration by providing and arranging input and output data blocks in main memory, while also considering or limiting the maximum available memory capacity of main memory.

[0014] Depending on the computational steps that must be performed to compute the neural network, memory operations often occur between layers of computation. These operations include copying to main memory or shifting memory regions within main memory. Memory operations are often time-consuming and involve a significant time component that is independent of the size of the memory region to be copied or shifted.

[0015] The core of the above method lies in planning the model parameters as related data blocks in a single memory operation. In this way, only a single copy operation is required to make all model parameters available in main memory. This increases the execution speed of the neural network by reducing the required memory operations.

[0016] In addition to the input or output data blocks for each computational layer of the neural network, memory regions are also allocated during the memory planning phase. Similarly, memory regions within main memory are allocated for the parameters used for the computations of many of the computational layers.

[0017] Memory planning generally assumes minimizing the total memory demand in main memory. However, this also has the side effect that, when performing network layer calculations, the model parameters of the calculation layer are copied to main memory in segments, particularly when the available memory space between already placed input and output data blocks is smaller than the model parameter blocks to be copied in main memory.

[0018] Depending on the hardware environment, memory accesses such as copy and shift operations can increase the overall processing time; therefore, it is desirable to minimize the number of required copy operations. In particular, the maximum available memory area can be considered during memory planning.

[0019] Therefore, memory planning can determine whether the network parameters can be copied into the main memory for one or more calculation steps, considering the sizes of the input data blocks and output data blocks for each of the calculation steps. Here, an attempt is made to write the network parameters from various parameter blocks into the main memory while associating them as much as possible. This is because this can eliminate the need for more than one copy operation. This can result in significant time savings in the processing during the evaluation of the neural network in a hardware environment, especially when the data copy operation to the main memory is complex and the offset ratio in the processing time is high, that is, when the ratio of the processing time not affected by the size of the memory area to be copied is high.

[0020] After memory planning, the model parameters can perform code generation that is loaded by a single copy operation into the memory area allocated to the model parameter block by the memory planning.

[0021] In the following, embodiments will be described in more detail based on the accompanying drawings.

Brief Description of the Drawings

[0022] [Figure 1] It is a schematic diagram showing a platform for code generation and implementation in a hardware environment. [Figure 2] It is a flowchart for explaining a method for performing memory planning and code generation. [Figure 3a] It is a diagram showing a comparison between the conventional procedure for arranging the memory area for model parameters in the main memory of a hardware environment and the procedure according to the present invention. [Figure 3b] It is a diagram showing a comparison between the conventional procedure for arranging the memory area for model parameters in the main memory of a hardware environment and the procedure according to the present invention.

Best Mode for Carrying Out the Invention

[0023] Description of Embodiment FIG. 1 shows a block diagram of a platform 1 for implementing the execution of code generation and the program code generated in a hardware environment 2. The hardware environment corresponds to, for example, a control device having a microcontroller, a microprocessor, or the like. The code generation is performed on a conventional computer 3 or a workstation under the setting of the configuration of the neural network. The computer 3 is configured to perform memory planning and code generation. In this case, the memory planning first performs the arrangement of a memory area for accommodating at least one input data block, an output data block, and at least one model parameter block for each calculation step of the neural network. The model parameter block includes all model parameters required to calculate each calculation step, for example, the weights of a fully connected layer and bias values.

[0024] When the code is generated, this code is transmitted to the hardware environment 2 and implemented or executed there.

[0025] FIG. 2 shows a flowchart for explaining a method for performing memory planning and code generation for providing program code for implementing a neural network.

[0026] In step S1, first, a neural network having calculation steps is set. Each of these calculation steps defines the type of layer to be calculated in the neural network, the input data block, and the resulting output data block. For the network layer, further, model parameters used according to the type of network layer may be set to calculate the data elements of the output data block from the data elements of the input data block.

[0027] In step S2, memory planning is performed, for example, using an SMT planner, and this memory planning is iterated sequentially through calculation steps, with each calculation step involving the arrangement of input data blocks and output data blocks for model parameters, as well as model parameter blocks. The memory planning preferably corresponds to a known combinatorial optimization method.

[0028] Each calculation step typically requires the prior provision of input data blocks and, if applicable, model parameter blocks, as well as the reservation of memory space for output data blocks. Input data blocks and model parameter blocks are often loaded into main memory by a copy operation from data memory, because processing of input data blocks, output data blocks, and model parameter blocks can only be performed from main memory.

[0029] Memory planning here assumes that for one or more sequential computation steps, the necessary model parameter blocks are bundled together and copied into the relevant memory area (relevant address area) in a single copy operation, thereby minimizing the time overhead of the copy operation.

[0030] In particular, the copy operation may be assumed to copy not only the model parameters of the computation layer currently under consideration, but also the input data blocks to be copied for the next computation step, in the same copy operation, thereby copying one input data block and one or more model parameter blocks to main memory as a memory area in which they relate.

[0031] Subsequently, in step S3, only one memory operation / copy operation is assumed during the code generation period for copying the relevant memory regions, while the model parameter blocks, which are stored separately from each other, are copied to main memory by a separate copy process.

[0032] Figures 3a and 3b show, for comparison, the results of memory planning using the conventional method or the method described above. Figure 3a shows how model parameters in multiple model parameter blocks MB1, MB2, MB3 are copied in separate, spaced memory regions surrounding the output data block AB of the relevant computation step, as is the case with the conventional memory planning method.

[0033] In contrast, Figure 3b shows that the model parameter blocks MB1, MB2, and MB3 are copied into main memory in a single copy operation.

[0034] A memory copy counter can be used to optimize the processing time used for copy operations. This counter can count or count the necessary copy operations when a model parameter block MB in a given calculation step is not connected to another model parameter block MB1, MB2, MB3, and therefore cannot be copied in a single copy operation. When finding the optimal memory planning, the SMT planner can minimize the number of copy operations or minimize the number of memory operations required in the target function.

Claims

1. A computer-implemented method for performing memory planning for code generation to determine code for computing a neural network in a hardware environment (2), comprising the following steps: - A step (S1) that provides sequential computation steps for multiple layers of the neural network, wherein for each computation step (O1), the size of one input data block (EB), the size of one output data block (AB), and the size of one or more model parameter blocks (MB1, MB2, MB3) are determined depending on the type of computation step, and the one or more model parameter blocks (MB1, MB2, MB3) have model parameters for each computation step, and step (S1) - A step of determining a rule for memory planning for each specific computation step that requires the use of the model parameters, the rule indicating that the model parameters are loaded into the relevant memory region in the main memory of the hardware environment (2), - Step (S2) of performing memory planning in which, for each calculation step, the memory areas of each of the input data block (EB), output data block (AB), and model parameter blocks (MB1, MB2, MB3) are determined in the main memory, taking into account the determined rules, A method that includes this.

2. The method according to claim 1, wherein the implementation of the memory planning includes using an optimization method in which the objective function minimizes the number of memory operations.

3. The method according to claim 1 or 2, wherein the implementation of the memory planning includes using an optimization method in which the objective function minimizes the total memory demand in the main memory.

4. The method according to any one of claims 1 to 3, wherein, after the memory planning, code generation is performed in which the model parameters are loaded in a single copy operation to the memory area allocated to the model parameter block (MB1, MB2, MB3) by the memory planning (S3).

5. An apparatus for carrying out the method described in any one of claims 1 to 4.

6. A computer program product comprising, when the program is executed by at least one data processing device, instructions for causing the at least one data processing device to perform a step of the method according to any one of claims 1 to 4.

7. A machine-readable storage medium comprising, when executed by at least one data processing device, an instruction causing the at least one data processing device to perform a step of the method according to any one of claims 1 to 4.

Images