Method and apparatus for memory planning for code generation of program code

Abstract

A method and apparatus for memory planning for code generation of program code. The invention relates to a computer-implemented method for performing memory planning for code generation for determining code for neural network calculations, comprising the following steps: providing successive calculation steps of a neural network, the sizes of an input data block and an output data block being determined for each calculation step; determining, for each particular computing step, a condition of a memory plan wherein the input data chunks in the storage area are at least partially contained in the output data chunks, the condition indicates that the storage area of the output data block allocated to the previous calculation step of the specific calculation step is included in the storage area of the output data block of the specific calculation step; and executing a memory plan, taking into account the determined conditions, in which memory plan the storage area of the respective input data block and output data block in the working memory is specified for each calculation step.

Description

Technical Field

[0001] This invention relates to a method and apparatus for memory planning in code generation for program code of artificial neural network computation in a hardware environment. The invention relates to the implementation of program code in a hardware environment, such as a control unit controlled by a microcontroller. The invention also relates to a method for memory planning for processing input data, output data, and network parameters. Background Technology

[0002] Specific hardware environments, such as microcontrollers in control units, require the creation of adaptive executable code to account for the characteristics and limitations of that specific hardware environment. This can be particularly problematic in areas where the amount of available working memory directly accessible to the microcontroller may be limited, or where memory migration or copying operations from flash or external memory to working memory may be especially complex due to hardware limitations.

[0003] The computational steps used for the corresponding layers of a neural network may require a large amount of memory because, for each computational layer, the input data block, network parameter block, and output data block need to be stored in the working memory in a callable manner and in a way that can be used by the microcontroller.

[0004] Existing code generators specify during memory planning which area of ​​working memory will store the data blocks required for each computation layer. During memory planning, in addition to allocating input and output data blocks to storage areas, corresponding storage areas are also assigned to data generated during computation within the computation layer.

[0005] Traditional neural network code generators typically assume that all input and output data blocks must reside in completely separate storage areas.

[0006] However, traditional memory planning methods do not consider information about the computation layer type. Therefore, they do not consider whether the output data block of one computation step corresponds to a portion of the input data block of the next computation step. This is the case, for example, in the computation of a concatenated layer, where concatenation links two input data blocks together.

[0007] In particular, copying memory regions from flash or external storage to working memory, as well as copying between memory regions in working memory, is typically a time-consuming memory operation, making memory planning essential to reduce the processing time of memory operations. Summary of the Invention

[0008] The objective of this invention is to provide an improved memory management for artificial neural network computation, wherein the number of memory operations can be reduced.

[0009] This task is accomplished by the method for performing memory planning for code generation for neural network computation as described in claim 1, and by the apparatus as described in the parallel independent claims.

[0010] Other design options are described in the dependent claims.

[0011] According to the first aspect, a computer-implemented method is provided for performing memory planning for code generation to determine code for neural network computation, the method comprising the following steps: - Provide the sequential computation steps of the neural network, wherein, for each computation step, the size of the input data block and the output data block are determined; - Determine the conditions for the memory planning for each specific computation step, wherein the input data blocks in the storage area are at least partially contained in the output data blocks, wherein the conditions indicate that the storage area of ​​the output data block of the previous computation step allocated to the specific computation step is contained in the storage area of ​​the output data block of the specific computation step; - Taking into account the determined conditions, a memory plan is performed in which the storage areas of the corresponding input data blocks and output data blocks in the working memory are specified for each computation step.

[0012] In particular, performing memory planning can include applying optimization methods, where the objective function considers minimizing the number of memory operations.

[0013] The computational steps used in the computational layers of a neural network are typically computed sequentially on the hardware environment. This means that the generated code specifies the order in which the input data is processed and the output data for the corresponding computational step is generated for that hardware environment. Each computational step extracts input data from one or more input data blocks and stores the resulting output data in one or more output data blocks. Input data blocks and output data blocks represent memory regions in the working memory corresponding to contiguous address spaces.

[0014] The hardware environment includes a computing unit, working memory, and data memory. The generated code for neural network computation is executed in the computing unit as computational steps, where input data is provided and output data is stored using input and output data blocks in the working memory. Access to data from the working memory is quick, while access to the data memory requires more time. The goal of memory planning is to reduce access time or total computation time by providing and placing input and output data blocks in the working memory, while simultaneously considering or limiting the maximum available memory space in the working memory.

[0015] Based on the computational steps required for neural network computation, memory operations often occur between actual layer computations. These operations involve copying or migrating memory regions. These memory operations are often time-consuming and account for a significant portion of the time, regardless of the size of the memory region being copied or migrated.

[0016] Within the framework of code generation to determine the code used for neural network computation, the above method specifies the following for memory planning: for a specific computation step, such as splicing, slicing, and padding, the output data block of the previous computation step is located in the address range of the working memory so that the next computation step can be executed immediately by assigning corresponding address pointers without memory operations, or with only a small amount of memory operations.

[0017] Therefore, the sequence of two computation steps can be optimized by placing the storage area for the output data block of the previous computation step and the storage area for the input data block of the next computation step, so that no time-consuming memory operations are required between them.

[0018] In particular, this memory planning specifies that the output data block of the preceding computational step that follows a particular computational step is placed in a storage area of ​​the working memory, such that this storage area can be used, in whole or in part, as the output data block of that particular computational step. In other words, according to the specifications of this memory planning, the input data block of that particular computational step contains an immutable portion of the output data block, thereby eliminating the need for memory operations and requiring only the use of address pointers.

[0019] This memory planning can be performed in a manner known per se, using a so-called SMT solver. The memory planning begins with a list defining the successive computational steps of the neural network, and associated memory regions for the input and output data blocks, not yet allocated to working memory address ranges. These memory regions are determined solely by their size. For each pair of consecutive, not necessarily directly consecutive, computational steps, corresponding conditions are added to the SMT solver, indicating whether the output and input data blocks should be located within the same address range.

[0020] Unless the computation steps involve splicing, slicing, or padding, and the corresponding input data blocks can be stored line by line, the storage areas for the input data blocks and output data blocks of each computation step must be stored separately.

[0021] It can be specified that if the particular computation step is a concatenation step, then the condition indicates that one or more computation steps preceding the concatenation step provide output data blocks in a storage area that is adjacent to the storage area where the output data of the preceding computation step should be concatenated in the concatenation step.

[0022] It can be specified that if the particular computation step is a filling step, then the condition indicates that the preceding computation step of the filling step provides an output data block in a storage area that is adjacent to a storage area occupied by the filling mode or subsequently written with the filling mode.

[0023] It can be specified that if the particular computation step is a slice step, then the condition indicates that the preceding computation step of the slice step provides an output data block in the storage area corresponding to the output data block portion of the particular computation step.

[0024] If a particular computation step is concatenating, slicing, or padding, an attempt is made to place the output data block of the previous computation step such that the corresponding other storage areas (in the address space) of the input data block belonging to that particular computation step are placed immediately above or below the output data block of the previous computation step (concatenation).

[0025] For a specific computational step corresponding to padding, the storage area occupied by the padding pattern can be located above or below the output data block.

[0026] For a specific computation step corresponding to a slice, it is assumed that a fragment of the relevant output data block from the previous computation step is an input data block, which can be done by dispatching the corresponding address and size. Since the output data block corresponds to a portion of the input data block, no copying operation is required. Attached Figure Description

[0027] The embodiments are then described in more detail with reference to the accompanying drawings. Wherein: Figure 1 A schematic diagram of a platform for code generation and implementation in a hardware environment is shown; Figure 2 The diagram illustrates a flowchart illustrating a method for memory planning for computational steps involving splicing, slicing, or filling; and Figure 3a and Figure 3b The application is shown. Figure 2 A schematic diagram of the preferred memory planning for two consecutive computational steps of a neural network without applying the method. Detailed Implementation

[0028] Figure 1A block diagram of a platform 1 for performing code generation and implementing the generated program code in a hardware environment 2 is shown. This hardware environment corresponds, for example, to a control unit with a microcontroller, microprocessor, etc. Code generation is performed on a conventional computer 3 or workstation, with a specified neural network configuration. The computer 3 is configured to perform memory planning and code generation, wherein the memory planning first involves placing storage areas for accommodating at least one input data block and at least one output data block for each computational step of the neural network. The model parameter block contains all the model parameters required for the computation of the corresponding computational step, such as the weights and biases of fully-connected layers.

[0029] If code is generated, it is transferred to hardware environment 2 and implemented or executed there.

[0030] Within the framework of the memory planning method described below, it can now be specified that the storage areas of the input and output data blocks of a computation step are placed within the address range of the working memory, such that for a particular computation step, access is provided that at least partially overlaps with the storage area of ​​the input data block of that computation step. This significantly saves memory space in the working memory.

[0031] Figure 2 This illustration demonstrates the process of memory planning using an SMT solver.

[0032] In step S1, the successive computation steps for neural network computation are first specified. These computation steps are each allocated an input data block and an output data block as a storage area of ​​a defined size.

[0033] In step S2, specific computation steps are identified, which include splicing, slicing, or padding. These specific computation steps have the following characteristics: at least a portion of the output data block of the previous computation step is contained in or completely corresponds to the input data block.

[0034] From this, the conditions for specific computation steps can be derived from the memory plan.

[0035] These conditions can be: - For the splicing step, the preceding computation step should provide the output data block in the storage area that is adjacent to the storage area where the output data of the preceding computation step should be connected. - For a fill step, the preceding computation step should provide an output data block in a storage area that is adjacent to a storage area occupied by the fill pattern or that can be written to by the fill pattern. - For the slicing step, the slicing step should specify the relevant input data block in the address range corresponding to the selectable portion of the output data in the output data block of the previous calculation step. - For other types of computation steps, the storage areas for input data blocks and output data blocks should be placed in separate storage areas.

[0036] Each storage region is assigned a lifetime, which specifies how long the data elements in that storage region must not be overwritten, and in particular, how many consecutive computation steps, and thus indicates that the storage region is occupied until it is no longer needed.

[0037] In step S3, memory planning is performed using the SMT solver. The goal of optimization is to reduce the number of time-consuming memory operations.

[0038] Here, for each computation step preceding one of the aforementioned computation steps, the output data block is positioned such that the data storage area can be expanded in association with that particular computation step. This is either an additional storage area in the case of concatenation, which should be connected to the storage area of ​​the output data block of the previous computation step, or, in the case of padding, i.e., adding padding patterns above and below the storage area of ​​the output data block of the previous computation step.

[0039] In the case of slicing, the input data block for that particular computation step is defined as a subset of the storage area of ​​the output data block of the previous computation step.

[0040] Especially when working memory is limited, it may not always be possible to optimize all specific computational steps in terms of memory operations in the manner described above. Therefore, an iterative optimization method can be used with an SMT solver, which allocates different storage regions for the output data blocks of the previous computational step for these specific computational steps.

[0041] Optimal optimization is achieved when, for all specific computation steps, the output data block of the previous computation step is written to the working memory, allowing the output data of the specific computation step's output data block to be obtained without performing a copy operation on the output data block of the previous computation step. For this, only corresponding address assignments (pointers) are needed to define the data of the output data block for a specific computation step.

[0042] Then, this memory plan is used for code generation.

[0043] Figure 3a and Figure 3bThis clarifies the location of the storage regions for the input and output data blocks EB1, EB2, AB1, and AB2 in the case of a convolutional layer followed by a concatenation layer that links the input data block EB1. Figure 3a As can be seen, the output data blocks of the convolutional layer supplement the other input data of the splicing layer. However, another copying operation is performed to provide the output data blocks of the splicing layer.

[0044] exist Figure 3b As shown, the output data block AB1 of the convolutional layer is entirely located within the address range of the storage area of ​​the input data block EB2 of the splicing layer; and no copying operation is required to provide the output data block AB2, because the input data block EB2 is a part of the output data block AB2.

Claims

1. A computer-implemented method for performing memory planning for code generation, the code generation being used to determine code for neural network computation, the method comprising the following steps: - Provides (S1) the sequential computation steps of the neural network, wherein, For each computation step, determine the size of the input data block (EB1, EB2) and the output data block (AB1, AB2); - Determine the conditions of the memory planning for each specific computation step (S2), wherein the input data block (EB1) in the storage area is at least partially contained in the output data block (AB2), wherein the conditions indicate that the storage area of ​​the output data block (AB1) of the previous computation step assigned to the specific computation step is contained in the storage area of ​​the output data block (Ab2) of the specific computation step. - Taking into account the determined conditions, perform (S3) memory planning, in which the storage areas in the working memory for the corresponding input data blocks (EB1, EB2) and output data blocks (AB1, AB2) are specified for each computation step.

2. The method according to claim 1, wherein, Performing the memory planning includes applying an optimization method, wherein the objective function considers minimizing the number of memory operations.

3. The method according to claim 1 or 2, wherein, The input data blocks (EB1, EB2) and the output data blocks (AB1, AB2) respectively indicate or are allocated to the storage regions, which have continuously increasing addresses.

4. The method according to any one of claims 1 to 3, wherein, If the specific calculation step is a splicing step, then the condition indicates that the preceding calculation step of the splicing step provides output data blocks (AB1, AB2) in the following storage areas, the storage areas being adjacent to the storage areas where the output data of the preceding calculation step should be connected in the splicing step.

5. The method according to any one of claims 1 to 4, wherein, If the specific computation step is a filling step, then the condition indicates that the preceding computation step of the filling step provides an output data block in a storage area adjacent to a storage area occupied by the filling mode or subsequently written with the filling mode.

6. The method according to any one of claims 1 to 5, wherein, If the specific computation step is a slicing step, then the condition indicates that the preceding computation step of the slicing step provides an output data block in the storage area corresponding to the output data block portion of the specific computation step.

7. The method according to any one of claims 1 to 6, wherein, Based on the results of the memory planning, code generation is performed for the hardware environment (2) and implemented in the hardware environment.

8. An apparatus for performing the method according to any one of claims 1 to 7.

9. A computer program product comprising instructions that, when executed by at least one data processing device, cause the data processing device to perform the steps of the method according to any one of claims 1 to 7.

10. A machine-readable storage medium comprising instructions that, when executed by at least one data processing device, cause the data processing device to perform the steps of the method according to any one of claims 1 to 7.

Images