Method for memory allocation during neural network execution

Abstract

This disclosure relates to a method for memory allocation during the execution of a neural network. According to one aspect, a method is proposed for defining the placement of temporary buffers used during the execution of an artificial neural network in volatile memory, the method comprising: determining the execution order of layers of the neural network; defining, according to the execution order of the layers, the placement of intermediate result buffers generated by each layer in a heap memory region of the volatile memory; determining at least one free region of the heap memory region during the execution of the layers, and defining the placement of the temporary buffers in at least one free region of the heap memory region according to the execution order of the layers.

Description

[0001] Cross-reference to related applications

[0002] This application claims priority to French Application No. 2013373, filed on December 16, 2020, which is incorporated herein by reference in its entirety. Technical Field

[0003] This disclosure generally relates to artificial neural networks, and in specific embodiments, to memory allocation during neural network mapping. Background Technology

[0004] Artificial neural networks can be implemented using integrated circuits such as microcontrollers. An artificial neural network typically consists of a series of layers of neurons. The artificial neural network takes a dataset as input to the first layer and passes the final result as the output of the last layer. Each layer, arranged between the first and last layers, generates blocks of intermediate result data, also known as tensor data, at its output. Other layers take this intermediate result data as input and thereby generate their corresponding intermediate result data.

[0005] During the execution of a neural network, each intermediate result is temporarily stored in a memory region of the integrated circuit (called the global memory region). However, integrated circuits have a finite amount of memory. Limiting the size required to store the intermediate results generated by the neural network layers would be advantageous.

[0006] The method is used to define the placement of intermediate result buffers in memory during the execution of a neural network, thereby limiting the size of the memory used to store intermediate results.

[0007] For example, such a method is disclosed in European patent application No. EP3663987 or French patent application No. 2004337.

[0008] In addition, the execution of a layer may require temporary storage of scratch data in memory. Scratch data is temporary intra-layer data used only by a given layer and needs to be allocated only during the execution of that given layer.

[0009] For example, temporary data can be weights, temporary intermediate results related to a layer, or copies of lookup tables. Temporary data can also be in-layer data for computationally intensive tasks such as loop unrolling. However, such temporary storage of temporary data is often not optimized. To minimize the inference time of a neural network, temporary storage of temporary data often results in the loss of RAM or flash memory.

[0010] It would be advantageous to provide a method for efficiently allocating and placing temporary data. Summary of the Invention

[0011] According to one aspect, a method is proposed for defining the placement of temporary buffers used during the execution of an artificial neural network in volatile memory. According to one embodiment, the method includes: determining the execution order of layers of the neural network; and defining, based on the execution order of the layers, the placement of intermediate result buffers generated by each layer in a heap memory region of the volatile memory. The method further includes: determining at least one free region of the heap memory region during the execution of the layers; and defining, based on the execution order of the layers, the placement of the temporary buffers in at least one free region of the heap memory region.

[0012] In one embodiment, the placement of intermediate result buffers determines the limitations of the heap memory area required to allocate these intermediate result buffers, based on the execution order of the neural network layers.

[0013] In this embodiment, intermediate result buffers are allocated in the heap memory area whenever the execution of a layer requires them. When the intermediate result buffers used for the execution of a layer have a cumulative size smaller than the size of the heap memory area, free areas appear in the heap memory area. This approach allows the available free areas in the heap memory area used to allocate intermediate result buffers to also be used to allocate temporary buffers.

[0014] Scratch buffers can be used for intra-layer optimization. For example, scratch buffers can be used to move layer weights from flash memory to random access memory (RAM). Scratch buffers can also be used for loop unrolling for computationally intensive tasks. Allocating scratch buffers using free areas of the heap memory region allows for optimization of inference time without increasing the size of RAM or flash memory. This method also allows for reduced power consumption for executing neural networks.

[0015] The order in which layers of a neural network are executed can be determined using a depth-first search algorithm (e.g., by using directed acyclic graphs).

[0016] The methods disclosed in European patent application No. EP3663987 or French patent application No. n20.04337 can be used to define the placement of intermediate result buffers in the heap memory area.

[0017] In one embodiment, the method includes, for each layer, sorting the temporary buffers associated with that layer from the largest buffer to the smallest buffer, arranging variable-size buffers after the smallest buffer, and defining the placement of the temporary buffers according to their order.

[0018] In one embodiment, the defined placement of the temporary buffer is the placement of the temporary buffer on top of at least one free region.

[0019] In one embodiment, the defined placement of the temporary buffer is the placement of the temporary buffer at the bottom of at least one free region.

[0020] In one embodiment, the placement of the defined temporary buffer is the placement of the temporary buffer on top of another temporary buffer.

[0021] In one embodiment, the method includes, for the placement of a temporary buffer: modifying the top of the heap memory region to obtain a free area with the size of the temporary buffer at the top of the heap memory region, and defining the placement of the temporary buffer in the free area at the top of the heap memory region.

[0022] In one embodiment, the method includes, for placement of a temporary buffer: modifying the bottom of the heap memory region to obtain a free area at the bottom of the heap memory region having the size of the temporary buffer, so as to place the temporary buffer in the free area at the bottom of the heap memory region.

[0023] In one embodiment, after defining the placement of a scratch buffer with a fixed size, a placement for a scratch buffer with a variable size is defined in at least one remaining free region of the heap memory area. Advantageously, when several remaining free regions are available for the placement of a scratch buffer with a variable size, the largest free region among these remaining free regions is selected for placing the scratch buffer.

[0024] Preferably, the method includes: storing the free regions for the layer in a list, and updating the list after each placement definition of the temporary buffer.

[0025] In one embodiment, a computer program product is provided that includes instructions that, when executed by a computer, cause the computer to perform the methods described above.

[0026] In one embodiment, an integrated circuit is provided, comprising: a non-volatile memory configured to store a neural network, a volatile memory, and a processor. The processor is configured to: determine the execution order of layers of the neural network; define, according to the execution order of the layers, the placement of intermediate result buffers generated by each layer in a heap memory region of the volatile memory; determine at least one free region of the heap memory region during the execution of the layers; and define, according to the execution order of the layers, the placement of temporary buffers in at least one free region of the heap memory region.

[0027] In one embodiment, the processor is also configured to, for each layer: sort the temporary buffers associated with that layer from the largest buffer to the smallest buffer; arrange variable-size buffers after the smallest buffer; and define the placement of the temporary buffers in that order.

[0028] In one embodiment, the processor is configured to define the placement of a temporary buffer on top of at least one free region.

[0029] In one embodiment, the processor is configured to define the placement of a temporary buffer at the bottom of at least one free region.

[0030] In one embodiment, the processor is configured to define placement on another temporary buffer.

[0031] Advantageously, the processor is configured to, for the placement of the scratch buffer, modify the top of the heap memory region to obtain a free area with the size of the scratch buffer at the top of the heap memory region; and define the placement of the scratch buffer in the free area at the top of the heap memory region.

[0032] In one embodiment, the processor is configured to, for the placement of the temporary buffer, modify the bottom of the heap memory region to obtain a free area at the bottom of the heap memory region with the size of the temporary buffer; and place the temporary buffer in the free area at the bottom of the heap memory region.

[0033] Preferably, the processor is configured to: after defining the placement of a temporary buffer with a fixed size, define the placement of a temporary buffer with a variable size in at least one remaining free region of the heap memory area. Advantageously, the processor is configured to: when there are several remaining free regions available for the placement of a temporary buffer with a variable size, select the largest free region among these remaining free regions for placing the temporary buffer.

[0034] In one embodiment, the processor is configured to store free regions for a given layer in a list and update the list after each placement definition of the staging buffer. Attached Figure Description

[0035] Other advantages and features of the invention will appear in the detailed description of the embodiments and implementations, but are not limiting, and are also shown in the accompanying drawings:

[0036] Figure 1 This is a block diagram of the integrated circuit in the embodiment;

[0037] Figure 2 This is a flowchart of the method in the embodiment;

[0038] Figure 3 This is a schematic diagram showing the buffering of intermediate results in the embodiments over time;

[0039] Figure 4 This is a flowchart of the method in the embodiment; and

[0040] Figure 5This is a diagram showing the placement of an example of intermediate result buffering and temporary buffering. Detailed Implementation

[0041] Figure 1 An integrated circuit (CI) is shown. The integrated circuit (CI) can be, for example, a microcontroller, a code generator, or any other object that can contain on-board or hardware-software architecture.

[0042] Integrated circuits (CIs) are configured to implement neural networks. Neural networks consist of a series of layers.

[0043] Integrated circuits (CIs) include non-volatile memory (MNVs) configured to store various parameters that define the architecture of a neural network, such as the configuration of its layers and their weights and parameters.

[0044] Integrated circuits (CIs) also include volatile memory (MV), such as RAM (Random Access Memory). The volatile memory (MV) is configured to store the executable code of the neural network and the program used to execute the neural network. The volatile memory (MV) also includes zones called heap memory zones, which can be dynamically allocated for storing data used during the execution of the neural network. As will be described in detail below, this data can be intermediate results generated by layers or temporary buffer data used for the execution of a given layer.

[0045] The heap memory region can be divided into faster memory sections and slower memory sections. Access to the faster memory section is performed faster than access to the slower memory section.

[0046] Integrated circuits (CI) also include processors (UTs). Processors (UTs) include, for example, microprocessors. The processor (UT) is coupled to volatile memory (MV). The processor is configured to allocate heap memory regions of volatile memory (MV) to store data during the execution of the neural network.

[0047] To optimize memory allocation during the execution of these neural networks, a method for defining the placement of buffers in volatile memory is implemented before the network is executed.

[0048] The processor (UT) is configured to implement such a method. Specifically, the processor (UT) can execute a computer program that includes instructions that, when executed by the processor, cause the processor to perform the method.

[0049] This method is as follows Figure 2 As shown. The method includes step 20, in which the processor (UT) determines the execution order of the layers of the neural network. This allows for obtaining the execution schedule of the neural network layers.

[0050] More specifically, the order of execution of layers in a neural network can be determined by a depth-first search algorithm (e.g., by using a directed acyclic graph known to those skilled in the art).

[0051] Then, the method includes step 21, in which the processor (UT) defines the placement of intermediate result buffers (also known as tensor buffers) in memory according to the execution order of the layers.

[0052] Specifically, the intermediate result buffer is used to store intermediate results (also known as tensor data). Intermediate results are data computed by each layer except the last, based on the data received at the layer's input. The intermediate results computed by one layer are passed to that layer's output and used by subsequent layers. Intermediate results need to be stored in memory until the layer that uses them is executed. The definition of the intermediate result buffer's placement allows for optimization of the size of the heap memory area required to store the intermediate results.

[0053] Specifically, the methods disclosed in European patent application No. EP3663987 or French patent application No. 2004337 can be used to define the placement of intermediate result buffers in the heap memory area.

[0054] Figure 3 The intermediate result buffer placement over time for each layer is shown. Figure 3 The placement of intermediate result buffers in the heap memory area (HMZ) is shown on the y-axis (vertical axis), and the execution time as the result of processing from layer 1 to layer N is shown on the x-axis (horizontal axis).

[0055] For example, during the execution time of some layers, intermediate result buffers TB1, TB2, and TB3 are placed in the heap memory area.

[0056] The size of the heap memory area (HMZ) is defined by the placement chosen for the intermediate result buffer. For example, in Figure 3 In this context, the size of the heap memory region, sz, is equal to sz = topz - botz, where topz is the top address of the heap memory region and botz is the bottom address. As shown in the figure, during the execution of most layers, the actual size of the heap memory region is not fully allocated. During the execution of most layers, some areas of the heap memory region are free. For example, in... Figure 3 In this context, regions FA1, FA2, and FA3 are free.

[0057] This method utilizes free areas in the heap memory region to allocate temporary buffers. Specifically, this method allows for the definition of temporary buffer placement within the heap memory region.

[0058] A temporary buffer can be used to store temporary data for layer optimization. In one embodiment, the temporary buffer is used to move layer weights from flash memory to RAM. In some embodiments, the temporary buffer can also be used for loop unrolling for computationally intensive tasks.

[0059] Temporary data is defined for the execution of a given layer of a neural network. Therefore, each temporary buffer is associated with a given layer of the neural network.

[0060] Scratch buffers can have a fixed size or a variable size. A variable-size scratch buffer is a scratch buffer with an unknown size that precedes the execution of the neural network. A scratch buffer with a variable size may or may not need a minimum size.

[0061] Specifically, the method includes step 22, in which the processor (UT) determines a free region of the heap memory area during the execution of a layer. More specifically, the free region of the heap memory area is determined for the execution time of each layer.

[0062] For example, in Figure 3 In order to continue execution of level 10, the processor (UT) requires the data stored in intermediate result buffers TB2 and TB3. These intermediate result buffers occupy only a given area in the heap memory region. Therefore, at the execution time of level 10, regions FA1 and FA2 are free. In this embodiment, the determined free regions are stored in a list.

[0063] The method then includes a process for defining the placement of temporary staging buffers in the free region of the heap memory area according to the execution order of the layers. The placement of temporary staging buffers is performed sequentially according to the execution schedule of the layers.

[0064] Specifically, in step 23, temporary buffers to be placed are stored in a queue according to the execution schedule. In this embodiment, temporary buffers associated with the same layer are stored in the queue from largest to smallest, with buffers of variable size arranged after the smallest buffer. As described below, after the placement of a buffer is defined, it is removed from the queue.

[0065] The process for defining the placement of temporary buffers includes step 24, where the processor (UT) checks if the queue is empty. If the queue is not empty, it means that at least one temporary buffer still needs to be placed. In this case, the next step is step 25.

[0066] If the queue is empty, it means there is no temporary buffer to place. In this case, the method ends at step 28.

[0067] When the queue is not empty, in step 25, the processor (UT) restores the first temporary buffer in the queue to put it into the heap memory area.

[0068] Specifically, in step 26, the processor (UT) checks the free regions of the heap memory area available at the execution time of the layer associated with the temporary buffer restored in step 25. This check is performed based on a list of free regions available at the execution time of the layer.

[0069] Then, in step 27, the processor (UT) defines the placement of the temporary buffer based on the constraints of the temporary buffer and the allowable movement.

[0070] Figure 4 The procedure for defining the placement of temporary buffers in the heap memory area is shown.

[0071] More specifically, in step 40, the processor (UT) checks whether latency optimization is needed for the scratch buffer. If latency optimization is needed, the processor (UT) in step 41 selects the faster memory portion of the heap memory region as the portion of the heap memory region where the scratch buffer will be placed. Next, the processor (UT) extracts the free regions of the faster memory portion from the list of free regions based on the execution time of the layer associated with the scratch buffer.

[0072] If latency optimization is not required, the processor (UT) selects the slower memory portion of the heap memory region in step 42 as the portion of the heap memory region where the scratch buffer will be placed. Next, the processor (UT) extracts the free regions of the slower memory portion from the list of free regions based on the execution time of the layer associated with the scratch buffer.

[0073] Then, in step 43, the processor (UT) evaluates whether it is possible to place the temporary buffer in the free area extracted from the list.

[0074] Specifically, the processor (UT) compares the size of the temporary buffer with the size of each free region extracted from the list. When the size of an extracted free region is greater than or equal to the size of the temporary buffer, that extracted free region is adapted to receive the temporary buffer.

[0075] If at least one free region is adapted to receive the temporary buffer, the processor (UT) proceeds directly to step 44, where the processor (UT) defines the placement of the temporary buffer.

[0076] If there is no free area adapted to receive the temporary buffer, then in step 44, the processor (UT) increases the size of the portion of the temporary buffer in the heap memory area that must be placed therein.

[0077] In one embodiment, the processor (UT) adds a top or bottom portion of the heap memory region to obtain a free area at the top or bottom portion of the heap memory region that is adapted to receive a temporary buffer.

[0078] In this embodiment, when there are no free regions at the top and bottom of the global memory size, the processor (UT) modifies the top and bottom addresses of the heap memory area to obtain a free region equal to the size of the temporary buffer.

[0079] However, when free regions exist at the top and bottom of the heap memory area respectively, the processor (UT) modifies the top and bottom addresses respectively to increase the size of the free region by the difference between the size of the temporary buffer and the initial size of the free region.

[0080] Then, the processor (UT) proceeds to step 45. In step 45, the processor (UT) defines the placement of the temporary buffer. In an embodiment, if the temporary buffer has a fixed size, the processor (UT) selects the smallest free region extracted from the list to place the temporary buffer, the size of which is adapted to receive the temporary buffer. The processor (UT) can then place the temporary buffer of a fixed size according to different possible placements.

[0081] A first placement of a fixed-size staging buffer includes placing the staging buffer on top of another staging buffer for layer execution, in order to minimize the final heap size and when a particular layer fulfills a request. In an embodiment, the staging buffer is placed on top of an already placed staging buffer that is larger than or equal in size to the staging buffer to be placed. The staging buffer can also be placed on top of an already placed staging buffer when there is a free region adjacent to an already placed staging buffer that is smaller than the staging buffer to be placed, and the size of this free region is greater than or equal to the difference between the staging buffer to be placed and the already placed staging buffer. In an embodiment, this free region is a free region adjacent to the bottom of the already placed staging buffer.

[0082] A second placement for a fixed-size temporary buffer includes placing the temporary buffer on top of the minimum free area whose size is adapted to receive the temporary buffer.

[0083] A third placement for a fixed-size temporary buffer includes placing the temporary buffer at the bottom of the smallest free area that is adapted to receive the temporary buffer.

[0084] In an embodiment, when all possible placements of a fixed-size temporary buffer associated with the same layer are defined, the processor defines the placement of a variable-size temporary buffer associated with the same layer.

[0085] To accommodate a temporary buffer of variable size, the processor (UT) selects the largest remaining set of free regions from the list to place the temporary buffer. In this embodiment, the remaining free regions are ordered in descending order of size.

[0086] If the number of variable-sized temporary buffers to be placed is less than or equal to the number of remaining free regions, then each temporary buffer can be placed within a given remaining free region ordered in descending order of size. In this case, the defined size of the temporary buffer is equal to the size of the remaining free region in which the temporary buffer is placed.

[0087] However, if the number of variable-size temporary buffers to be placed is greater than the number of remaining free regions, the processor calculates an average of the number of temporary buffers to be placed in the remaining free regions. Therefore, variable-size temporary buffers can be placed in the remaining free regions along with at least one other variable-size temporary buffer. In an embodiment, the defined size of the temporary buffer to be placed is equal to the size of the remaining free region in which the temporary buffer is placed divided by the number of temporary buffers to be placed in that remaining free region.

[0088] Furthermore, in cases where a variable-size temporary buffer requires a minimum size larger than the size of each remaining free region, in step 44, the top or bottom of the heap memory region can be increased to obtain a free region at the top or bottom of the heap memory region with a size equal to the minimum size required for the temporary buffer. The temporary buffer is then placed in this free region.

[0089] After the temporary buffer is placed, update the free region list and remove the temporary buffer from the queue.

[0090] Then, starting from step 24, this process is repeated to place the next temporary buffer in the queue until the queue is empty. When the queue is empty, there are no more temporary buffers to place. The method ends at step 28.

[0091] However, the process used to define the placement of scratch buffers can be repeated to try different placements for each scratch buffer. This allows finding an optimal placement for the scratch buffer. Specifically, the placement can be optimized according to given criteria. For example, a criterion could be to avoid an increase in the heap memory area defined by the placement of intermediate result buffers. Another criterion could be to ensure that the placement scheme allows for a minimum heap memory area size. Alternatively, another criterion could be to ensure that the placement scheme allows for a minimum fragmented memory area. Yet another criterion could be to ensure that the placement scheme allows for maximizing the placement of scratch buffers in the faster memory portions of the heap memory area to accelerate the execution of neural networks.

[0092] Figure 5 This illustrates an example of the final scheme for placing intermediate result buffers and temporary buffers. The placement of the intermediate result buffer is related to... Figure 3 The same applies to buffers TB1, TB2, and TB3. Therefore, the placement of intermediate result buffers defines the heap memory area between addresses botz and topz.

[0093] In this embodiment, some temporary buffers are placed at the top or bottom of some free regions. For example, temporary buffer SB1 is placed at the top of the free region, and temporary buffer SB2 is placed at the bottom of the free region.

[0094] In this embodiment, some other temporary buffers are placed on top of the already placed temporary buffers. For example, temporary buffer SB3 is placed on top of the temporary buffer that is already placed on top of the free area. Another temporary buffer SB4 is placed on top of temporary buffer SB3.

[0095] In an embodiment, the top or bottom of the heap memory area is increased to accommodate temporary buffers when no free area is available.

[0096] For example, to place the temporary buffer SB5, the address of the top of the heap memory area is incremented to address i_topz to obtain a free area of ​​size SB5 at the top of the heap memory area. Then, the temporary buffer SB5 is placed in this free area.

[0097] Furthermore, to accommodate the temporary buffer SB6, the address at the bottom of the heap memory region is modified to address f_botz, thus obtaining a free area at the bottom of the heap memory region equal to the size of the temporary buffer SB6. The temporary buffer SB6 is then placed within this free area.

[0098] To accommodate the scratch buffer SB7, the top of the heap memory region is increased to address f_topz because the size of the free area at the top of the heap memory region defined before the placement of scratch buffer SB6 is smaller than that of scratch buffer SB7. In this embodiment, the top of the heap memory region is increased by the difference between the size of scratch buffer SB7 and the size of the free area at the top of the heap memory region defined before the placement of scratch buffer SB7.

[0099] Finally, the final heap memory area FHMZ is defined between the addresses f_topz and f_botz.

[0100] After defining the placement of intermediate result buffers and temporary buffers, the processor (UT) can execute the neural network. When the processor (UT) executes the neural network, memory is allocated according to the defined placement of intermediate result buffers and temporary buffers.

[0101] The above method allows the use of available free space in the heap memory area that was used to allocate intermediate result buffers in order to allocate temporary buffers instead.

[0102] By using free areas in the heap memory region to allocate temporary buffers, inference time can be optimized with minimal impact on the size of RAM or flash memory.

[0103] This method also allows for a reduction in the power consumption of neural network execution.

[0104] Although the description has been detailed, it should be understood that various changes, substitutions, and alterations can be made without departing from the spirit and scope of this disclosure as defined by the appended claims. In different drawings, the same elements are indicated by the same reference numerals. Furthermore, the scope of this disclosure is not intended to be limited to the specific embodiments described herein, as it will be readily understood from this disclosure by those skilled in the art that existing or future processes, machines, manufactures, material compositions, components, methods, or steps can perform substantially the same functions or achieve substantially the same results as the corresponding embodiments described herein. Accordingly, the appended claims are intended to include such processes, machines, manufactures, material compositions, components, methods, or steps within their scope.

[0105] Accordingly, the specification and drawings are simply regarded as a description of this disclosure as defined by the appended claims, and are intended to cover any and all modifications, variations, combinations or equivalents falling within the scope of this disclosure.

Claims

1. A method comprising: Determine the execution order of layers in an artificial neural network; In the heap memory area of ​​volatile memory, the placement of intermediate result buffers generated by each layer is defined according to the execution order of the layers; For each layer of execution, determine the free region of the heap memory area; as well as In the free region of the heap memory area, the placement of temporary buffers is defined based on the execution order of the layers. The temporary buffer is configured to store temporary data, which is temporary intra-layer data used only by a given layer and needs to be allocated only during the execution of that given layer.

2. The method according to claim 1, further comprising: The temporary buffers associated with the corresponding layers are sorted from the largest buffer to the smallest buffer; as well as Before defining the placement of the temporary buffer, arrange the variable-size temporary buffer after the minimum buffer.

3. The method of claim 1, wherein defining the placement includes: Define the placement of the temporary buffer at the top of the free area of ​​the heap memory region.

4. The method of claim 1, wherein defining the placement includes: Define the placement of the temporary buffer at the bottom of the free area of ​​the heap memory region.

5. The method of claim 1, wherein defining the placement includes: Define the temporary buffer as replacing the placement of another temporary buffer.

6. The method according to claim 1, further comprising: Regarding the placement of the temporary buffer, the top of the heap memory area is modified to obtain a free area with the size of the temporary buffer at the top of the heap memory area; as well as Regarding the placement of the temporary buffer, the placement of the temporary buffer in the free area at the top of the heap memory area is defined.

7. The method according to claim 1, further comprising: Regarding the placement of the temporary buffer, the bottom of the heap memory area is modified to obtain a free area at the bottom of the heap memory area with the size of the temporary buffer. as well as Regarding the placement of the temporary buffer, the placement of the temporary buffer in the free area at the bottom of the heap memory area is defined.

8. The method of claim 1, wherein defining the placement comprises: After defining the placement of the temporary buffer with a fixed size, the placement of the temporary buffer with a variable size in the remaining free area of ​​the heap memory region is defined.

9. The method according to claim 1, further comprising: Store the free region of the heap memory area in a list; as well as The list is updated after each placement definition of the temporary buffer.

10. A non-transitory computer-readable medium storing computer instructions, which, when executed by one or more processors, cause the processors to perform: Determine the execution order of layers in an artificial neural network; In the heap memory area of ​​volatile memory, the placement of intermediate result buffers generated by each layer is defined according to the execution order of the layers; For each layer of execution, determine the free region of the heap memory area; as well as In the free region of the heap memory area, the placement of temporary buffers is defined based on the execution order of the layers. The temporary buffer is configured to store temporary data, which is temporary intra-layer data used only by a given layer and needs to be allocated only during the execution of that given layer.

11. The non-transitory computer-readable medium storing computer instructions according to claim 10, wherein when the computer instructions are executed by one or more processors, the processors cause the processors to perform: The temporary buffers associated with the corresponding layers are sorted from the largest buffer to the smallest buffer; and Before defining the placement of the temporary buffer, arrange the variable-size temporary buffer after the minimum buffer.

12. The non-transitory computer-readable medium storing computer instructions according to claim 10, wherein defining the placement includes: Define the placement of the temporary buffer at the top of the free area of ​​the heap memory region.

13. The non-transitory computer-readable medium storing computer instructions according to claim 10, wherein the placement is defined as comprising: Define the placement of the temporary buffer at the bottom of the free area of ​​the heap memory region.

14. The non-transitory computer-readable medium for storing computer instructions according to claim 10, wherein defining the placement includes: Define the temporary buffer as replacing the placement of another temporary buffer.

15. The non-transitory computer-readable medium storing computer instructions according to claim 10, wherein when the computer instructions are executed by one or more processors, the processors cause the processors to perform: Regarding the placement of the temporary buffer, the top of the heap memory region is modified to obtain a free area with the size of the temporary buffer at the top of the heap memory region; and Regarding the placement of the temporary buffer, the placement of the temporary buffer in the free area at the top of the heap memory area is defined.

16. An integrated circuit, comprising: Non-volatile memory storage devices, including artificial neural network instructions; Volatile memory storage devices; as well as A processor, coupled to the non-volatile memory storage device and the volatile memory storage device, wherein the processor is configured to execute the artificial neural network instructions to: Determine the execution order of layers in an artificial neural network. In the heap memory area of ​​the volatile memory storage device, the placement of intermediate result buffers generated by each layer is defined according to the execution order of the layers. For each level of execution, determine the free region of the heap memory area, and In the free area of ​​the heap memory region, the placement of temporary buffers is defined according to the execution order of the layers. The temporary buffer is configured to store temporary data, which is temporary intra-layer data used only by a given layer and needs to be allocated only during the execution of that given layer.

17. The integrated circuit of claim 16, wherein the processor is configured to execute the artificial neural network instructions to: The temporary buffers associated with the corresponding layers are sorted from the largest buffer to the smallest buffer; and Before defining the placement of the temporary buffer, arrange the variable-size temporary buffer after the minimum buffer.

18. The integrated circuit of claim 16, wherein defining the placement includes: Define the placement of the temporary buffer at the top of the free area of ​​the heap memory region.

19. The integrated circuit of claim 16, wherein defining the placement includes: Define the placement of the temporary buffer at the bottom of the free area of ​​the heap memory region.

20. The integrated circuit of claim 16, wherein defining the placement includes: Define the temporary buffer as replacing the placement of another temporary buffer.

Images