Device and method for distributing intermediate data from an artificial neural network

Abstract

The present disclosure provides an apparatus and method for allocating intermediate data from an artificial neural network. According to one aspect, a method for determining the placement of a data block generated by a neural network in a memory area for memory allocation includes: developing an initial sequence of placements of blocks, selecting each placement from a plurality of possible placements, the initial sequence being defined as a candidate sequence; developing at least one modified sequence of placements according to replacement of a given placement of the initial sequence through stored unselected placements; and if the planned size of the memory area obtained by the modified sequence is less than the size of the memory area of ​​the candidate sequence, the modified sequence becomes a candidate sequence; once each modified sequence is developed, the placement for the allocated block is a placement of the placement sequence defined as the candidate sequence.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of French Application No. 2004337, filed on April 30, 2020, which is incorporated herein by reference. Technical Field

[0003] Embodiments and implementations relate to artificial neural network apparatus and methods, and more particularly, to the allocation of memory during execution of these neural networks. Background Art

[0004] Artificial neural networks typically consist of layers of neurons.

[0005] The artificial neural network takes the dataset as input in the first layer and outputs the final result as output from the last layer.

[0006] Each layer of the neural network until the last layer allows the generation of intermediate data blocks at the output of the layer, representing the intermediate results. These intermediate data blocks are used as inputs to other layers, and other layers generate their own intermediate data blocks based on these intermediate data blocks. Summary of the invention

[0007] Artificial neural networks can be implemented by integrated circuits such as microcontrollers.

[0008] During the execution of the neural network, intermediate data is temporarily stored in a memory area of ​​the integrated circuit memory (called the global memory area). However, some integrated circuits have a memory of limited size.

[0009] Therefore, it is important to limit the size required for storage of intermediate data generated by the layers of a neural network.

[0010] In order to limit the size required for storage of intermediate data, methods should be provided that allow temporary storage of intermediate data blocks according to an optimized placement in the global memory area.

[0011] Known allocation solutions do not always allow to obtain an optimal allocation of memory. In particular, these allocation solutions do not always allow to obtain an optimized global memory area size.

[0012] For example, one known solution consists in placing intermediate data blocks only in free areas of the global memory area.

[0013] Such a solution may lead to excessive memory fragmentation, resulting in wasted memory if the free area of ​​memory becomes too small during the execution of the neural network.

[0014] Another known solution consists of an overlay allocation, wherein, for each layer, an intermediate data block generated by this layer may overwrite at least a part of an intermediate data block used as input for this layer in a global memory area.

[0015] The disadvantage of this solution is that it only works for sequential neural networks, which are neural networks where the intermediate data blocks generated by each layer are only used as input to the layer immediately following the last layer.

[0016] Therefore, this solution is not suitable for neural networks that use the output data of some layers as input to multiple layers. In fact, these output data cannot be overwritten after being used by subsequent layers when other layers need them.

[0017] Therefore, there is a need to propose a method that allows reducing the size of the memory required to store intermediate data blocks generated by the layers of a neural network.

[0018] According to one aspect, a method for determining the placement of data blocks (referred to as intermediate data blocks) generated by each layer of an artificial neural network in a global memory area for memory allocation is proposed, the method comprising: defining a plurality of possible placements of intermediate data blocks of each layer in the global memory area; developing an initial sequence of placements of intermediate data blocks of each layer from the first layer to the last layer of the neural network, selecting each placement of the initial sequence from a plurality of possible placements of the intermediate data blocks to be placed, the initial sequence allowing the definition of a planned size of the global memory area, the initial sequence being defined as a candidate sequence; storing unselected placements of the intermediate data blocks of each layer; developing at least one placement sequence modified compared to the initial sequence based on a given placement of the initial sequence, the development of the modified sequence comprising modifying the given placement by the stored unselected placement; and if the planned size of the global memory area obtained by the modified sequence is smaller than the planned size of the global memory area of ​​the candidate sequence, the modified sequence becomes a candidate sequence; the placement of the intermediate data blocks for memory allocation is a placement sequence referred to as a selection sequence, and once each modified sequence is developed, it is defined as a candidate sequence.

[0019] Some possible placements of intermediate data blocks may result in an increase in the size of the global memory area.

[0020] This approach allows trying multiple placement sequences of the intermediate data blocks in order to determine which placement sequence allows obtaining a planned size of the global memory region, eg the narrowest possible size.

[0021] The number of sequences tested (including the initial sequence) can be greater than or equal to 2.

[0022] This approach thus allows, for example, finding an optimal placement for each intermediate data block in order to minimize the size of the global memory area.

[0023] By minimizing the size of the global memory area, this method allows the use of integrated circuits with a small memory size. Thus, this method allows, for example, the use of inexpensive integrated circuits.

[0024] The size of the global memory region is dynamically defined according to the placement selected during the development of the placement sequence. Specifically, when the selected placement of the intermediate data block requires an increase in the size of the global memory region, the global memory region is increased.

[0025] Preferably, all possible modification sequences are exploited, that is to say all possible placement sequences are tried, in order to obtain a placement sequence which allows a maximum optimization of the size of the global memory area.

[0026] However, the number of modification sequences to be developed may also be limited in order to speed up the execution of this determination method by the processing unit.The placement sequence selected for the memory allocation may not be optimal, but will still be optimal among the developed placement sequences.

[0027] In an advantageous embodiment, a priority order is assigned to each possible placement of each intermediate data block, and the selection of the placement of each intermediate data block for development of the initial sequence is performed according to the priority order of the placement.

[0028] The specification of an order of priority allows reducing the number of placement sequences to be developed in order to obtain a placement sequence that allows minimizing the planned size of the global memory area.

[0029] The specification of a priority order allows reducing the execution time of the methods.

[0030] Preferably, the placements not selected for the initial sequence are stored from bottom to top in the stack according to the priority order of each intermediate data block.

[0031] The stack may have a limited size. A small stack size allows to speed up the execution of the method.

[0032] Advantageously, each modification sequence is developed so that each modification sequence retains the same placement as the initial sequence until a placement associated with the same intermediate data block as the placement placed at the top of the stack. The placement of the initial sequence is replaced by the placement placed at the top of the stack and then removed from the stack. The placement of the intermediate data blocks of the subsequent layers of the modification sequence is selected from the possible placements for these intermediate data blocks. The unselected placements are stored in the stack.

[0033] In an advantageous embodiment, a modification sequence is developed until the stack is empty.

[0034] Advantageously, each placement stored in the stack is associated with a size of the global memory region that would be obtained by the placement if the global memory region were selected from among the various possible placements of the intermediate data block.

[0035] Preferably, the method comprises, before each development modification sequence, comparing the size of the global memory area associated with each placement of the stack with a planned size of the global memory area obtained for the candidate sequence, and then removing from the stack placements associated with a size of the global memory area that is larger than the planned size.

[0036] This removal allows avoiding the construction of modified sequences that cannot be selected as candidate sequences. Thus, this removal allows speeding up the determination method and freeing up space in the stack if the stack has a limited size.

[0037] Preferably, the method comprises, before each development modification sequence, organizing the placement of stacks associated with the same intermediate data block according to the size of the global memory area associated therewith and according to the execution order of these placements.

[0038] This organization allows for faster acquisition of the optimal placement sequence.

[0039] Advantageously, an area of ​​the global memory area selected for the placement of an intermediate data block is considered to be released as soon as the placement of data blocks of all layers that are input to the intermediate data block placed in this area is selected.

[0040] In an advantageous embodiment, the defined placement of the intermediate data block comprises placing the intermediate data block in a free area of ​​the global memory area.

[0041] This placement allows avoiding increasing the size of the global memory area.

[0042] In an advantageous embodiment, the defined placement of an intermediate data block generated by a layer comprises superimposing the intermediate data block on an intermediate data block that is input to the layer.

[0043] An intermediate data block generated by a layer is superimposed onto an intermediate data block that is used as input to the same layer, so that at least part of the intermediate data block is overwritten as input to the layer. Thus, this placement is used when the data of the data block used as input is not used again during the execution of the neural network. This placement can also avoid increasing the size of the global memory area.

[0044] In an advantageous embodiment, the defined placement of the intermediate data block comprises increasing the memory size of the global memory region to insert the intermediate data block. The size of the global memory region may be increased so as to allow the placement of the complete intermediate data block in a new portion of the global memory region resulting from the increase in the size of the global memory region. Alternatively, the global memory region may be increased so as to allow the placement of the data block by superimposing a portion of the intermediate data block onto intermediate data already stored in the global memory region, which are then overwritten, and storing the remainder of the intermediate data block in a new portion of the global memory region resulting from the increase in the size of the global memory region.

[0045] According to another aspect, a method for allocating a global memory size is proposed, wherein the global memory size is selected to be equal to a planned size obtained by a selection sequence defined by the above method for determining placement, and then memory is allocated according to the determined consecutive placement.

[0046] According to another aspect, there is provided an integrated circuit configured to receive a multi-layer artificial neural network and comprising:

[0047] - a memory configured to store intermediate blocks of data generated by each layer of the neural network in an area referred to as a global memory area; and

[0048] - A processing unit configured to implement the above-described method for determining the placement of intermediate data blocks in a global memory area and then to implement the above-described allocation method during the execution of the neural network. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Other advantages and features of the present invention will become apparent upon review of the detailed description of non-limiting embodiments and examples and the accompanying drawings, in which:

[0050] Figure 1 An integrated circuit is shown;

[0051] Figure 2 A flow chart for determining and allocating the size of a global memory region is shown;

[0052] Figure 3a-3c shows the placement of intermediate data blocks in the global memory area;

[0053] Figure 4a-4d shows the overlay placement of intermediate data blocks in the global memory area;

[0054] Figure 5a-5d shows the mandatory allocation placement of intermediate data blocks in the global memory area;

[0055] Figure 6The development and storage of an initial sequence consisting of three placements is shown;

[0056] Figure 7a showing a plurality of modified placement sequences developed from an initial placement sequence; and

[0057] Figure 7b Modifications to the stack are shown before the modification sequence is developed. DETAILED DESCRIPTION

[0058] Figure 1 An integrated circuit CI is shown. For example, the integrated circuit CI may be a microcontroller, a code generator or any other object that may contain on-board or hardware-software architecture.

[0059] The integrated circuit CI is configured to implement a neural network. The neural network comprises a series of layers. Each layer except the last layer is configured to generate an intermediate data block at the output of the layer from a data block received at the input of the layer. The input block received at the input may be an initial data block received at the input of the first layer, or an intermediate data block generated by an upstream layer in the series of layers.

[0060] Specifically, the data of each intermediate data block represents the intermediate result of the neural network.

[0061] Each layer of the neural network is assigned a depth level. Specifically, the first layer of a series of layers is considered the shallowest layer of the neural network, while the last layer is considered the deepest layer of the network.

[0062] The neural network may or may not be a sequential neural network. Specifically, the neural network may include some layers that generate output data that are used as inputs to multiple layers with different depth levels in the neural network.

[0063] The integrated circuit CI comprises a volatile memory MV, for example a RAM (“Random Access Memory”) memory.

[0064] The volatile memory MV is configured to be able to record the execution code of the neural network and the program allowing the execution of the neural network.

[0065] The volatile memory MV also includes a so-called global memory area ( Figure 1 An area (not shown) may be allocated for storing data as input and data generated at the output of each layer of the neural network.

[0066] The integrated circuit CI also includes a non-volatile MNV memory configured to record various parameters defining the architecture of the neural network, i.e. the configuration of the layers and weights of the neural network.

[0067] The integrated circuit CI also comprises a processing unit UT configured to determine the size of a global memory area to be allocated in order to be able to store all intermediate data blocks in the volatile memory MV during the execution of the neural network. To this end, the processing unit UT is coupled to the volatile memory MV and is configured to determine the size of the intermediate data blocks associated with the layers of the neural network.

[0068] For example, the processing unit UT comprises a microprocessor.

[0069] When executing a neural network, the layers are executed in an order defined according to the architecture of the neural network. Therefore, each layer generates an intermediate data block when it is executed. Each intermediate data block is then placed (ie, stored) in an allocated area of ​​the global memory area.

[0070] In order to optimize the placement of intermediate data blocks in the global memory area to minimize the size of the global memory area, a method for determining the placement of intermediate data blocks is implemented before executing the neural network.

[0071] The processing unit UT is configured to implement this method for determining the placement of intermediate data chunks.

[0072] Figure 2 A flow chart showing the various steps leading to determining and allocating the size of a global memory region including all intermediate data blocks.

[0073] Step 20 comprises receiving a structure of a neural network to be stored in a non-volatile memory MNV.

[0074] The processing unit UT is configured to extract the architecture, i.e. the configuration of the layers, the operations between the layers and the weights of the neural network.

[0075] The method comprises a step 21 in which the computing unit determines possible placements of intermediate data blocks generated by each layer in a global memory area according to the architecture of the neural network.

[0076] Some placements may result in an increase in the size of the global memory area.

[0077] Depending on the configuration of the neural network (specifically, the order in which each layer is executed), possible placements of intermediate data blocks are selected from multiple predetermined placements.

[0078] For example, Figure 3a , Figure 3b and Figure 3cThe placement shown includes placing the intermediate data block BA in a free area ZL of the global memory area ZG. The free area ZL can be obtained in the global memory area when the area for storing the intermediate data block can be de-allocated during the execution of the neural network. More specifically, the area storing the intermediate data block can be de-allocated during the execution of the neural network when each layer with the intermediate data block as input has been executed.

[0079] exist Figure 3b In the example, the intermediate data block BA is placed in the lower part of the free area ZL, and in Figure 3c In the example above, the intermediate data blocks BA are placed in the upper part of the free area ZL. These placements are referred to below as insertion placements.

[0080] Such a placement allows avoiding increasing the size of the global memory area.

[0081] Figure 4a , Figure 4b , Figure 4c , Figure 4d An "overlay" placement is shown.

[0082] exist Figure 4a In , an attempt is made to place an intermediate data block BA1 generated at the output of a layer in a global memory area ZG having a memory area ZO1 dedicated to storing intermediate data blocks BO1 obtained at the input of this layer.

[0083] The global memory area ZG also has a free area ZL1 between the block BO1 and a further memory area ZO2, and the further memory area ZO2 is dedicated to storing an intermediate data block BO2 from another layer of the neural network. The intermediate data block BA1 has a size s b The free area ZL1 has a size S1 and extends from the height O f Extends to height O f +S1. After the data of the intermediate data block BA1 is calculated, the data of the intermediate data block BO1 as input is no longer used. Figure 4b As shown, during the execution of the neural network, the intermediate data block BA1 can be placed on the area ZO1 and on the free area ZL1 by overwriting part of the intermediate data block BO1 that is the input of this layer. Then, the intermediate data block BA1 is placed from the height O b Extends to height O b +S b .

[0084] In addition, Figure 4cIn FIG. 1 , an attempt is made to place an intermediate data block BA2 generated at the output of a layer of the neural network in a global memory region ZG, which has an area ZO3 for storing an intermediate data block BO3 as input to the same layer. In addition, the global memory region ZG may also have a free area ZL2 between the area ZO3 and one end of the global memory region (here, the lower end of the global memory region). The intermediate data block BA2 has a size S b The free area ZL has a size S1 and extends from the height O f Extends to height O f +S1.

[0085] Here, during the execution of the neural network, after the data of the intermediate data block BA2 is calculated, the data of the intermediate data block BO3 as input is no longer used. Therefore, one possible placement includes: during the execution of the neural network, by overwriting a part of the intermediate data block BO3 as input, the intermediate data block BA2 is placed on the area ZO3 and placed in the free area ZL. However, the size of the global memory area is not enough to be able to include the intermediate data block BA2. Therefore, if Figure 4d As shown, it is necessary to increase the size of the global memory area by defining a new end of the global memory area so that the intermediate data block BA2 can be placed in the global memory area ZG. f The lower end defined at is modified to be at height O b Then, the intermediate data block BA2 is limited at height O b and height O b +S b Extend between.

[0086] Figure 5a , Figure 5b , Figure 5c , Figure 5d A "force allocation" placement is shown. For these placements, the memory size of the global memory area ZG is increased to insert the intermediate data blocks.

[0087] Specifically, in Figure 5a In the example, an attempt is made to place an intermediate data block BA3 generated at the output of one layer of the neural network in a global memory area ZG, which has at least one area ZO1 for storing an intermediate data block BO1 of another layer of the neural network. The global memory area ZG may also have a free area ZL1 between the area ZO1 and the upper end of the global memory area. The intermediate data block BA3 has a size S b The free area ZL has a smaller area than S b size.

[0088] Here, the data of the intermediate data block BO1 as input to this layer cannot be overwritten. In addition, the free area ZL is not enough to place the entire intermediate data block BA3. Therefore, the placement includes increasing the size of the global memory area ZG so that the intermediate data block BA can be placed by redefining the upper end of the global memory area.

[0089] Specifically, in Figure 5b In the height O f +S b The upper limit of the global memory area ZG is defined at the height O so that the intermediate data block BA3 is placed at the top of the global memory area, that is, at the height O f and height O f +S b between.

[0090] Figure 5c The global memory area in which the intermediate data block BA4 must be placed is Figure 5a The difference is that it includes a free area ZL2 located between the intermediate data block ZO2 and the lower end of the global memory area ZG.

[0091] Therefore, in Figure 5c In the example, the lower limit of the global memory area ZG is limited to a height of O. f +S f -S b In order to place the intermediate data block BA4 at the bottom of the global memory area ZG, that is, at the height O f +S f -S b and height O f +S f between.

[0092] Furthermore, a priority order is advantageously established between the various possible placements of each intermediate data block. The highest priority order is given to the placement that is arbitrarily considered to be optimal. For example, it may be advantageous to consider the insertion placement to be optimal relative to the overlay placement and the forced allocation placement. For the independently acquired data blocks, the placement is considered to be optimal, but when the placement sequence is considered as a whole, the placement may not be optimal.

[0093] The following steps of the method allow determining a sequence of placements of intermediate data blocks, allowing obtaining a global memory area of ​​minimum planned size. Each placement of the placement sequence corresponds to the placement of an intermediate data block of a layer of the neural network.

[0094] Thus, the method includes developing an initial sequence of placements of intermediate data blocks generated by each layer of the neural network, from the first layer to the second to last layer of the neural network (22). In other words, the initial sequence of placements is developed by determining the placements of the intermediate data blocks generated by each layer from the shallowest layer to the deepest layer of the neural network, layer by layer.

[0095] In step 23, each placement of the initial sequence is selected from a plurality of possible placements of the intermediate data block to be placed. Specifically, the selected placement is the placement with the highest priority order compared to other possible placements of the intermediate data block to be placed.

[0096] During the development of the initial sequence, for each selected placement, the size of the global memory region ZG is calculated and updated. In addition, the size of the global memory region ZG that would be obtained if other placements were selected from the possible placements is also calculated.

[0097] The initial sequence allows defining the planned size of the global memory region. Specifically, the planned size of the global memory region is the size obtained by the last selected placement.

[0098] The initial sequence is defined as a candidate sequence for memory allocation for storing intermediate data blocks during execution of the neural network.

[0099] The method also includes storing (24) unselected placements of intermediate data blocks for each layer. Also stored is a size of the global memory region that would be available if other placements were selected from the possible placements.

[0100] More specifically, the unselected placements are stored in a stack PL according to their priority order and the depth of their associated layer in the neural network. Thus, the unselected placements of the first layer of the neural network are stored at the bottom of the stack, while the placements of the last layer are stored at the top of the stack.

[0101] exist Figure 6 An example of the development and storage of the initial sequence SI is shown in FIG. 1 . In this example, the initial sequence SI includes three placements. At the beginning of the development, the global memory area has an initial size zi0.

[0102] Specifically, for the first intermediate data block, there can be two placements: d and e. Since the priority order of placement d is higher than that of placement e, the placement d is selected for the first intermediate data block. For the global memory area, the placement d requires a size of zi1. Therefore, the size of the memory area is updated to be equal to zi1. Then, the placement e and the size of the global memory area that can be obtained by selecting the placement are stored in the stack PL.

[0103] Then, the method determines the placement for the second intermediate data block. For this intermediate data block, there are four placements: a, b, d and e. Since the priority order of placement a is higher than that of placement b, d and e, placement a is selected for the second intermediate data block. For the global memory area, this placement requires a size of zi2. Then, in the stack PL, placements b, d and e are stored according to their priority order and the size of the global memory area that can be obtained by selecting these placements.

[0104] Then, the method determines the placement for the third intermediate data block. For this intermediate data block, there are three placements: b, d and e. Since the priority order of placement b is higher than that of placement d and e, placement b is selected for the third intermediate data block. For the global memory area, this placement requires size zf. Then, placement d and e and the size of the global memory area that can be obtained by selecting these placements are stored in the stack PL.

[0105] The placement has been selected for all intermediate data blocks, and then the initial sequence is completed. Therefore, size zf is specified as the planned size of the global memory area for this initial placement sequence.

[0106] The initial sequence may not be the optimal sequence of placement, ie, the placement sequence that allows the size of the global memory region to be minimized. In particular, the placement that can be arbitrarily selected as optimal may not be the placement that will ultimately allow the size of the global memory region to be reduced as much as possible.

[0107] Therefore, if Figure 2 As shown, the method includes: developing at least one modified placement sequence relative to the initial sequence based on (from) a given placement of the initial sequence. The development of the modified sequence includes modifying the given placement by an unselected placement stored in a stack. The unselected placement and the given placement are associated with the same intermediate data block.

[0108] Preferably, the modification sequence is developed taking into account the order of placement in the stack layers.

[0109] Specifically, the placement at the top of the stack is the placement selected to modify the initial sequence placement associated with the same intermediate data block.The placement used to modify the initial sequence is then removed from the stack.

[0110] The modified sequence is the same as the initial sequence until the placement is replaced by the placement at the top of the stack. In other words, the initial sequence placement of data blocks associated with the shallower layers is maintained in the modified sequence compared to the initial sequence placement of data blocks associated with the middle data blocks whose placement has been modified.

[0111] Then, choose the placement of the data blocks for subsequent layers.

[0112] Once all layers have been processed and placement determined for each intermediate data block, the modification sequence is complete.

[0113] Specifically, the placement of the intermediate data blocks is selected from the possible placements of the intermediate data blocks according to the established priority order.

[0114] For the initial sequence, the sizes of the global memory regions that were not selected for placement and would be available if those placements were selected are stored.

[0115] The method can then develop other modified sequences based on the initial sequence. Specifically, for each new modified sequence to be developed, the placement of the initial sequence is maintained until the placement associated with the intermediate data block is associated with the placement placed at the top of the stack. The placement is replaced by the placement placed at the top of the stack. Then, as described above, placements are selected for the intermediate data blocks of the subsequent layers, and the unselected placements and the sizes of the global memory areas that would be available if these placements were selected are stored.

[0116] Preferably, a sequence of modifications is developed until the stack is empty.

[0117] Thus, the method allows multiple placement sequences to be developed based on an initial placement sequence. Figure 7a An example of a placement sequence is shown in FIG. In this example, the placement sequence SI is an initial sequence. Specifically, a plurality of modified placement sequences are developed, including a modified sequence SM. Compared to the initial sequence SI, the modified sequence SM allows a reduced global memory region size to be obtained. Therefore, the modified sequence becomes a new candidate sequence NSC.

[0118] The development of multiple modification sequences allows testing different possible placements in order to determine the best placement sequence, ie the placement sequence that allows minimizing the planned size of the global memory region.

[0119] Specifically, if the planned size of the global memory region obtained by the modified sequence is smaller than the planned size of the global memory region of the previous candidate sequence, each modified sequence may be defined as a new candidate sequence that replaces the previous candidate sequence.

[0120] Therefore, for each completed modification sequence, the method includes comparing the planned size of the global memory area obtained by the completed modification sequence with the planned size of the global memory area obtained by the candidate sequence. If the planned size of the global memory area obtained by the completed modification sequence is less than the size defined by the candidate sequence, the completed modification sequence becomes a new candidate sequence.

[0121] Therefore, the placement sequence that is a candidate sequence when the stack becomes empty is the sequence selected for the allocation of intermediate data blocks during the execution of the neural network. In step 26, this candidate sequence is selected as the optimized placement sequence.

[0122] Preferably, if Figure 7b As shown, before developing the modification sequence, placements that imply increasing the size of the global memory area to a stack larger than the planned size of the global memory area obtained with the candidate sequence are removed from the queue to avoid developing the modification sequence from these placements. Thus, this removal allows the execution of the method to be accelerated.

[0123] In addition, if Figure 7b As shown, preferably, before developing the modification sequence, the placements of the stack associated with the same intermediate data block are arranged according to the size of the global memory area that would be available if these placements were previously selected. Specifically, the placement that allows obtaining the size of the smallest global memory area relative to other placements associated with the same intermediate data block is placed above these other placements in the stack.

[0124] Other arrangements of the stack are also possible. Specifically, the placement can be arranged in the stack by considering its proximity to the execution of the last layer of the neural network. This arrangement allows obtaining a faster execution of the method.

[0125] As mentioned before, the placement can also be arranged according to the size of the global memory area that has been obtained and also by considering the proximity of the placement with respect to the execution of the last layer of the neural network in the case where the global memory size between at least two placements of the stack is equal.

[0126] The size of the stack can also be limited to speed up the method. The smaller the stack size, the greater the risk of not finding the optimal placement sequence. Therefore, preferably, the size of the stack will be limited to a number of placements comprised between 1 and 100, more specifically between 1 and 50, preferably between 1 and 30.

[0127] Once a placement sequence that is considered optimal has been determined, the neural network can be executed.

[0128] At the beginning of the execution of the neural network, the processing unit UT is configured to allocate in the volatile memory MV a global memory area whose size is equal to the planned size obtained by the placement sequence considered to be optimal.

[0129] Once the global memory area is allocated, the processing unit UT is configured to calculate the absolute addresses of each placement of the intermediate data blocks considered as the optimal placement sequence and then to place the intermediate data blocks according to these placements.

[0130] The "absolute address" refers to the address of the location of the intermediate data block selected in the global storage area, which is used to store the intermediate data blocks of the neural network.

[0131] The method for determining the placement of intermediate data blocks allows finding an optimal placement for each intermediate data block to minimize the size of the global memory area.

[0132] By minimizing the size of the global memory area, the method for determining placement allows the use of integrated circuits with smaller memory sizes. Thus, this method allows the use of inexpensive integrated circuits.

[0133] Although the present invention has been described with reference to exemplary embodiments, this description is not intended to be interpreted in a limiting sense. With reference to the specification, those skilled in the art will appreciate various modifications and combinations of the illustrative embodiments and other embodiments of the present invention. Therefore, the appended claims include any such modifications or embodiments.

Claims

1. A method for determining placement of intermediate data blocks generated by a layer of an artificial neural network for memory allocation in a global memory area, the method comprising: defining a plurality of possible placements of the intermediate data blocks for each layer in the global memory region; Assign a priority order to each possible placement of each intermediate data block; developing an initial sequence of placements of the intermediate data blocks for each layer from the first to the last layer of the neural network, selecting each placement of the initial sequence from the plurality of possible placements of the intermediate data blocks to be placed, the initial sequence allowing to define the planned size of the global memory area, the initial sequence being set as a candidate sequence; storing in a stack unselected placements of the intermediate data blocks of each layer according to the priority order of the intermediate data blocks and the depth in the artificial neural network of a corresponding layer associated with each intermediate data block; developing from a given placement of the initial sequence at least one placement sequence modified compared to the initial sequence, the developing comprising modifying the given placement by a stored non-selected placement; According to the planned size of the global memory area obtained by the modified sequence being smaller than the planned size of the global memory area of ​​the candidate sequence, setting the modified sequence as the candidate sequence; as well as The placement sequence set as the candidate sequence once each modification sequence is developed is used as the placement of the intermediate data blocks for the memory allocation.

2. The method of claim 1, wherein the selection of placement of each intermediate data block for developing the initial sequence is performed according to the priority order of placement. The method of claim 2 , wherein storing the unselected placements is performed from bottom to top in the stack.

4. The method according to claim 3, further comprising: developing each modified sequence such that each modified sequence maintains the same placement as the initial sequence until a placement associated with the same intermediate data block as the placement placed at the top of the stack, the placement of the primary sequence is replaced by the placement placed at the top of the stack and then removed from the stack; as well as A placement of intermediate data blocks of a subsequent layer of the modification sequence is selected from possible placements for these intermediate data blocks, the non-selected placements being stored in the stack.

5. The method according to claim 4, further comprising: The modification sequence is developed until the stack is empty.

6. The method of claim 4, wherein each placement stored in the stack is associated with a size of the global memory region obtainable by selecting that placement from possible placements of intermediate data blocks.

7. The method according to claim 6, further comprising: before each development of the modified sequence, comparing the size of the global memory region associated with each placement of the stack with a planned size of the global memory region obtained for the candidate sequence; as well as A placement is removed from the stack, the placement being associated with a size of the global memory region that is greater than the planned size.

8. The method according to claim 6, further comprising: Before each development of the modification sequence, the placement of the stack associated with the same intermediate data block is organized according to the size of the global memory area associated therewith and according to the execution order of these placements.

9. The method of claim 3, wherein the stack has a finite size.

10. The method according to claim 1, wherein a region of the global memory area selected for placement of an intermediate data block is considered to be released as soon as placement of data blocks of all layers as inputs of the intermediate data block placed in the region is selected.

11. The method of claim 10, wherein the defined placement of the intermediate data block comprises placing the intermediate data block in a free area of ​​the global memory area.

12. The method of claim 1, wherein the defined placement of an intermediate data block generated by a layer comprises superimposing the intermediate data block on an intermediate data block that is input to the same layer.

13. The method of claim 1, wherein the defining placement of the intermediate data block comprises increasing a memory size of the global memory area to insert the intermediate data block.

14. A method for allocating global memory size, the method comprising: defining a plurality of possible placements of intermediate data blocks generated by layers of the artificial neural network in a global memory region; Assign a priority order to each possible placement of each intermediate data block; developing an initial sequence of placements of the intermediate data blocks for each layer from the first to the last layer of the neural network, selecting each placement of the initial sequence from the plurality of possible placements of the intermediate data blocks to be placed, the initial sequence allowing to define the planned size of the global memory area, the initial sequence being set as a candidate sequence; storing in a stack unselected placements of the intermediate data blocks of each layer according to the priority order of the intermediate data blocks and the depth in the artificial neural network of a corresponding layer associated with each intermediate data block; developing from a given placement of the initial sequence at least one placement sequence modified compared to the initial sequence, the developing comprising modifying the given placement by a stored non-selected placement; According to the planned size of the global memory area obtained by the modified sequence being smaller than the planned size of the global memory area of ​​the candidate sequence, setting the modified sequence as the candidate sequence; using the placement sequence set as the candidate sequence once each modification sequence is developed as the placement of the intermediate data blocks for memory allocation; selecting the global memory size to be equal to the planned size obtained by the candidate sequence; as well as Memory is allocated based on a determined contiguous placement.

15. The method of claim 14, wherein the selection of placement of each intermediate data block for developing the initial sequence is performed according to the priority order of placement. The method of claim 15 , wherein storing the unselected placements is from bottom to top in the stack.

17. The method according to claim 16, further comprising: developing each modified sequence such that each modified sequence maintains the same placement as the initial sequence until a placement associated with the same intermediate data block as the placement placed at the top of the stack, the placement of the primary sequence is replaced by the placement placed at the top of the stack and then removed from the stack; as well as A placement of intermediate data blocks of a subsequent layer of the modification sequence is selected from possible placements for these intermediate data blocks, the non-selected placements being stored in the stack.

18. An integrated circuit configured to receive a multi-layer artificial neural network, the integrated circuit comprising: a memory configured to store in a global memory region intermediate data blocks generated by each of the multiple layers of the neural network; as well as a processing unit coupled to the memory, the processing unit being configured to: defining a plurality of possible placements of the intermediate data blocks for each layer in the global memory region; Assign a priority order to each possible placement of each intermediate data block; developing an initial sequence of placements of the intermediate data blocks for each layer from the first to the last layer of the neural network, selecting each placement of the initial sequence from the plurality of possible placements of the intermediate data blocks to be placed, the initial sequence allowing to define the planned size of the global memory area, the initial sequence being set as a candidate sequence; storing in a stack unselected placements of the intermediate data blocks of each layer according to the priority order of the intermediate data blocks and the depth in the artificial neural network of a corresponding layer associated with each intermediate data block; developing, based on a given placement of the initial sequence, at least one placement sequence modified compared to the initial sequence, the developing comprising modifying the given placement by a stored non-selected placement; According to the planned size of the global memory area obtained by modifying the sequence being smaller than the planned size of the global memory area of ​​the candidate sequence, setting the modified sequence as the candidate sequence; using the placement sequence that was set as the candidate sequence once each modification sequence was developed as placement for the intermediate data block in the global memory area; as well as During execution of the neural network: Selecting a global memory size equal to the planned size obtained by the candidate sequence; and Allocate memory based on determined contiguous placement.

19. The integrated circuit of claim 18, wherein the selection of placement of each intermediate data block for developing the initial sequence is performed according to the priority order of placement.

20. The integrated circuit of claim 19, wherein the memory is further configured to store the unselected placements from bottom to top in the stack.

21. The integrated circuit of claim 20, wherein the processing unit is further configured to: developing each modified sequence such that each modified sequence maintains the same placement as the initial sequence until a placement associated with the same intermediate data block as the placement placed at the top of the stack, the placement of the primary sequence being replaced by the placement placed at the top of the stack and then removed from the stack; and A placement of intermediate data blocks of a subsequent layer of the modification sequence is selected from possible placements for these intermediate data blocks, the non-selected placements being stored in the stack.

Images