Memory allocation method and device, chip, electronic equipment and storage medium

Abstract

The invention discloses a memory allocation method and device, a chip, electronic equipment and a storage medium. According to the main technical scheme, the method comprises the steps that life cycles of at least two data blocks corresponding to a target operator of a neural network are analyzed in sequence; when it is determined that the life cycle of the target data block meets a preset allocation condition, a corresponding first memory space pair is allocated to the target data block, and the first memory space pair is two adjacent memory spaces in a first memory; loading target calculation corresponding to the target data block to the first calculation memory space to execute calculation, and loading data required by next calculation of the target calculation to the second calculation memory space; compared with the prior art, the method has the advantages that the limitation of memory isolation of each calculation is broken, hardware can be combined for optimization when life cycle analysis is carried out on the at least two data blocks corresponding to the target operator, parallel execution of target calculation and loading of next calculation is realized, and the utilization rate of hardware resources is further improved.

Description

Technical Field

[0001] The present application relates to the field of chip technology, and in particular to a method and device for allocating memory, a chip, an electronic device, and a storage medium. Background Art

[0002] With the advancement of technology, the demand for high-performance parallel computing in the consumer electronics market is exploding, especially in machine vision, artificial intelligence, cloud computing, software-defined radio and other emerging fields, which have great demands for heterogeneous computing systems. Heterogeneous computing enables computing units on the same system-on-chip to complete their respective computing tasks through programming, thus achieving higher efficiency and lower power consumption than a single computing unit.

[0003] In order for neural network models to be efficiently computed on heterogeneous platforms, in addition to high-performance hardware computing platforms, efficient compiler compilation is also required. The compiler must not only adapt to changes in the neural network model algorithm, but also meet the demands of diverse hardware and the resource requirements of different deployment environments in the final binary output.

[0004] In order to solve the problem of small memory capacity on the device side, one calculation needs to be split into multiple calculations. Each calculation after splitting is called a tiling. The loading of the data corresponding to each tiling also needs to be divided into multiple loadings into the device side memory. When allocating memory for the data corresponding to the current tiling input and the output data, by default, each subsequent tiling can reuse all the memory of the previous tiling. If the memory occupied by the data of the current tiling has not been released, the data of the next tiling cannot be loaded. Summary of the invention

[0005] The present application provides a method, device, chip, electronic device and storage medium for allocating memory, and its main purpose is to solve the problem in the related art that the next tiling data cannot be loaded when the memory occupied by the current tiling data has not been released.

[0006] According to a first aspect of the present application, a method for allocating memory is provided, comprising:

[0007] The life cycles of at least two data blocks corresponding to the target operator of the neural network are analyzed in sequence; each data block corresponding to the target operator of the neural network is a data block occupied in the first memory by data required for each calculation after the target operator is split into at least two calculations;

[0008] In the case of determining that the life cycle of the target data block meets the preset allocation condition, allocating a corresponding first memory space pair for the target data block, the first memory space pair being two adjacent memory spaces in the first memory, including a first computing memory space and a second computing memory space;

[0009] The target calculation corresponding to the target data block is loaded into the first calculation memory space for execution, and the data required for the next calculation of the target calculation is loaded into the second calculation memory space.

[0010] In some embodiments, the step of loading the target calculation corresponding to the target data block into the first calculation memory space for performing the calculation comprises:

[0011] The target calculation of the target operator in the second memory is loaded into the memory address corresponding to the first moment in the first calculation memory space to perform calculation, the first calculation memory space has reserved memory space for storing N moments, and the second memory is the input side of the target operator in the first memory.

[0012] In some embodiments, after loading data required for a next calculation of the target calculation into the second calculation memory space, the method further includes:

[0013] Using the output result of the target operator as the input of the next operator;

[0014] Alternatively, the output result of the target operator is cached in the first memory, and then the output result of the target operator is moved from the first memory to the second memory.

[0015] In some embodiments, caching the output result of the target operator in the first memory includes:

[0016] The output result of the target operator in the first computing memory space is stored in the first storage memory space in the second memory pair; wherein the second memory pair is two adjacent memory spaces in the first memory, including the first storage memory space and the second storage memory space;

[0017] The output result of the target operator in the second computing memory space is stored in the second storage memory space in the second memory pair.

[0018] In some embodiments, after caching the output result of the target operator in the first memory, the method further includes:

[0019] Determining whether the first memory pair and the second memory pair meet a preset merging condition;

[0020] If it is determined that the first memory pair and the second memory pair satisfy the preset merging condition, determining a first address of the computing memory space in the first memory pair loaded by the second memory;

[0021] Determine a second address of the computing memory space output to the second memory in the second memory pair;

[0022] The memory space corresponding to the first address and the memory space corresponding to the second address are merged to obtain a merged memory space.

[0023] In some embodiments, merging the memory space corresponding to the first address with the memory space corresponding to the second address includes:

[0024] Determine a first space length of the first address and a second space length of the second address;

[0025] The relatively larger space length between the first space length and the second space length is used as the target space length;

[0026] The memory space corresponding to the first address and the memory space corresponding to the second address are merged based on the target space length to obtain the merged memory space.

[0027] In some embodiments, after the target calculation corresponding to the target data block is loaded into the first calculation memory space for execution, the method further includes:

[0028] Determine an unallocated space address in the first computing memory space;

[0029] The unallocated space address is further allocated to other operators or a calculation corresponding to other operators, and the other operators are operators other than the target operator.

[0030] According to a second aspect of the present application, a device for allocating memory is provided, comprising:

[0031] An analysis unit, used to analyze the life cycles of at least two data blocks corresponding to the target operator of the neural network in sequence; each data block corresponding to the target operator of the neural network is a data block occupied in the first memory by data required for each calculation after the target operator is split into at least two calculations;

[0032] A first allocation unit is used to allocate a corresponding first memory space pair to the target data block when it is determined that the life cycle of the target data block meets a preset allocation condition, wherein the first memory space pair is two adjacent memory spaces in the first memory, including a first computing memory space and a second computing memory space;

[0033] A loading unit is used to load the target calculation corresponding to the target data block into the first calculation memory space to perform the calculation, and load the data required for the next calculation of the target calculation into the second calculation memory space.

[0034] In some embodiments, the loading unit is also used to load the target calculation of the target operator in the second memory to the memory address corresponding to the first moment in the first calculation memory space to perform calculation, and the first calculation memory space is reserved with memory space for storing N moments, and the second memory is the input side of the target operator in the first memory.

[0035] In some embodiments, the apparatus further comprises:

[0036] a first determining unit, configured to use the output result of the target operator as the input of the next operator after the loading unit loads the data required for the next calculation of the target calculation into the second calculation memory space;

[0037] A cache unit, used to cache the output result of the target operator in the first memory;

[0038] A transport unit is used to transport the output result of the target operator from the first memory to the second memory.

[0039] In some embodiments, the cache unit is further configured to:

[0040] The output result of the target operator in the first computing memory space is stored in the first storage memory space in the second memory pair; wherein the second memory pair is two adjacent memory spaces in the first memory, including the first storage memory space and the second storage memory space;

[0041] The output result of the target operator in the second computing memory space is stored in the second storage memory space in the second memory pair.

[0042] In some embodiments, the apparatus further comprises:

[0043] A second determining unit, configured to determine whether the first memory pair and the second memory pair meet a preset merging condition after the cache unit caches the output result of the target operator in the first memory;

[0044] The third determining unit is further configured to determine a first address of a computing memory space in the first memory pair loaded by the second memory when it is determined that the first memory pair and the second memory pair satisfy the preset merging condition;

[0045] a fourth determining unit, configured to determine a second address of the computing memory space output to the second memory in the second memory pair;

[0046] A merging unit is used to merge the memory space corresponding to the first address with the memory space corresponding to the second address to obtain a merged memory space.

[0047] In some embodiments, the merging unit comprises:

[0048] Determine a first space length of the first address and a second space length of the second address;

[0049] The relatively larger space length between the first space length and the second space length is used as the target space length;

[0050] The memory space corresponding to the first address and the memory space corresponding to the second address are merged based on the target space length to obtain the merged memory space.

[0051] In some embodiments, the apparatus further comprises:

[0052] A fifth determining unit, configured to determine an unallocated space address in the first computing memory space after the loading unit loads the target computing corresponding to the target data block into the first computing memory space for computing and performs computing;

[0053] The second allocation unit is further used to continue to allocate the unallocated space address to other operators or a calculation corresponding to other operators, and the other operators are operators other than the target operator.

[0054] According to a third aspect of the present application, an electronic device is provided, including:

[0055] at least one processor; and

[0056] a memory communicatively connected to the at least one processor; wherein,

[0057] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described in the first aspect.

[0058] According to the fourth aspect of the present application, a chip is provided, comprising one or more interface circuits and one or more processors; the interface circuit is used to receive a signal from a memory of an electronic device and send the signal to the processor, the signal comprising a computer instruction stored in the memory, and when the processor executes the computer instruction, the electronic device executes the method described in the first aspect above.

[0059] According to a fifth aspect of the present application, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method described in the first aspect above.

[0060] According to a sixth aspect of the present application, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the method according to the first aspect is implemented.

[0061] The method, device, chip, electronic device and storage medium for allocating memory provided in the present application analyze the life cycles of at least two data blocks corresponding to the target operator of the neural network in turn; each data block corresponding to the target operator of the neural network is a data block occupied in the first memory by the data required for each calculation after the target operator is split into at least two calculations; when it is determined that the life cycle of the target data block meets the preset allocation conditions, a corresponding first memory space pair is allocated to the target data block, and the first memory space pair is two adjacent memory spaces in the first memory, including a first calculation memory space and a second calculation memory space; the target calculation corresponding to the target data block is loaded into the first calculation memory space for execution, and the data required for the next calculation of the target calculation is loaded into the second calculation memory space; compared with the related art, the embodiment of the present application breaks the limitation of the isolation of each calculation memory, so that it can be optimized in combination with hardware when performing the life cycle analysis of at least two data blocks corresponding to the target operator, so as to realize the parallel execution of the target calculation and the loading of the next calculation, and further improve the utilization rate of hardware resources.

[0062] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present application.

[0064] Figure 1 A flowchart of a method for allocating memory provided in an embodiment of the present application;

[0065] Figure 2 A schematic diagram of data interaction between a first memory and a second memory provided in an embodiment of the present application;

[0066] Figure 3 A schematic diagram of a first memory space pair provided in an embodiment of the present application;

[0067] Figure 4A schematic diagram of a first memory pair and a second memory pair provided in an embodiment of the present application;

[0068] Figure 5 A schematic diagram of a merged memory space provided in an embodiment of the present application;

[0069] Figure 6 A schematic diagram of memory fragmentation after a ticking buffer provided in an embodiment of the present application;

[0070] Figure 7 A schematic diagram of memory fragmentation of a first memory space pair provided in an embodiment of the present application;

[0071] Figure 8 A schematic diagram of the structure of a device for allocating memory provided in an embodiment of the present application;

[0072] Fig. 9 A schematic diagram of the structure of another device for allocating memory provided in an embodiment of the present application;

[0073] Fig.10 A schematic block diagram of an example electronic device provided for an embodiment of the present application. DETAILED DESCRIPTION

[0074] The following is a description of exemplary embodiments of the present application in conjunction with the accompanying drawings, including various details of the embodiments of the present application to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0075] The following describes the method, device, chip, electronic device and storage medium for allocating memory according to the embodiments of the present application with reference to the accompanying drawings.

[0076] Figure 1 A flowchart of a method for allocating memory provided in an embodiment of the present application.

[0077] like Figure 1 As shown, the method comprises the following steps:

[0078] Step 101, analyze the life cycles of at least two data blocks corresponding to the target operator of the neural network in turn; each data block corresponding to the target operator of the neural network is a data block occupied by the data required for each calculation in the first memory after the target operator is split into at least two calculations.

[0079] For ease of understanding, Figure 2 As shown, Figure 2A schematic diagram of data interaction between a first memory and a second memory provided in an embodiment of the present application. Before the neural network model starts calculating on a heterogeneous platform, the model parameters and model inputs need to be moved from the second memory (host memory) to the first memory (device memory). Each data block in the first memory has a memory address in the first memory, and then the host sends instructions to the device, including the memory address of the operation and the type of calculation. After the calculation is completed, the result is cached in the first memory, and the cached data also has a corresponding address in the first memory, and finally the result is output to the second memory.

[0080] The life cycle analysis of the target operator can be implemented in the following ways but is not limited to: sequentially traverse the target operator in the neural network, determine the branch structure of the target operator, and determine the calculation order of the target operator in the branch structure. It should be noted that the target operator is any operator currently being processed in the neural network, not just one operator.

[0081] In the embodiments of the present application, in order to adapt to the processing resources on the device side, a target operator is usually split into multiple calculations (tiling). For example, when the processing resources on the device side are sufficient to process one target operator, the target operator may not be split. When the processing resources on the device side can only process 1 / 10 of the target operator at a time, the target operator may be split into 10 calculations. Specifically, the number of splits of the target operator is not limited in the embodiments of the present application.

[0082] Step 102, when it is determined that the life cycle of the target data block meets the preset allocation conditions, a corresponding first memory space pair is allocated to the target data block, the first memory space pair being two adjacent memory spaces in the first memory, including a first computing memory space and a second computing memory space.

[0083] After completing the life cycle analysis of at least two data blocks corresponding to the target operator, the start time start_ts, end time end_ts and offset_flag of each data block are obtained, where start_ts indicates that the memory is applied from this moment, end_ts indicates that the memory is released after this moment, offset_flag=0 indicates that the start time and the end time belong to the same calculation, that is, the same tiling, and offset_flag=1 indicates that the start time and the end time belong to two adjacent tilings.

[0084] It should be noted that when the target operator is input into the first memory, offset_flag carries the identifier of offset_flag, and offset_flag can be obtained by parsing it.

[0085] The preset allocation condition in the embodiment of the present application is that when offset_flag=1 and start_ts is less than or equal to end_ts, the target data block needs to allocate twice the memory space, that is, the corresponding first memory space pair is allocated to the target data block, also known as ping-pong buffer, and the first memory space pair (ping-pong buffer) is two adjacent memory spaces in the first memory, including a first memory space for calculation (ping-pong buffer) and a second memory space for calculation (pong buffer).

[0086] In order to facilitate the understanding of the first memory space pair (ping-pong buffer), as follows Figure 3 As shown, Figure 3 A schematic diagram of a first memory space pair provided for an embodiment of the present application, wherein mem_bytes represents the first memory, the white square L represents the input data buffer, that is, the data that needs to be loaded into the first memory, and the white square S represents the output result after the calculation of op2 is completed, which is cached in the first memory (device-side memory). t0, t1, and t2 represent three moments of a tiling, which represent the life cycle of the relevant memory. op0, op1, and op2 represent three operators. The input of op0 is the loaded data, the output of op0 is the input of op1, and the output of op1 is the input of op2. The output of op2 is first cached in the first memory, and then stored in the second memory on the host side.

[0087] Step 103: Load the target calculation corresponding to the target data block into the first calculation memory space to perform the calculation, and load the data required for the next calculation of the target calculation into the second calculation memory space.

[0088] The target calculation (first calculation) is loaded into the first calculation memory space for execution, and the data required for the next calculation (second calculation) of the target calculation is loaded into the second calculation memory space. After storing the calculation result of the first calculation, the second calculation is calculated at the same time, and the third calculation is loaded into the first calculation memory space, and the cycle continues. Since the loading of data and the calculation can be performed in parallel, the hardware resources are efficiently utilized.

[0089] The method for allocating memory provided by the present application analyzes the life cycles of at least two data blocks corresponding to the target operator of the neural network in sequence; each data block corresponding to the target operator of the neural network is a data block occupied in the first memory by the data required for each calculation after the target operator is split into at least two calculations; when it is determined that the life cycle of the target data block meets the preset allocation conditions, a corresponding first memory space pair is allocated to the target data block, and the first memory space pair is two adjacent memory spaces in the first memory, including a first calculation memory space and a second calculation memory space; the target calculation corresponding to the target data block is loaded into the first calculation memory space for execution, and the data required for the next calculation of the target calculation is loaded into the second calculation memory space; compared with the related art, the embodiment of the present application breaks the limitation of the isolation of each calculation memory, so that it can be optimized in combination with hardware when performing the life cycle analysis of at least two data blocks corresponding to the target operator, so as to realize the parallel execution of the target calculation and the loading of the next calculation, and further improve the utilization rate of hardware resources.

[0090] As a refinement of the above step 103, the target calculation corresponding to the target data block is loaded into the first computing memory space for execution of the calculation, including: the target calculation of the target operator in the second memory is loaded into the memory address corresponding to the first moment in the first computing memory space for execution of the calculation, the first computing memory space has reserved memory space for storing N moments, and the second memory is the input side of the target operator in the first memory. Among them, the size of N is related to the life cycle of the target operator, and the specific value is not limited in the embodiment of this application.

[0091] Please continue reading Figure 3 , load one of the calculations corresponding to the target operator according to the ping-pong buffer allocated according to the life cycle. During the first tiling (ping time) calculation process, op0 uses the first calculation memory space (ping buffer), and at the same time loads the data required for the next tiling into the second calculation memory space (pong buffer). During the second tiling (pong time) calculation process, op0 uses the second calculation memory space (pong buffer), and at the same time loads the data required for the next tiling into the first calculation memory space (ping buffer), and so on.

[0092] Based on the computing characteristics of the neural network, after the data required for the next calculation of the target calculation is loaded into the second computing memory space, the output result of the target operator is used as the input of the next operator (scenario one); or, the output result of the target operator is cached in the first memory, and then the output result of the target operator is moved from the first memory to the second memory (scenario two).

[0093] For the above scenario one, caching the output result of the target operator in the first memory includes: storing the output result of the target operator in the first computing memory space in the first storage memory space in the second memory pair; wherein the second memory pair is two adjacent memory spaces in the first memory, including the first storage memory space and the second storage memory space; storing the output result of the target operator in the second computing memory space in the second storage memory space in the second memory pair.

[0094] In order to facilitate understanding of the first memory pair and the second memory pair, as shown in FIG. Figure 4 As shown, Figure 4 A schematic diagram of a first memory pair and a second memory pair provided in an embodiment of the present application, a first computing memory space (load ping-pong buffer) and a second computing memory space (load-pong buffer), a first storage memory space (store ping-pong buffer) and a second storage memory space (store-pong buffer), Figure 4 DDR refers to the second memory space, and the direction of the arrow indicates the direction in which data is transferred between the second memory and the second memory.

[0095] In order to further save space resources in the first memory, after caching the output result of the target operator in the first memory, determine whether the first memory pair and the second memory pair meet the preset merging condition; if it is determined that the first memory pair and the second memory pair meet the preset merging condition, determine the first address of the computing memory space loaded by the second memory in the first memory pair; determine the second address of the computing memory space output to the second memory in the second memory pair; merge the memory space corresponding to the first address with the memory space corresponding to the second address to obtain a merged memory space.

[0096] In the embodiment of the present application, the ping-pong buffer (ping-pong load buffer and ping-pong store buffer) is merged into a merged memory space (tick-tick buffer). The tick-tick buffer represents that there are three blocks of memory in the first memory, and these three blocks of memory are cyclically used in every three tilings.

[0097] In an embodiment of the present application, the preset merge condition is set as the end_ts of the load buffer is less than the start_ts of the store buffer, and both the load buffer and the store buffer are allocated ping-pong buffers. When it is determined that the first memory pair and the second memory pair meet the preset merge condition, the first address of the computing memory space (load ping-pong buffer or load-pong buffer) loaded by the second memory in the first memory pair is determined, and the second address of the computing memory space (store ping-pong buffer or store-pong buffer) output to the second memory in the second memory pair is determined. The memory space corresponding to the first address is merged with the memory space corresponding to the second address to obtain a merged memory space. Figure 5 As shown, Figure 5 A schematic diagram of a merged memory space provided in an embodiment of the present application.

[0098] As a refinement of the above embodiment, when merging the memory space corresponding to the first address and the memory space corresponding to the second address, the following method can be used but is not limited to: determining the first space length of the first address and the second space length of the second address, taking the relatively larger space length between the first space length and the second space length as the target space length, and merging the memory space corresponding to the first address and the memory space corresponding to the second address based on the target space length to obtain the merged memory space.

[0099] In some embodiments, memory is defragmented after allocating the tick buffer, such as Figure 6 As shown, Figure 6 A schematic diagram of memory fragmentation after a tick buffer is provided in an embodiment of the present application. Since the original ping-pong buffer (ping-pong load buffer and ping-pong store buffer) has been merged, the memory fragments left by the original ping-pong buffer need to be deleted first. After allocating a new tick buffer, new memory fragments will be generated. The memory fragments are as follows: Figure 6 As shown, Figure 6 The size here indicates the memory size required by the buffer (the first space length or the second space length).

[0100] Presented by Figure 3After allocating a ping-pong buffer, corresponding memory fragments will be generated. In order to utilize the unallocated space, after the target calculation corresponding to the target data block is loaded into the first calculation memory space for calculation, the unallocated space address in the first calculation memory space is determined, and the unallocated space address is further allocated to other operators or a calculation corresponding to other operators. The other operators are operators other than the target operator. Figure 7 The memory occupied at the first and second moments shown is unallocated space. The addresses of these unallocated spaces are recorded during the compilation process and can be allocated to other buffers for use, thereby efficiently utilizing the memory.

[0101] Corresponding to the above-mentioned method for allocating memory, the present invention also provides a device for allocating memory. Since the device embodiment of the present invention corresponds to the above-mentioned method embodiment, details not disclosed in the device embodiment can be referred to the above-mentioned method embodiment, and will not be repeated in the present invention.

[0102] Figure 8 A schematic diagram of a memory allocation device provided in an embodiment of the present application is shown in FIG. Figure 8 As shown, including:

[0103] The analysis unit 81 is used to analyze the life cycles of at least two data blocks corresponding to the target operator of the neural network in sequence; each data block corresponding to the target operator of the neural network is a data block occupied in the first memory by the data required for each calculation after the target operator is split into at least two calculations;

[0104] A first allocation unit 82 is used to allocate a corresponding first memory space pair to the target data block when it is determined that the life cycle of the target data block meets a preset allocation condition, wherein the first memory space pair is two adjacent memory spaces in the first memory, including a first computing memory space and a second computing memory space;

[0105] The loading unit 83 is used to load the target calculation corresponding to the target data block into the first calculation memory space to perform the calculation, and load the data required for the next calculation of the target calculation into the second calculation memory space.

[0106] The memory allocation device provided in the present application analyzes the life cycles of at least two data blocks corresponding to the target operator of the neural network in sequence; each data block corresponding to the target operator of the neural network is a data block occupied by the data required for each calculation in the first memory after the target operator is split into at least two calculations; when it is determined that the life cycle of the target data block meets the preset allocation conditions, a corresponding first memory space pair is allocated to the target data block, and the first memory space pair is two adjacent memory spaces in the first memory, including a first calculation memory space and a second calculation memory space; the target calculation corresponding to the target data block is loaded into the first calculation memory space for execution, and the data required for the next calculation of the target calculation is loaded into the second calculation memory space; compared with the related art, the embodiment of the present application breaks the limitation of the isolation of each calculation memory, so that it can be optimized in combination with hardware when performing the life cycle analysis of at least two data blocks corresponding to the target operator, so as to realize the parallel execution of the target calculation and the loading of the next calculation, and further improve the utilization rate of hardware resources.

[0107] Furthermore, in a possible implementation of this embodiment, as Fig. 9 As shown, the loading unit 83 is also used to load the target calculation of the target operator in the second memory into the memory address corresponding to the first moment in the first calculation memory space to perform calculation, and the first calculation memory space is reserved with memory space for storing N moments, and the second memory is the input side of the target operator in the first memory.

[0108] Furthermore, in a possible implementation of this embodiment, as Fig. 9 As shown, the device also includes:

[0109] A first determining unit 84 is configured to use the output result of the target operator as the input of the next operator after the loading unit 83 loads the data required for the next calculation of the target calculation into the second calculation memory space;

[0110] A cache unit 85, configured to cache the output result of the target operator in the first memory;

[0111] The transfer unit 86 is used to transfer the output result of the target operator from the first memory to the second memory.

[0112] Furthermore, in a possible implementation of this embodiment, as Fig. 9 As shown, the cache unit 85 is also used for:

[0113] The output result of the target operator in the first computing memory space is stored in the first storage memory space in the second memory pair; wherein the second memory pair is two adjacent memory spaces in the first memory, including the first storage memory space and the second storage memory space;

[0114] The output result of the target operator in the second computing memory space is stored in the second storage memory space in the second memory pair.

[0115] Furthermore, in a possible implementation of this embodiment, as Fig. 9 As shown, the device also includes:

[0116] A second determining unit 87, configured to determine whether the first memory pair and the second memory pair satisfy a preset merging condition after the cache unit 85 caches the output result of the target operator in the first memory;

[0117] The third determining unit 88 is further configured to determine a first address of a computing memory space in the first memory pair loaded by the second memory when it is determined that the first memory pair and the second memory pair satisfy the preset merging condition;

[0118] A fourth determining unit 89, configured to determine a second address of the computing memory space output to the second memory in the second memory pair;

[0119] The merging unit 810 is used to merge the memory space corresponding to the first address with the memory space corresponding to the second address to obtain a merged memory space.

[0120] Furthermore, in a possible implementation of this embodiment, as Fig. 9 As shown, the merging unit 810 includes:

[0121] Determine a first space length of the first address and a second space length of the second address;

[0122] The relatively larger space length between the first space length and the second space length is used as the target space length;

[0123] The memory space corresponding to the first address and the memory space corresponding to the second address are merged based on the target space length to obtain the merged memory space.

[0124] Furthermore, in a possible implementation of this embodiment, as Fig. 9 As shown, the device also includes:

[0125] A fifth determining unit 811 is configured to determine an unallocated space address in the first computing memory space after the loading unit 83 loads the target computing corresponding to the target data block into the first computing memory space for computing execution;

[0126] The second allocation unit 812 is further used to continue to allocate the unallocated space address to other operators or a calculation corresponding to other operators, and the other operators are operators other than the target operator.

[0127] It should be noted that the above explanation of the method embodiment is also applicable to the device of this embodiment, and the principle is the same, which is not limited in this embodiment.

[0128] According to an embodiment of the present application, the present application also provides an electronic device, a chip, a readable storage medium and a computer program product.

[0129] The present application also provides a chip, comprising one or more interface circuits and one or more processors; the interface circuit is used to receive a signal from a memory of an electronic device and send the signal to the processor, the signal comprising a computer instruction stored in the memory, and when the processor executes the computer instruction, the electronic device executes the method of allocating memory described in the above embodiment.

[0130] Fig.10 A schematic block diagram of an example electronic device 900 that can be used to implement an embodiment of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.

[0131] like Fig.10As shown, the device 900 includes a computing unit 901, which can perform various appropriate actions and processes according to a computer program stored in a ROM (Read-Only Memory) 902 or a computer program loaded from a storage unit 908 to a RAM (Random Access Memory) 903. In the RAM 903, various programs and data required for the operation of the device 900 can also be stored. The computing unit 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An I / O (Input / Output) interface 905 is also connected to the bus 904.

[0132] A number of components in the device 900 are connected to the I / O interface 905, including: an input unit 906, such as a keyboard, a mouse, etc.; an output unit 907, such as various types of displays, speakers, etc.; a storage unit 908, such as a disk, an optical disk, etc.; and a communication unit 909, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 909 allows the device 900 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0133] The computing unit 901 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a CPU (Central Processing Unit), a GPU (Graphic Processing Units), various dedicated AI (Artificial Intelligence) computing chips, various computing units running machine learning model algorithms, a DSP (Digital Signal Processor), and any appropriate processor, controller, microcontroller, etc. The computing unit 901 performs the various methods and processes described above, such as a method for allocating memory. For example, in some embodiments, the method for allocating memory may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 908. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 900 via the ROM 902 and / or the communication unit 909. When the computer program is loaded into the RAM 903 and executed by the computing unit 901, one or more steps of the method described above may be performed. Alternatively, in other embodiments, the computing unit 901 may be configured to execute the aforementioned method of allocating memory in any other appropriate manner (eg, by means of firmware).

[0134] Various embodiments of the systems and techniques described above herein may be implemented in digital electronic circuit systems, integrated circuit systems, FPGAs (Field Programmable Gate Arrays), ASICs (Application-Specific Integrated Circuits), ASSPs (Application Specific Standard Products), SOCs (System On Chips), CPLDs (Complex Programmable Logic Devices), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that may be executed and / or interpreted on a programmable system including at least one programmable processor that may be a special purpose or general purpose programmable processor that may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0135] The program code for implementing the method of the present application can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, implements the functions / operations specified in the flow chart and / or block diagram. The program code can be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0136] In the context of the present application, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include electrical connections based on one or more lines, portable computer disks, hard disks, RAM, ROM, EPROM (Electrically Programmable Read-Only-Memory) or flash memory, optical fiber, CD-ROM (Compact Dis sc Read-Only Memory), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0137] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (Cathode-Ray Tube) or LCD (Liquid Crystal Display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball), through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0138] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: LAN (Local Area Network), WAN (Wide Area Network), the Internet, and blockchain networks.

[0139] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0140] It should be noted that artificial intelligence is a discipline that studies how computers can simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.), and includes both hardware-level and software-level technologies. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing; artificial intelligence software technologies mainly include computer vision technology, speech recognition technology, natural language processing technology, as well as machine learning / deep learning, big data processing technology, knowledge graph technology, and other major directions.

[0141] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in the present disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in the present application can be achieved, and this document is not limited here. The above specific implementation methods do not constitute limitations on the scope of protection of the present application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application should be included in the scope of protection of the present application.

Claims

1. A method for allocating memory, characterized in that: include: The life cycles of at least two data blocks corresponding to the target operator of the neural network are analyzed in sequence; each data block corresponding to the target operator of the neural network is a data block occupied in the first memory by data required for each calculation after the target operator is split into at least two calculations; In the case of determining that the life cycle of the target data block meets the preset allocation condition, allocating a corresponding first memory space pair for the target data block, the first memory space pair being two adjacent memory spaces in the first memory, including a first computing memory space and a second computing memory space; The target calculation corresponding to the target data block is loaded into the first calculation memory space for execution, and the data required for the next calculation of the target calculation is loaded into the second calculation memory space.

2. The method according to claim 1, characterized in that: The step of loading the target calculation corresponding to the target data block into the first calculation memory space for performing the calculation comprises: The target calculation of the target operator in the second memory is loaded into the memory address corresponding to the first moment in the first calculation memory space to perform calculation, the first calculation memory space has reserved memory space for storing N moments, and the second memory is the input side of the target operator in the first memory.

3. The method according to claim 2, characterized in that After loading data required for a next calculation of the target calculation into the second calculation memory space, the method further includes: Using the output result of the target operator as the input of the next operator; Alternatively, the output result of the target operator is cached in the first memory, and then the output result of the target operator is moved from the first memory to the second memory.

4. The method according to claim 3, characterized in that The caching the output result of the target operator in the first memory includes: The output result of the target operator in the first computing memory space is stored in the first storage memory space in the second memory pair; wherein the second memory pair is two adjacent memory spaces in the first memory, including the first storage memory space and the second storage memory space; The output result of the target operator in the second computing memory space is stored in the second storage memory space in the second memory pair.

5. The method according to claim 4, characterized in that After caching the output result of the target operator in the first memory, the method further includes: Determining whether the first memory pair and the second memory pair meet a preset merging condition; If it is determined that the first memory pair and the second memory pair satisfy the preset merging condition, determining a first address of the computing memory space in the first memory pair loaded by the second memory; Determine a second address of the computing memory space output to the second memory in the second memory pair; The memory space corresponding to the first address and the memory space corresponding to the second address are merged to obtain a merged memory space.

6. The method according to claim 5, characterized in that The merging of the memory space corresponding to the first address and the memory space corresponding to the second address comprises: Determine a first space length of the first address and a second space length of the second address; The relatively larger space length between the first space length and the second space length is used as the target space length; The memory space corresponding to the first address and the memory space corresponding to the second address are merged based on the target space length to obtain the merged memory space.

7. The method according to claim 2, characterized in that After the target calculation corresponding to the target data block is loaded into the first calculation memory space for execution, the method further includes: Determine an unallocated space address in the first computing memory space; The unallocated space address is further allocated to other operators or a calculation corresponding to other operators, and the other operators are operators other than the target operator.

8. A device for allocating memory, characterized in that: include: An analysis unit, used to analyze the life cycles of at least two data blocks corresponding to the target operator of the neural network in sequence; each data block corresponding to the target operator of the neural network is a data block occupied in the first memory by data required for each calculation after the target operator is split into at least two calculations; A first allocation unit is used to allocate a corresponding first memory space pair to the target data block when it is determined that the life cycle of the target data block meets a preset allocation condition, wherein the first memory space pair is two adjacent memory spaces in the first memory, including a first computing memory space and a second computing memory space; A loading unit is used to load the target calculation corresponding to the target data block into the first calculation memory space to perform the calculation, and load the data required for the next calculation of the target calculation into the second calculation memory space.

9. The device according to claim 8, characterized in that The loading unit is also used to load the target calculation of the target operator in the second memory to the memory address corresponding to the first moment in the first calculation memory space to perform calculation, and the first calculation memory space is reserved with memory space for storing N moments, and the second memory is the input side of the target operator in the first memory.

10. The device according to claim 9, characterized in that The device also includes: a first determining unit, configured to use the output result of the target operator as the input of the next operator after the loading unit loads the data required for the next calculation of the target calculation into the second calculation memory space; A cache unit, used to cache the output result of the target operator in the first memory; A transport unit is used to transport the output result of the target operator from the first memory to the second memory.

11. The device according to claim 10, characterized in that The cache unit is also used for: The output result of the target operator in the first computing memory space is stored in the first storage memory space in the second memory pair; wherein the second memory pair is two adjacent memory spaces in the first memory, including the first storage memory space and the second storage memory space; The output result of the target operator in the second computing memory space is stored in the second storage memory space in the second memory pair.

12. The device according to claim 11, characterized in that The device also includes: A second determining unit, configured to determine whether the first memory pair and the second memory pair meet a preset merging condition after the cache unit caches the output result of the target operator in the first memory; The third determining unit is further configured to determine a first address of a computing memory space in the first memory pair loaded by the second memory when it is determined that the first memory pair and the second memory pair satisfy the preset merging condition; a fourth determining unit, configured to determine a second address of the computing memory space output to the second memory in the second memory pair; A merging unit is used to merge the memory space corresponding to the first address with the memory space corresponding to the second address to obtain a merged memory space.

13. The device according to claim 12, characterized in that The merging unit comprises: Determine a first space length of the first address and a second space length of the second address; The relatively larger space length between the first space length and the second space length is used as the target space length; The memory space corresponding to the first address and the memory space corresponding to the second address are merged based on the target space length to obtain the merged memory space.

14. The device according to claim 9, characterized in that The device also includes: A fifth determining unit, configured to determine an unallocated space address in the first computing memory space after the loading unit loads the target computing corresponding to the target data block into the first computing memory space for computing and performs computing; The second allocation unit is further used to continue to allocate the unallocated space address to other operators or a calculation corresponding to other operators, and the other operators are operators other than the target operator.

15. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

16. A chip, characterized in that: It comprises one or more interface circuits and one or more processors; the interface circuit is used to receive a signal from a memory of an electronic device and send the signal to the processor, the signal includes a computer instruction stored in the memory, and when the processor executes the computer instruction, the electronic device executes the method described in any one of claims 1 to 7.

17. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-7.

18. A computer program product, characterized in that The invention comprises a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 7.

Images