Storage space allocation method, terminal device and computer readable storage medium

Abstract

The present application is applicable to the field of data processing technology, and provides a storage space allocation method, a terminal device, and a computer-readable storage medium, the method comprising: if there is a second output tensor of a second layer group with the same load as the first output tensor of a first layer group in a neural network, and the second output tensor of the second layer group has been assigned a first storage number, then the first storage number is assigned to the first output tensor of the first layer group; the first data volume of the first output tensor of the first layer group is calculated; and the size of the storage area corresponding to the first storage number is updated according to the first data volume. Through the above method, the utilization rate of storage space can be improved in the data storage process of the neural network.

Description

Technical Field

[0001] The present application belongs to the field of data processing technology, and in particular, relates to a storage space allocation method, a terminal device, and a computer-readable storage medium. Background Art

[0002] The neural network processor is used to perform computing tasks of the neural network model, and the compiler supporting the neural network processor is used to compile the neural network model to generate machine code that can perform computing tasks on the neural network processor. During the compilation process, the compiler usually merges multiple consecutive layers in the neural network model into a layer group. Data transmission between layer groups is achieved through external storage space, and data transmission between layers within a layer group is achieved through the internal storage space of the neural network processor. Therefore, the compiler needs to allocate storage area for the layer group in the external storage space.

[0003] In the prior art, the compiler allocates different storage areas to different layer groups in the external storage space. When the number of input / output ends of a layer group is uncertain, the connection relationship between the layer groups becomes more complicated, and the data of different output ends of the layer group may be used for different network computing tasks. In this case, if the storage space is allocated in units of layer groups, multiple output tensors may be stored in the same storage area, resulting in inconvenience in the use of the storage space. If a different storage area is allocated for each output tensor of the layer group, the data transmission pressure of the external storage space will increase and the utilization rate of the storage space will be reduced. Summary of the invention

[0004] The embodiments of the present application provide a storage space allocation method, a terminal device, and a computer-readable storage medium, which can improve the utilization rate of storage space during the data storage process of a neural network.

[0005] In a first aspect, an embodiment of the present application provides a storage space allocation method, which is applied to a neural network, wherein the neural network includes a plurality of layer groups, each layer group includes at least one continuous layer, each layer group has at least one input end layer group and at least one output end layer group, and the method includes:

[0006] If there is a second output tensor of a second layer group having the same load as the first output tensor of the first layer group in the neural network, and the second output tensor of the second layer group has been assigned a first storage number, the first storage number is assigned to the first output tensor of the first layer group, wherein the load of the first output tensor is the output tensor of the output end layer group of the first layer group, and the storage number is used to indicate the order of the storage areas in the first storage space;

[0007] Calculating a first data volume of a first output tensor of the first layer group;

[0008] The size of the storage area corresponding to the first storage number is updated according to the first data amount.

[0009] In a neural network, if the loads of two output tensors are the same, it means that the output data of the two layer groups need to be fused together. In an embodiment of the present application, the storage space is divided based on the output tensor of the layer group as a fine-grained granularity, which can effectively avoid the occurrence of data confusion during the storage of neural network data. Furthermore, the same storage number is assigned to the output tensors with the same load, and the same storage number corresponds to a continuous storage area, which is equivalent to assigning the output tensors with the same load to the same continuous storage area. Through the above method, it is equivalent to fusing the output tensors with the same load in the storage space, reducing the number of data transmissions between the storage space and the processor, thereby reducing the data transmission pressure of the storage space. In addition, the size of the corresponding storage area is updated according to the amount of data of the output tensor with the same load, so that the storage area in the storage space is reused as much as possible, thereby improving the utilization of the storage space.

[0010] In a second aspect, an embodiment of the present application provides a storage space allocation device, which is applied to a neural network, wherein the neural network includes a plurality of layer groups, each layer group includes at least one continuous layer, each layer group has at least one input end layer group and at least one output end layer group, and the device includes:

[0011] a first allocation unit, configured to allocate the first storage number to the first output tensor of the first layer group if there is a second output tensor of the second layer group having the same load as the first output tensor of the first layer group in the neural network, and the second output tensor of the second layer group has been allocated a first storage number, wherein the load of the first output tensor is the output tensor of the output end layer group of the first layer group, and the storage number is used to indicate the order of storage areas in the first storage space;

[0012] a data calculation unit, configured to calculate a first data quantity of a first output tensor of the first layer group;

[0013] A region updating unit is used to update the size of the storage region corresponding to the first storage number according to the first data amount.

[0014] In a third aspect, an embodiment of the present application provides a terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements a storage space allocation method as described in any one of the first aspects above when executing the computer program.

[0015] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium. An embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and wherein the computer program, when executed by a processor, implements a storage space allocation method as described in any one of the first aspects above.

[0016] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when executed on a terminal device, enables the terminal device to execute the storage space allocation method described in any one of the above-mentioned first aspects.

[0017] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0019] Figure 1 is a schematic diagram of a neural network processing system provided in an embodiment of the present application;

[0020] Figure 2 It is a schematic diagram of the layer group division provided in the embodiment of the present application;

[0021] Figure 3 It is a flowchart of a storage space allocation method provided in an embodiment of the present application;

[0022] Figure 4 is a schematic diagram of a storage space allocation process provided in an embodiment of the present application;

[0023] Figure 5 is a schematic diagram of a second storage space provided in an embodiment of the present application;

[0024] Figure 6 is a schematic diagram of an address transfer code provided in an embodiment of the present application;

[0025] Figure 7 is a structural block diagram of a storage space allocation device provided in an embodiment of the present application;

[0026] Figure 8 It is a schematic diagram of the structure of the terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0028] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0029] As used in this specification and the appended claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting" depending on the context.

[0030] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0031] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the phrases "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. appearing in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways.

[0032] First, we introduce the technical background. Figure 1 , is a schematic diagram of a neural network processing system provided in an embodiment of the present application. Figure 1 As shown, the neural network processing system includes a neural network processor 11, a compiler 12 supporting the neural network processor, and an external storage space 13. The neural network processor is used to perform computing tasks of the neural network model, and the compiler is used to compile the neural network model to generate machine code that can perform computing tasks on the neural network processor.

[0033] During the compilation process, the compiler usually merges multiple consecutive layers in the neural network model into a layer group. Data transmission between layer groups is realized through external storage space, while data transmission between layers within a layer group is realized through the internal storage space of the neural network processor.

[0034] In an application scenario, the neural network processor includes internal storage space and computing units. The compiler allocates external storage space for each layer group, and stores the storage space allocation result (such as the storage address of the external storage space corresponding to each layer group) in the internal storage space of the neural network processor. When the neural network processor executes the computing task, the computing unit in the neural network processor obtains the storage address of the external storage space corresponding to each layer group from the internal storage space, and obtains the output data of each layer group from the external storage space according to the storage address, and then executes the computing task according to the obtained output data.

[0035] When early compilers performed layer fusion, they usually fused several consecutive layers of the same branch in the neural network model together to obtain a layer group. The number of output layer groups of this layer group is usually 1, and the upper limit of the number of input layer groups is also known (usually up to 2). Figure 2 As shown in (a), the number of output layer groups of each layer group (layer_group) is 1, and the number of input layer groups is at most 2. When allocating external storage space, the compiler usually allocates different storage areas to different layer groups. For single-input / single-output layer groups, the data association between layer groups is single, and the existing storage space allocation method can meet the storage space allocation requirements and ensure the utilization of storage space.

[0036] As compiler performance improves, when the compiler performs layer fusion, it may fuse together several consecutive layers of different branches in the neural network model, and may obtain a multi-input / multi-output layer group. Figure 2 As shown in (b), layer group 1 (layer_group1) has two output ports, and layer group 2 (layer_group2) has two input ports. When the number of input ports / output ports of a layer group is uncertain, the connection relationship between layer groups becomes more complicated, and the data of different output ports of a layer group may be used for different network computing tasks. In this case, if storage space is allocated by layer group, multiple output tensors may be stored in the same storage area, making the use of the storage area inconvenient; if a different storage area is allocated for each output tensor of a layer group, the data transmission pressure of the external storage space will increase, reducing the utilization of the storage space.

[0037] It should be noted that, in the embodiment of the present application, the load of the output tensor a of the layer group A refers to all the output tensors of the layer group at the output end of the layer group A. The drive of the output tensor a of the layer group A refers to all the output tensors of the layer group at the input end of the layer group A. Since the data at the input / output end of the layer group may be one-dimensional data, two-dimensional data, or multi-dimensional data, in the embodiment of the present application, the data at the input end of the layer group is collectively referred to as the input tensor, and the data at the output end of the layer group is collectively referred to as the output tensor.

[0038] In order to solve the above problems, an embodiment of the present application provides a storage space allocation method.

[0039] See also Figure 3 , is a flowchart of a storage space allocation method provided in an embodiment of the present application. As an example but not a limitation, the method may include the following steps:

[0040] S301, if there is a second output tensor of a second layer group having the same load as the first output tensor of the first layer group in the neural network, and the second output tensor of the second layer group has been assigned a first storage number, then the first storage number is assigned to the first output tensor of the first layer group.

[0041] The storage number is used to indicate the order of the storage areas in the first storage space.

[0042] In the embodiment of the present application, in the process of allocating storage space, each output tensor of each layer group in the neural network is traversed in sequence according to the level of the neural network. Therefore, the layer groups at the front level are allocated storage space before the layer groups at the back level, and the multiple output tensors of each layer group can be allocated storage space in sequence according to a pre-set order.

[0043] For example, Figure 2 As shown in (b), the output tensor LG0_out0 of layer group 0 (layer_group0) and the output tensor LG2_out0 of layer group 2 (layer_group2) have the same load layer group 3 (layer_group3) output tensor LG3_out0. The layers of layer group 0 are in the front, and the layers of layer group 2 are in the back. When traversing to layer group 2, layer group 0 has been assigned a storage number, then the storage number assigned to LG0_out0 is assigned to LG2_out0, that is, LG0_out0 and LG2_out0 are assigned the same storage number.

[0044] The storage number is used to indicate the order of storage areas in the storage space. For example, if the address range of the storage space is 0000-1000, the storage number corresponding to the storage area 0000-0100 is set to BUF0, and the storage number corresponding to the storage area 0101-0200 is set to BUF1. That is, the storage area corresponding to BUF0 is located before the storage area corresponding to BUF1.

[0045] It should be noted that the above are only examples of storage areas and storage numbers. In actual applications, the sizes of storage areas allocated to different layer groups may be different, and the form of storage numbers is not specifically limited.

[0046] In a neural network, some layer groups require multiple input terminals for their operation types, such as concatenation (connecting multiple inputs) and element-wise operations (element-wise operations on multiple inputs). The same storage number can be assigned to multiple input tensors of the layer groups that perform these operations. Among them, for layer groups that perform high-dimensional concatenation operations, if their multiple input tensors are assigned to the same continuous storage space, it is equivalent to completing data fusion in the storage space, and there is no need for the neural network processor to obtain data from the external storage space and perform fusion, thereby reducing the hardware computing cost. For layer groups that perform non-high-dimensional concatenation operations, if their multiple input tensors are assigned to the same continuous storage space, the hardware computing cost will not be reduced. In this case, it is not meaningful to execute S301.

[0047] Therefore, optionally, in S301, it is further determined whether the layer group where the first output tensor of the first layer group and the second output tensor of the second layer group have the same load is a preset layer group, and if so, the first storage number of the second output tensor of the second layer group is assigned to the first output tensor of the first layer group. The preset layer group may be a layer group for performing a concatenate operation.

[0048] S302, calculating a first data volume of a first output tensor of the first layer group.

[0049] The input and output data of each layer group in a neural network are usually tensor data, that is, multidimensional data. Usually, the dimension of the output tensor is (n, c, h, w), where n represents the batch number of two-dimensional data, c represents the number of channels, h represents the height of the two-dimensional data, and w represents the width of the two-dimensional data. The amount of data in the output tensor can be calculated based on the dimension of the output tensor and the number of bits occupied by each data in the output tensor. For example, multiply n, c, h, w and bit to calculate the total number of bits occupied by the output tensor, and determine the total number of bits as the amount of data in the output tensor.

[0050] S303: Update the size of the storage area corresponding to the first storage number according to the first data amount.

[0051] Continuing the example in S301, assume that the storage number assigned to LG0_out0 is BUF0, the size of the corresponding storage area is 0.1M, and the data volume of LG2_out0 is 0.2M. After BUF0 is assigned to LG2_out0, the size of the storage area corresponding to BUF0 is updated from 0.1M to 0.3M (0.1M+0.2M).

[0052] It should be noted that in the above steps S301-S305, these processes can be regarded as pre-allocation of external storage space, which only determines the order of storage areas corresponding to each layer group and the size of the storage areas. At this time, the external storage space has not been divided, that is, the space address corresponding to each layer group in the external storage space has not been determined.

[0053] In a neural network, if the loads of two output tensors are the same, it means that the output data of the two layer groups need to be fused together. In an embodiment of the present application, the storage space is divided based on the output tensor of the layer group as a fine-grained granularity, which can effectively avoid the occurrence of data confusion during the storage of neural network data. Furthermore, the same storage number is assigned to the output tensors with the same load, and the same storage number corresponds to a continuous storage area, which is equivalent to assigning the output tensors with the same load to the same continuous storage area. Through the above method, it is equivalent to fusing the output tensors with the same load in the storage space, reducing the number of data transmissions between the storage space and the processor, thereby reducing the data transmission pressure of the storage space. In addition, the size of the corresponding storage area is updated according to the amount of data of the output tensor with the same load, so that the storage area in the storage space is reused as much as possible, thereby improving the utilization of the storage space.

[0054] In one embodiment, the method further comprises:

[0055] S304: If there is a second output tensor of a second layer group having the same load as the first output tensor of the first layer group in the neural network, and the second output tensor of the second layer group is not assigned a storage number, or there is no second output tensor of a second layer group having the same load as the first output tensor of the first layer group in the neural network, a second storage number is assigned to the first output tensor of the first layer group according to the free storage area in the first storage space.

[0056] For example, Figure 2 As shown in (b), if the first output tensor of the first layer group currently traversed is LG0_out0, the second output tensor of the second layer group with the same load as LG0_out0 is LG2_out0. Since the level of layer_group2 is after layer_group0, layer_group2 has not yet been assigned a storage number. This situation belongs to the situation that there is a second output tensor of the second layer group with the same load as the first output tensor of the first layer group in the neural network, and the second output tensor of the second layer group has not been assigned a storage number, then LG0_out0 is assigned a second storage number.

[0057] like Figure 2As shown in (b), if the first output tensor of the first layer group currently traversed is LG3_out0, its load is only LG4_out0. In this case, there is no second output tensor of the second layer group in the neural network with the same load as the first output tensor of the first layer group, and the second storage number is assigned to LG3_out0.

[0058] S305: Set the size of the storage area corresponding to the second storage number according to the first data amount.

[0059] In the case described in S304, the first output tensor of the first layer group does not need to be assigned the same storage number as other output tensors, that is, it does not need to be assigned the same continuous storage area as other output tensors. Therefore, the size of the storage area corresponding to the first output tensor of the first layer group is only determined by the data amount of the first output tensor.

[0060] In one embodiment, after assigning a storage number to the first output tensor of the first layer group, the method further comprises:

[0061] If all loads of the target tensor have been assigned storage numbers, the storage number of the target tensor is recycled and the storage area corresponding to the storage number of the target tensor is released, wherein the target tensor is any output tensor in the driver of the first output tensor.

[0062] For example, Figure 2 As shown in (b), if the first output tensor of the first layer group currently traversed is LG2_out0, its drivers include LG1_out0 and LG1_out1. When the target tensor is LG1_out0, since the load of LG1_out0 is only LG2_out0, after LG2_out0 has been assigned a storage number, the storage area corresponding to LG1_out0 is no longer necessary to retain, and the storage number of LG1_out0 and the storage area corresponding to LG1_out0 are recycled. This is equivalent to releasing the association between LG1_out0 and its corresponding storage number, and the association between LG1_out0 and its corresponding storage area.

[0063] Through the above method, it is possible to avoid storing unnecessary data, saving storage space. The released storage area can also be reused by subsequent layer groups, which can further improve the utilization of storage space.

[0064] In one embodiment, the process of allocating the second storage number to the first output tensor of the first layer group in S304 may include:

[0065] S3041: If there is no allocated storage area in the current first storage space, the starting storage number is determined as the second storage number of the first output tensor of the first layer group.

[0066] like Figure 2 As shown in (b), if the first output tensor of the first layer group currently traversed is lG_out0, that is, the output tensor of the first storage number to be allocated in the neural network. Assuming that the starting storage number is BUF0, at this time, there is no allocated storage area in the storage space, and BUF0 is allocated to lG_out0. Correspondingly, the storage area corresponding to BUF0 is located at the front end of the storage space.

[0067] Correspondingly, in S305, the size of the storage area corresponding to the starting storage number is set to the first data amount.

[0068] S3042, if there is an allocated storage area in the first storage space, determine whether there is a recycled storage number;

[0069] If there is currently a recycled storage number, assigning the second storage number to the first output tensor according to the recycled storage number;

[0070] If there is no recycled storage number currently, the next storage number of the currently allocated storage number is determined as the second storage number.

[0071] It can be seen from the above embodiment that for a target tensor, when each load of the target tensor has been allocated a storage area, the storage area corresponding to the target tensor is released. In this way, the free storage area of ​​the storage space may include the released free storage area and the free storage area that has not been allocated. Optionally, the released free storage area can be preferentially selected.

[0072] Optionally, if there are multiple recycled storage numbers, any recycled storage number can be assigned to the first output tensor of the first layer group.

[0073] Optionally, assign the smallest recycled storage number to the first output tensor of the first layer group.

[0074] Exemplarily, the currently allocated storage numbers are BUF0, BUF1, BUF3, and BUF5, wherein BUF2 and BUF4 are recycled storage numbers. If the storage number of BUF2 is smaller than the storage number of BUF4, BUF2 is allocated to the first output tensor of the first layer group.

[0075] In the case described in S3042, since the size of the storage area corresponding to the recovered storage number is still the data amount of the previously allocated output tensor, it may be smaller than the first data amount of the first output tensor. In this case, correspondingly, in S305, the size of the storage area corresponding to the second storage number is updated to the first data amount. If the size of the storage area corresponding to the recovered storage number is larger than the first data amount, there is no need to update the size of the storage area corresponding to the second storage number.

[0076] Optionally, the allocation can also be made according to the size of the storage area corresponding to the recovered storage number. Specifically:

[0077] If there are multiple recycled storage numbers, determine whether there is a storage area (referred to as a target area) greater than or equal to the first data volume in the storage area corresponding to the recycled storage number;

[0078] If the target area exists, the storage number corresponding to any target area is assigned to the first output tensor, or the smallest recycled number in the target area is assigned to the first output tensor;

[0079] If the target region does not exist, the storage number corresponding to the largest target region is assigned to the first output tensor. In this case, the size of the largest target region is updated to the first data volume accordingly.

[0080] It should be noted that in the above steps S301-S305, these processes can be regarded as pre-allocation of external storage space, which only determines the order of storage areas corresponding to each output tensor of each layer group and the size of the storage area. At this time, the external storage space has not been divided, that is, the spatial address corresponding to each layer group in the external storage space has not been determined. After traversing each output tensor of each layer group of the neural network, that is, pre-allocating storage numbers for each tensor of each layer group and setting the size of the corresponding storage area, the external storage space is divided according to the above pre-allocation results.

[0081] In one embodiment, after traversing each output tensor of each layer group in the neural network, the method further includes:

[0082] I. Calculate, according to the size of the storage area corresponding to each storage number, a first relative address offset of the storage area corresponding to each storage number in the first storage space.

[0083] For example, the sizes of the storage areas corresponding to BUF0, BUF1, BUF2, and BUF3 are A, B, C, and D, respectively. Then the first relative address offsets of BUF0, BUF1, BUF2, and BUF3 in the first storage space are 0, A, A+B, and A+B+C, respectively.

[0084] II. According to the size of the output tensor of the layer group corresponding to each storage number, calculate the second relative address offset of each output tensor of each layer group in the corresponding storage area.

[0085] For example, if the output tensor corresponding to BUF1 has only LGn_outx (output tensor x of layer group n), the second relative address offset of LGn_outx in the storage area corresponding to BUF1 is 0. If the output tensors corresponding to BUF2 have LGn_outx and LGm_outy (output tensor y of layer group m), and the sizes of the output data of LGn_outx and LGm_outy are a and b respectively, the second relative address offset of LGn_outx in the storage area corresponding to BUF2 is 0, and the second relative address offset of LGm_outy in the storage area corresponding to BUF2 is a.

[0086] III. Calculate the storage address of each output tensor of each layer group in the first storage space according to the starting address of the first storage space, the first relative address offset and the second relative address offset.

[0087] Continuing with the examples in steps I and II, for LGn_outx, its storage address in the first storage space is the starting address of the first storage space + (A + B) + 0; for LGm_outy, its storage address in the first storage space is the starting address of the first storage space + (A + B) + a.

[0088] See also Figure 4 , is a schematic diagram of the storage space allocation process provided by the embodiment of the present application. In the embodiment of the present application, the correspondence between the output tensor and the storage number can be stored in the occupancy mapping table, and the size of the storage area corresponding to the output tensor can be stored in the buffer size update mapping table. Figure 4 As shown, the storage space allocation process includes:

[0089] S401, obtaining the connection relationship between the input / output tensors of each layer group in the neural network.

[0090] The connection relationship includes the connection relationship between each input tensor and output tensor of each layer group, that is, determining the load and drive of each output tensor. In the embodiment of the present application, the load of the output tensor a of layer group A refers to all the output tensors of the output end layer group of layer group A. The drive of the output tensor a of layer group A refers to all the output tensors of the input end layer group of layer group A.

[0091] S402, when traversing to the output tensor tensorx (LGn_tensorx) of layer group n, determine whether the layer group where any load of LGn_tensorx is located is a concatenate layer group, and all drivers of the output tensor of the concatenate layer group are not drivers of other concatenate layer groups.

[0092] If the layer group where any load of LGn_tensorx is located is a concatenate layer group, and all drivers of the output tensors of the concatenate layer group are not drivers of other concatenate layer groups, execute S403;

[0093] If the layer group where any load of LGn_tensorx is located is a concatenate layer group, but any driver of the output tensor of the concatenate layer group is the same as the driver of other concatenate layer groups, or the layer group where any load of LGn_tensorx is located is not a concatenate layer group, execute S405.

[0094] S403, determine whether a buffer has been allocated to any driver other than LGn_tensorx in the load of LGn_tensorx.

[0095] This step is equivalent to determining whether there is a second output tensor of the second layer group with the same load as LGn_tensorx, and the second output tensor has been assigned a storage number.

[0096] If a buffer has been allocated to any driver except layer_group n in the load of LGn_tensorx, execute S404; if not, execute S405.

[0097] S404, assign the storage number of the driver that has been allocated a buffer except LGn_tensorx in the load of LGn_tensorx to LGn_tensorx, and calculate the size of the storage area corresponding to the storage number.

[0098] Steps S402 - S404 are equivalent to executing the above steps S301 - S303 .

[0099] S405: Determine whether there is a mapping relationship in the occupancy status mapping table.

[0100] This step is equivalent to determining whether the allocated storage area is stored in the current storage space.

[0101] If it exists, execute S406; if it does not exist, execute S407.

[0102] S406, allocating BUF0 to LGn_tensorx, and setting the size of the storage area corresponding to BUF0 to the size of LGn_tensorx.

[0103] S407, according to the mapping relationship in the occupancy mapping table, find the released buffer or allocate a new buffer to LGn_tensorx.

[0104] Steps S405 - S407 are equivalent to executing steps S3041 - S3042 .

[0105] S408, clearing the connection relationship between LGn_tensorx and its driver. If the load of the driver of LGn_tensorx is empty, then deleting the mapping relationship of the driver of LGn_tensorx in the occupancy mapping table, that is, releasing the storage area occupied by the driver of LGn_tensorx.

[0106] Step S408 is equivalent to executing Figure 3 The embodiment includes a step of reclaiming the storage number of the target output tensor and releasing the storage area corresponding to the storage number of the target output tensor.

[0107] S409, determining whether each output tensor of each layer group in the neural network is traversed.

[0108] If so, execute S410; if not, determine whether each output tensor in the currently traversed layer group is traversed; if there are still untraversed output tensors in the currently traversed layer group, continue to execute S402-S409 for the next output tensor of the current layer group; if each output tensor in the currently traversed layer group has been traversed, continue to execute S402-S409 for each output tensor in the next layer group.

[0109] S410: Calculate the relative address offset of each buffer in the first storage space according to the size of each buffer.

[0110] In the embodiment of the present application, the first storage space may use a double data rate synchronous dynamic random access memory (Double Data Rate, DDR).

[0111] S411, calculating the relative address offset of each layer group in its corresponding buffer according to the size of each buffer and the layer group corresponding to each buffer.

[0112] Finally, according to the starting address of the first storage space and the calculation results of S410 and S411, the storage address of each output tensor of each layer group in the first storage space is calculated.

[0113] Steps S410 - S411 are equivalent to steps I - III in the above embodiment.

[0114] Below Figure 2 Taking the layer group division shown in (b) in the example, the allocation process of the first storage space is explained. During the memory allocation process, the update status of the occupancy mapping table and the buffer size update mapping table is shown in the following table:

[0115]

[0116]

[0117] As shown in the table above, the allocation process of the first storage space is:

[0118] 1) Because there is no buffer in the initial state, a new buffer (storage area in the first storage space), namely BUF0 (Buffer ID number increases from 0) is created for LG0_OUT0 (Layer-Group 0 output tensor 0), and two mapping tables (occupancy mapping table and buffer size update mapping table) are used to record that BUF0 is occupied by LG0_OUT0 and the size of BUF0 is the output size of LG0_OUT0 (LG0_OUT0_size).

[0119] 2) Because BUF0 has not been released, a new buffer, namely BUF1, is created for LG1_OUT0, and the fact that BUF1 is occupied by LG1_OUT0 and the size of BUF1 is the output size of LG1_OUT0 is recorded in two mapping tables respectively.

[0120] 3) Because BUF0 and BUF1 have not been released, a new buffer, namely BUF2, is created for LG1_OUT1, and the fact that BUF1_OUT1 is occupied by LG2 and the size of BUF2 is the output size of LG1_OUT1 is recorded in two mapping tables respectively.

[0121] 4) Assume that layer_group3 is a concatenate layer of the channel dimension. Since the output of LG2_OUT0 and the output of LG0_OUT0 are sent to the same concatenate layer, the results of LG2_OUT0 and LG0_OUT0 are used as the input of LG4_OUT0. LG2_OUT0 and LG0_OUT0 can be assigned to the same buffer, namely BUF0. In two mapping tables, it is recorded that BUF0 is occupied by LG2_OUT0 and the size of BUF0 is the sum of the output sizes of LG2_OUT0 and LG0_OUT0. After the calculation of LG2_OUT0 is completed, the results of LG1_OUT0 and LG1_OUT1 are useless, so the record of BUF1 being occupied by LG1_OUT0 is cleared in the first mapping table, and the record of BUF2 being occupied by LG1_OUT1 is cleared.

[0122] 5) layer_group3 is the Concatenatelayer of the channel dimension. The hardware does not need to do any operation. The Concatenate function is implemented by memory management. Therefore, the two mapping tables remain unchanged when traversing to LG3_OUT0.

[0123] 6) When allocating a buffer to LG4_OUT0, two free buffers (BUF1, BUF2) are selected first to see which buffer space is greater than or equal to the output size of LG4_OUT0. If the BUF1 space is greater than or equal to the output size of LG4_OUT0, BUF1 is allocated to LG4_OUT0, and the first mapping table records that BUF1 is occupied by LG4_OUT0. After the calculation of LG4_OUT0 is completed, the results of LG0_OUT0 and LG2_OUT0 are useless, so the record of BUF0 occupied by LG0_OUT0 and LG2_OUT0 is cleared in the first mapping table.

[0124] 7) After the entire network scan is completed, the final number and size of buffers required, as well as the buffer to which each output tensor is assigned and the address offset of the output tensor in the buffer are obtained.

[0125] The above steps are the process of the compiler performing memory management, allocating a storage area in the first storage space for each output tensor of each layer group of the neural network, and calculating the final allocation result of each output tensor, that is, the address offset corresponding to each output tensor in the first storage space.

[0126] In one embodiment, after calculating the storage address of each output tensor of each layer group in the first storage space, the method further includes an address transfer process, that is, the compiler transfers the result of memory management (calculated storage address) to the neural network processor. The specific steps may include:

[0127] The compiler sends the storage address of each output tensor of each layer group in the first storage space to the neural network processor.

[0128] The neural network processor calculates an address pointer corresponding to each storage address, and stores the storage address into a second storage space inside the neural network processor according to the address pointer.

[0129] For example, see Figure 5 , is a schematic diagram of the second storage space provided in the embodiment of the present application. Figure 5 As shown, the pointer storage location of the DDR address of the layer group input tensor and output tensor is statically allocated in the local memory stack space of the neural network processor, and the two pointers point to the storage location of the DDR address of the input tensor and the output tensor in the local memory of the neural network processor respectively. At the same time, the DDR addresses of the input tensor and the output tensor are allocated in the local memory stack space of the neural network processor at the shared storage start position of the local memory of the neural network processor. Because the total number of input / output tensors of each layer group may be different, the DDR addresses of the input tensor and the output tensor are dynamic at the shared storage end position of the local memory of the neural network processor. In this example, the number of input tensors of a layer group is 3, and the number of output tensors is 2, so the DDR address of the input / output tensor occupies 5 address storage spaces of the local memory stack space of the neural network processor, the pointer DATA_IN_ADDR_PTR of the DDR address of the input tensor points to the TENSOR0_ADDR position, and the pointer DATA_OUT_ADDR_PTR of the DDR address of the output tensor points to the TENSOR3_ADDR position.

[0130] When the neural network processor executes the computing task of the neural network, the neural network processor reads the DDR address pointer of the input / output tensor from the second storage space, and then reads the storage address of the input / output tensor in the DDR according to the address pointer, and then obtains the tensor data stored in the storage address.

[0131] See also Figure 6 , is a schematic diagram of the address transfer code provided in the embodiment of the present application. Figure 6The address transfer pseudocode shown is divided into two parts: the first part of the code is the process of storing the DDR address of the input / output tensor and its pointer to the stack space. The generation time is after the compiler performs memory management. The DDR address of each input / output tensor is calculated based on the result of memory management, and then the calculated DDR address and its pointer are written to the stack space through the st (store) instruction. The second part of the code is the process of reading the pointer from the stack space and then reading the DDR address of the input / output tensor. The acquisition method is to first read the pointer from the stack space through the ld (load) instruction, and then use the pointer to read the input and output DDR address.

[0132] Through the above-mentioned address transfer method, the address storage location can be dynamically allocated in the internal storage space of the neural network processor using the address pointer, so as to flexibly cope with the situation where the number of input / output tensors of the layer group in the neural network is uncertain, thereby greatly improving the robustness and adaptability of the address transfer.

[0133] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0134] Corresponding to the storage space allocation method described in the above embodiment, Figure 7 This is a structural block diagram of the storage space allocation device provided in an embodiment of the present application. For the sake of convenience of explanation, only the parts related to the embodiment of the present application are shown.

[0135] Reference Figure 7 , the device comprises:

[0136] The first allocation unit 51 is used to allocate the first storage number to the first output tensor of the first layer group if there is a second output tensor of the second layer group with the same load as the first output tensor of the first layer group in the neural network, and the second output tensor of the second layer group has been allocated a first storage number, wherein the load of the first output tensor is the output tensor of the output end layer group of the first layer group, and the storage number is used to indicate the order of storage areas in the first storage space.

[0137] A data calculation unit 52 is used to calculate a first data amount of a first output tensor of the first layer group.

[0138] The area updating unit 53 is used to update the size of the storage area corresponding to the first storage number according to the first data amount.

[0139] Optionally, the device 7 further comprises:

[0140] The second allocation unit 54 is used to allocate a second storage number to the first output tensor of the first layer group according to the free storage area in the first storage space if there is a second output tensor of the second layer group with the same load as the first output tensor of the first layer group in the neural network and the second output tensor of the second layer group is not allocated a storage number, or there is no second output tensor of the second layer group with the same load as the first output tensor of the first layer group in the neural network.

[0141] The area calculation unit 55 is used to set the size of the storage area corresponding to the second storage number according to the first data amount.

[0142] Optionally, the device 7 further comprises:

[0143] The space release unit 56 is used to allocate a storage number to the first output tensor of the first layer group. If all loads of the target tensor have been allocated storage numbers, the storage number of the target tensor is recovered and the storage area corresponding to the storage number of the target tensor is released, wherein the target tensor is any output tensor in the driver of the first output tensor.

[0144] Optionally, the second allocation unit 54 is further configured to:

[0145] If no allocated storage area currently exists in the first storage space, the starting storage number is determined as the second storage number of the first output tensor of the first layer group.

[0146] Optionally, the second allocation unit 54 is further configured to:

[0147] If there is an allocated storage area in the current first storage space, determine whether there is a recycled storage number; if there is a recycled storage number, allocate the second storage number to the first output tensor according to the recycled storage number; if there is no recycled storage number, determine the next storage number of the currently allocated storage number as the second storage number.

[0148] Optionally, the device 7 further comprises:

[0149] The space division unit 57 is used to calculate the first relative address offset of the storage area corresponding to each storage number in the first storage space according to the size of the storage area corresponding to each storage number after assigning a storage number to each output tensor of each layer group in the neural network; calculate the second relative address offset of each output tensor of each layer group in the respective corresponding storage area according to the size of the output tensor of the layer group corresponding to each storage number; calculate the storage address of each output tensor of each layer group in the first storage space according to the starting address of the first storage space, the first relative address offset and the second relative address offset.

[0150] Optionally, the device 7 further comprises:

[0151] The address transfer unit 58 is used to send the storage address of each output tensor of each layer group in the first storage space to the neural network processor after calculating the storage address of each output tensor of each layer group in the first storage space according to the starting address of the first storage space, the first relative address offset and the second relative address offset, so as to instruct the neural network processor to calculate the address pointer corresponding to each storage address and store the storage address in the second storage space inside the neural network processor according to the address pointer.

[0152] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0153] in addition, Figure 7 The storage space allocation device shown may be a software unit, a hardware unit, or a combination of software and hardware units built into an existing terminal device, or may be integrated into the terminal device as an independent accessory, or may exist as an independent terminal device.

[0154] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0155] Figure 8 Schematic diagram of the structure of the terminal device provided in the embodiment of the present application. Figure 8 As shown, the terminal device 6 of this embodiment includes: at least one processor 60 ( Figure 8 Only one is shown in the figure) a processor, a memory 61, and a computer program 62 stored in the memory 61 and executable on the at least one processor 60, and when the processor 60 executes the computer program 62, the steps in any of the above-mentioned storage space allocation method embodiments are implemented.

[0156] The terminal device may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The terminal device may include, but is not limited to, a processor and a memory. Those skilled in the art will understand that Figure 8 It is only an example of the terminal device 6 and does not constitute a limitation on the terminal device 6. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, it may also include input and output devices, network access devices, etc.

[0157] The processor 60 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0158] In some embodiments, the memory 61 may be an internal storage unit of the terminal device 6, such as a hard disk or memory of the terminal device 6. In other embodiments, the memory 61 may also be an external storage device of the terminal device 6, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device 6. Further, the memory 61 may also include both an internal storage unit and an external storage device of the terminal device 6. The memory 61 is used to store an operating system, an application program, a boot loader, data, and other programs, such as the program code of the computer program. The memory 61 may also be used to temporarily store data that has been output or is to be output.

[0159] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0160] An embodiment of the present application provides a computer program product. When the computer program product runs on a terminal device, the terminal device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0161] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the device / terminal device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0162] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0163] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0164] In the embodiments provided in the present application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0165] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0166] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A storage space allocation method, characterized in that: Applied to a neural network, the neural network includes a plurality of layer groups, each layer group includes at least one continuous layer, each layer group has at least one input end layer group and at least one output end layer group, and the output data of each layer group is tensor data; the method includes: If there is a second output tensor of a second layer group having the same load as the first output tensor of the first layer group in the neural network, and the second output tensor of the second layer group has been assigned a first storage number, the first storage number is assigned to the first output tensor of the first layer group, wherein the load of the first output tensor is the output tensor of the output end layer group of the first layer group, and the storage number is used to indicate the order of the storage areas in the first storage space; Calculating a first data volume of a first output tensor of the first layer group; The size of the storage area corresponding to the first storage number is updated according to the first data amount.

2. The storage space allocation method according to claim 1, characterized in that: The method further comprises: If there is a second output tensor of a second layer group having the same load as the first output tensor of the first layer group in the neural network, and the second output tensor of the second layer group is not assigned a storage number, or there is no second output tensor of a second layer group having the same load as the first output tensor of the first layer group in the neural network, a second storage number is assigned to the first output tensor of the first layer group according to a free storage area in the first storage space; The size of the storage area corresponding to the second storage number is set according to the first data amount.

3. The storage space allocation method according to claim 2, characterized in that: After assigning a storage number to the first output tensor of the first layer group, the method further includes: If all loads of the target tensor have been assigned storage numbers, the storage number of the target tensor is recycled and the storage area corresponding to the storage number of the target tensor is released, wherein the target tensor is any output tensor in the driver of the first output tensor.

4. The storage space allocation method according to claim 3, characterized in that: The allocating a second storage number to the first output tensor of the first layer group according to the free storage area in the first storage space includes: If there is an allocated storage area in the first storage space, determining whether there is a recycled storage number; If there is currently a recycled storage number, assigning the second storage number to the first output tensor according to the recycled storage number; If there is no recycled storage number currently, the next storage number of the currently allocated storage number is determined as the second storage number.

5. The storage space allocation method according to claim 2, characterized in that: The allocating a second storage number to the first output tensor of the first layer group according to the free storage area in the first storage space includes: If no allocated storage area currently exists in the first storage space, the starting storage number is determined as the second storage number of the first output tensor of the first layer group.

6. The storage space allocation method according to claim 1, characterized in that: The method further comprises: After assigning a storage number to each output tensor of each layer group in the neural network, calculating a first relative address offset of the storage area corresponding to each storage number in the first storage space according to the size of the storage area corresponding to each storage number; According to the size of the output tensor of the layer group corresponding to each storage number, calculate the second relative address offset of each output tensor of each layer group in the corresponding storage area; The storage address of each output tensor of each layer group in the first storage space is calculated according to the starting address of the first storage space, the first relative address offset, and the second relative address offset.

7. The storage space allocation method according to claim 6, characterized in that: After calculating the storage address of each output tensor of each layer group in the first storage space according to the start address of the first storage space, the first relative address offset, and the second relative address offset, the method further includes: The storage address of each output tensor of each layer group in the first storage space is sent to the neural network processor to instruct the neural network processor to calculate the address pointer corresponding to each storage address and store the storage address in the second storage space inside the neural network processor according to the address pointer.

8. A storage space allocation device, characterized in that: Applied to a neural network, the neural network includes a plurality of layer groups, each layer group includes at least one continuous layer, each layer group has at least one input end layer group and at least one output end layer group, and the output data of each layer group is tensor data; the device includes: a first allocation unit, configured to allocate the first storage number to the first output tensor of the first layer group if there is a second output tensor of the second layer group having the same load as the first output tensor of the first layer group in the neural network, and the second output tensor of the second layer group has been allocated a first storage number, wherein the load of the first output tensor is the output tensor of the output end layer group of the first layer group, and the storage number is used to indicate the order of storage areas in the first storage space; a data calculation unit, configured to calculate a first data quantity of a first output tensor of the first layer group; A region updating unit is used to update the size of the storage region corresponding to the first storage number according to the first data amount.

9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images