Static memory allocation method, device, equipment and medium

Abstract

The present application relates to the field of data storage technology, and specifically to a static memory allocation method, device, equipment and medium for improving the utilization rate of memory resources. The method includes: obtaining multiple tensors of a target model, determining the time information and spatial size of each tensor occupied in memory; sorting the multiple tensors according to preset indicators, and assigning first address offsets to the multiple tensors according to preset rules; if the tail address offset of any tensor is greater than a theoretical threshold, adjusting the sorting of the multiple tensors, and reassigning first address offsets to the multiple tensors according to preset rules; the tail address offset of each tensor is the sum of the first address offset and spatial size of each tensor, and the theoretical threshold is the maximum value of the sum of the spatial sizes of the tensors occupied in memory at each moment; until the multiple tensors meet any one of the multiple tuning targets, the memory address of each tensor is determined according to the first address offset and tail address offset of each tensor.

Description

Technical Field

[0001] The present application relates to the field of data storage technology, and in particular to a static memory allocation method, apparatus, device and medium. Background Art

[0002] Deep learning forward reasoning mainly consists of operators and tensors containing the intermediate calculation results of the operators. To reduce inference time, memory is usually pre-allocated for these tensors. If memory is allocated independently for each tensor, the inference engine will use a lot of memory. This memory allocation method is not suitable for devices that are sensitive to memory resources.

[0003] In fact, these tensors do not need to coexist. Multiple tensors with no dependencies can share the same memory. In forward reasoning, the information of these tensors can be known in advance, so static memory allocation can be used to improve the utilization of memory resources. Summary of the Invention

[0004] Embodiments of the present application provide a static memory allocation method, apparatus, device, and medium for improving the utilization rate of memory resources.

[0005] In a first aspect, the present application provides a static memory allocation method, comprising:

[0006] Get multiple tensors of the target model and determine the time information and space size of each tensor in memory;

[0007] Sorting the multiple tensors according to a preset index, and assigning a first address offset to the multiple tensors according to a preset rule; wherein the first address offset represents the offset between the first address of each tensor and the first address of the memory pool, the preset rule includes that the memory addresses of tensors with overlapping time information do not overlap, and the time information of tensors with overlapping memory addresses does not overlap, and the memory address of each tensor is determined according to the first address offset and the space size of each tensor;

[0008] If the tail address offset of any tensor is greater than the theoretical threshold, the order of the multiple tensors is adjusted, and the first address offsets are reassigned to the multiple tensors according to the preset rule; wherein the tail address offset of each tensor is the sum of the first address offset and the space size of each tensor, and the theoretical threshold is the maximum value of the sum of the space sizes occupied by the tensors at each moment;

[0009] Until the plurality of tensors meet any one of the plurality of tuning targets, a memory address of each tensor is determined according to a first address offset and a tail address offset of each tensor.

[0010] In an embodiment of the present application, when the tail address offset of any tensor is greater than the theoretical threshold, the sorting of multiple tensors is adjusted, and the first address offsets of multiple tensors are reassigned according to preset rules until multiple tensors meet the tuning target. Then, the memory address of each tensor is determined based on the first address offset and tail address offset of each tensor. By adjusting the sorting and setting the tuning target, the optimal sorting method is tried to be found, and then the optimal memory allocation result of multiple tensors is obtained, thereby improving the utilization rate of memory resources.

[0011] In a possible embodiment, sorting the multiple tensors according to preset indices and allocating first address offsets to the multiple tensors according to preset rules includes:

[0012] Generate multiple memory rectangles corresponding to multiple memory operations; wherein the multiple memory operations represent operations of the multiple tensors occupying memory, the length of each memory rectangle represents the time length of each memory operation, and the width of each memory rectangle represents the space size of each memory operation;

[0013] The multiple memory rectangles are sorted according to preset indicators and arranged in a memory container corresponding to the memory pool according to preset rules; wherein the first address of the memory pool corresponds to the bottom of the memory container, and the minimum distance between each memory rectangle and the bottom is the first address offset of each tensor.

[0014] In a possible embodiment, if the tail address offset of any tensor is greater than a theoretical threshold, adjusting the order of the multiple tensors and reassigning the first address offsets to the multiple tensors according to the preset rule includes:

[0015] If the width of the target rectangle exceeds a theoretical threshold, the order of the multiple memory rectangles is adjusted and rearranged in the memory container according to the preset rule; wherein the target rectangle is the smallest rectangle containing the multiple arranged memory rectangles, and the width of the target rectangle represents the maximum value among the tail address offsets of the multiple tensors.

[0016] In an embodiment of the present application, the static memory allocation problem is converted into a two-dimensional rectangular packing problem, the memory operation is geometrically abstracted, a memory rectangle is constructed according to the time information and spatial size of the memory occupied by each tensor, the memory is converted into a rectangle, the memory pool is converted into a memory container, and multiple rectangles are arranged in the memory container. By adjusting the positions of the multiple rectangles arranged in the memory container, the tuning goal is achieved.

[0017] In a possible embodiment, after the plurality of tensors satisfy any one of a plurality of tuning objectives, the method further includes:

[0018] According to the position of each memory rectangle in the memory container, the first address offset and the last address offset of each tensor are determined.

[0019] In a possible embodiment, adjusting the order of the multiple memory rectangles includes:

[0020] Classify the multiple memory rectangles to obtain high-risk memory rectangles, medium-risk memory rectangles, low-risk memory rectangles, and risk-free memory rectangles; wherein, the high-risk memory rectangle includes a first memory rectangle corresponding to a first tensor whose tail address offset is greater than the theoretical threshold, and a second memory rectangle that temporally overlaps with the first memory rectangle; the medium-risk memory rectangle includes a third memory rectangle corresponding to a second tensor whose tail address offset is equal to the theoretical threshold, and a fourth memory rectangle that temporally overlaps with the third memory rectangle; the low-risk memory rectangle includes a memory rectangle that temporally overlaps with the high-risk memory rectangle and the medium-risk memory rectangle; and the risk-free memory rectangle includes memory rectangles among the multiple memory rectangles excluding the high-risk memory rectangle, the medium-risk memory rectangle, and the low-risk memory rectangle;

[0021] Adjusting the sorting of the high-risk memory rectangles and the sorting of the medium-risk memory rectangles respectively;

[0022] The adjusted high-risk memory rectangle, the adjusted medium-risk memory rectangle, the low-risk memory rectangle, and the risk-free memory rectangle are merged in sequence.

[0023] In an embodiment of the present application, multiple memory rectangles are divided into four categories according to the risk level that affects the tuning target. Only the sorting of medium and high-risk memory rectangles is adjusted, and low-risk memory rectangles and risk-free memory rectangles are simply merged in descending order of risk level. This ensures the tuning effect while reducing the algorithm complexity.

[0024] In a possible embodiment, adjusting the order of the high-risk memory rectangles and the order of the medium-risk memory rectangles respectively includes:

[0025] dividing the high-risk memory rectangles into a plurality of groups, and dividing the low-risk memory rectangles into a plurality of groups according to a time overlap length;

[0026] Sort the memory rectangles of each group according to the preset index and arrange them in the memory container according to the preset rule;

[0027] If the width of the target rectangle exceeds the theoretical threshold, adjusting the order of the memory rectangles of each group and rearranging them in the memory container according to the preset rule;

[0028] Until the memory rectangle of each group meets any one of the multiple tuning targets, the memory rectangles of each group after adjustment and sorting are derived in sequence to obtain the adjusted high-risk memory rectangle and the adjusted medium-risk memory rectangle.

[0029] In this embodiment, high-risk and medium-risk memory rectangles are further divided into multiple groups, and the memory rectangles in each group are optimized sequentially, further reducing algorithm complexity. Furthermore, memory rectangles with high overlap are clustered according to their length, and memory rectangles with high overlap are grouped together, which improves the effectiveness of layout optimization.

[0030] In a possible embodiment, the preset indicator includes at least one indicator of the length of the memory rectangle, the width of the memory rectangle, and the time overlap length between the memory rectangles.

[0031] In an embodiment of the present application, preprocessing is performed on multiple memory rectangles, and the multiple memory rectangles are sorted according to one or more indicators, which can speed up the tuning speed and improve the tuning results.

[0032] In a possible embodiment, the multiple tuning targets include that no tensor has a tail address offset greater than the theoretical threshold, the same sorting result occurs, and the number of adjustments is greater than or equal to a preset number.

[0033] In a second aspect, the present application provides a static memory allocation device, comprising:

[0034] The acquisition module is used to obtain multiple tensors of the target model and determine the time information and space size of each tensor in memory;

[0035] an allocation module, configured to sort the multiple tensors according to a preset index and allocate a first address offset to the multiple tensors according to a preset rule; wherein the first address offset represents the offset between the first address of each tensor and the first address of the memory pool, the preset rule includes that the memory addresses of tensors with overlapping time information do not overlap, and the time information of tensors with overlapping memory addresses does not overlap, and the memory address of each tensor is determined based on the first address offset and the spatial size of each tensor;

[0036] The allocation module is further configured to adjust the order of the multiple tensors and reassign the first address offsets to the multiple tensors according to the preset rule if the tail address offset of any tensor is greater than a theoretical threshold; wherein the tail address offset of each tensor is the sum of the first address offset and the space size of each tensor, and the theoretical threshold is the maximum value of the sum of the space sizes occupied by the tensors at each moment;

[0037] A determination module is configured to determine a memory address of each tensor according to a first address offset and a tail address offset of each tensor until the plurality of tensors meet any one of a plurality of tuning targets.

[0038] In a possible embodiment, the allocation module is specifically configured to:

[0039] Generate multiple memory rectangles corresponding to multiple memory operations; wherein the multiple memory operations represent operations of the multiple tensors occupying memory, the length of each memory rectangle represents the time length of each memory operation, and the width of each memory rectangle represents the space size of each memory operation;

[0040] The multiple memory rectangles are sorted according to preset indicators and arranged in a memory container corresponding to the memory pool according to preset rules; wherein the first address of the memory pool corresponds to the bottom of the memory container, and the minimum distance between each memory rectangle and the bottom is the first address offset of each tensor.

[0041] In a possible embodiment, the allocation module is specifically configured to:

[0042] If the width of the target rectangle exceeds a theoretical threshold, the order of the multiple memory rectangles is adjusted and rearranged in the memory container according to the preset rule; wherein the target rectangle is the smallest rectangle containing the multiple arranged memory rectangles, and the width of the target rectangle represents the maximum value among the tail address offsets of the multiple tensors.

[0043] In a possible embodiment, the determining module is further configured to:

[0044] After the plurality of tensors meet any one of the plurality of tuning targets, a first address offset and a tail address offset of each tensor are determined according to a position of each memory rectangle in the memory container.

[0045] In a possible embodiment, the allocation module is specifically configured to:

[0046] Classify the multiple memory rectangles to obtain high-risk memory rectangles, medium-risk memory rectangles, low-risk memory rectangles, and risk-free memory rectangles; wherein, the high-risk memory rectangle includes a first memory rectangle corresponding to a first tensor whose tail address offset is greater than the theoretical threshold, and a second memory rectangle that temporally overlaps with the first memory rectangle; the medium-risk memory rectangle includes a third memory rectangle corresponding to a second tensor whose tail address offset is equal to the theoretical threshold, and a fourth memory rectangle that temporally overlaps with the third memory rectangle; the low-risk memory rectangle includes a memory rectangle that temporally overlaps with the high-risk memory rectangle and the medium-risk memory rectangle; and the risk-free memory rectangle includes memory rectangles among the multiple memory rectangles excluding the high-risk memory rectangle, the medium-risk memory rectangle, and the low-risk memory rectangle;

[0047] Adjusting the sorting of the high-risk memory rectangles and the sorting of the medium-risk memory rectangles respectively;

[0048] The adjusted high-risk memory rectangle, the adjusted medium-risk memory rectangle, the low-risk memory rectangle, and the risk-free memory rectangle are merged in sequence.

[0049] In a possible embodiment, the allocation module is specifically configured to:

[0050] dividing the high-risk memory rectangles into a plurality of groups, and dividing the low-risk memory rectangles into a plurality of groups according to a time overlap length;

[0051] Sort the memory rectangles of each group according to the preset index and arrange them in the memory container according to the preset rule;

[0052] If the width of the target rectangle exceeds the theoretical threshold, adjusting the order of the memory rectangles of each group and rearranging them in the memory container according to the preset rule;

[0053] Until the memory rectangle of each group meets any one of the multiple tuning targets, the memory rectangles of each group after adjustment and sorting are derived in sequence to obtain the adjusted high-risk memory rectangle and the adjusted medium-risk memory rectangle.

[0054] In a possible embodiment, the preset indicator includes at least one indicator of the length of the memory rectangle, the width of the memory rectangle, and the time overlap length between the memory rectangles.

[0055] In a possible embodiment, the multiple tuning targets include that no tensor has a tail address offset greater than the theoretical threshold, the same sorting result occurs, and the number of adjustments is greater than or equal to a preset number.

[0056] In a third aspect, the present application provides an electronic device, comprising:

[0057] a memory for storing program instructions;

[0058] The processor is configured to call the program instructions stored in the memory and execute the method according to any one of the first aspects according to the obtained program instructions.

[0059] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a computer, the computer executes any one of the methods according to the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0061] Figure 1 A schematic diagram of an application scenario of a static memory allocation method provided in an embodiment of the present application;

[0062] Figure 2 A schematic diagram of a static memory allocation method provided in an embodiment of the present application Figure 1 ;

[0063] Figure 3 A schematic diagram of a target rectangle provided in an embodiment of the present application;

[0064] Figure 4 A schematic diagram of the structure of a memory allocation device provided in an embodiment of the present application;

[0065] Figure 5 A schematic diagram of a static memory allocation method provided in an embodiment of the present application Figure 2 ;

[0066] Figure 6 A schematic diagram of the structure of a layout space optimizer provided in an embodiment of the present application;

[0067] Figure 7 A flowchart of a medium- and high-risk memory tuner provided in an embodiment of the present application;

[0068] Figure 8 A structural diagram of a static memory allocation device provided in an embodiment of the present application;

[0069] Figure 9 A structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0070] In order to make the purpose, technical solutions and advantages of the present invention clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application. Unless there is a conflict, the embodiments in this application and the features in the embodiments can be combined with each other in any way. In addition, although a logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in an order different from that here.

[0071] The terms "first" and "second" in the specification and claims of this application and the accompanying drawings are used to distinguish different objects, not to describe a specific order. In addition, the terms "comprising" and "including" and any variations thereof are intended to cover non-exclusive protection. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units that are inherent to the process, method, product, or device.

[0072] In the embodiment of the present application, "multiple" may mean at least two, for example, two, three or more, and the embodiment of the present application does not limit this.

[0073] Before introducing the static memory allocation method provided in the embodiment of the present application, in order to facilitate understanding, the background technology of the embodiment of the present application is first introduced in detail.

[0074] The traditional tensor memory reuse method mainly sorts multiple tensors by memory size and allocates memory to each tensor in turn according to this sorting. The allocated memory is added to the pre-allocated list. For a certain tensor to be allocated, it is checked whether there is memory of appropriate size and no overlapping life cycle in the pre-allocated list. If so, the memory is allocated to the tensor and marked for use. If not, new memory is allocated for the tensor to be allocated and the new memory is added to the pre-allocated list. However, if the memory sizes required by multiple tensors vary greatly, tensors with small memory will reuse large memory, resulting in a large amount of free memory and low memory resource utilization.

[0075] In view of this, an embodiment of the present application provides a static memory allocation method, which can be executed by a memory allocation device. The memory allocation device can be implemented by a terminal or a server, and the terminal can be, for example, a mobile terminal, a fixed terminal, or a portable terminal, such as a mobile phone, a multimedia computer, a multimedia tablet, a desktop computer, a notebook computer, a tablet computer, etc. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms, but is not limited to these.

[0076] The following briefly introduces the application scenarios to which the technical solutions of the embodiments of the present application can be applied. It should be noted that the application scenarios described below are only used to illustrate the embodiments of the present application and are not limiting. In the specific implementation process, the technical solutions provided by the embodiments of the present application can be flexibly applied according to actual needs.

[0077] Please refer to Figure 1 , is a schematic diagram of an application scenario of the static memory allocation method provided in an embodiment of the present application, wherein the application scenario includes multiple tensors 101 to be allocated and a memory allocation device 102.

[0078] Specifically, after the memory allocation device 102 obtains multiple tensors 101 to be allocated, it allocates memory for each tensor. The meaning of tensor is discussed above. The specific process of how the memory allocation device 102 allocates memory for the multiple tensors 101 to be allocated will be described in detail below.

[0079] As described above, the application scenarios are combined with Figure 1 The application scenario shown is Figure 1 The memory allocation device 102 in the embodiment of the present invention performs a static memory allocation method. Figure 2 , which is a flow chart of a static memory allocation method provided in an embodiment of the present application.

[0080] S201. Obtain multiple tensors of the target model and determine the time information and space size of each tensor occupied in memory.

[0081] The target model primarily consists of operators and multiple tensors containing the intermediate computational results of the operators. Examples of target models include neural network models and deep learning forward inference models. After acquiring the target model, the memory allocation device sequentially retrieves multiple tensors of the target model in the order in which the operators are executed. By analyzing the dependencies between these multiple tensors, it can determine the time and space required for each tensor to occupy memory. The time information includes the start and end times of each tensor's memory occupation, and the space occupied by each tensor is also known as the memory size.

[0082] S202 , sorting the multiple tensors according to preset indices, and assigning first address offsets to the multiple tensors according to preset rules.

[0083] In one possible embodiment, the preset indicator includes at least one of the length of time a tensor occupies memory, the size of the memory occupied by a tensor, and the time overlap length between tensors. The length of time each tensor occupies memory is the difference between the start time and the end time of each tensor occupying memory, and the time overlap length between tensors refers to the length of time any two tensors occupy memory simultaneously.

[0084] For example, the time information that tensor A occupies memory is 8:00-8:10, and the time information that tensor B occupies memory is 8:05-8:15. Then the time length that tensor A and tensor B occupy memory is 10 minutes. Tensor A and tensor B both occupy memory during the period of 8:05-8:10, so the time overlap length between tensor A and tensor B is 5 minutes.

[0085] After allocating memory to a device or multiple tensors, the multiple tensors can be sorted according to preset indicators, either from large to small or from small to large, and the first address offsets are assigned to the sorted multiple tensors in sequence according to preset rules. The first address offset represents the offset between the first address of each tensor and the first address of the memory pool. The preset rules include that the memory addresses of tensors with overlapping time information do not overlap, and the time information of tensors with overlapping memory addresses does not overlap. The memory address of each tensor is determined based on the first address offset and space size of each tensor.

[0086] For example, two tensors are ordered as follows: Tensor A, Tensor B. Tensor A and Tensor B both need to occupy memory between 8:05 and 8:10. First, a starting address offset is allocated to Tensor A, and then a starting address offset is allocated to Tensor B. The starting address offset of Tensor B must be greater than or equal to the ending address offset of Tensor A to ensure that the memory addresses of Tensor A and Tensor B do not overlap.

[0087] In one possible embodiment, to maximize memory conservation, the preset rules may further include ensuring that the tail address offset of the first tensor is equal to the first address offset of the second tensor, where the first tensor is sorted before the second tensor, and the second tensor is the tensor whose time information overlaps with the first tensor and is closest to the first tensor. The tail address offset of each tensor represents the offset between the tail address of each tensor and the first address of the memory pool, and the tail address offset of each tensor is the sum of the first address offset and the spatial size of each tensor.

[0088] For example, the order of four tensors is: Tensor A, Tensor B, Tensor C, Tensor D. The time information of Tensor A, Tensor C, and Tensor D overlaps. If Tensor A is the first tensor, then Tensor C is the second tensor. If Tensor C is the first tensor, then Tensor D is the second tensor. The tail address offset of Tensor A is equal to the first address offset of Tensor C, and the tail address offset of Tensor C is equal to the first address offset of Tensor D.

[0089] S203: If the tail address offset of any tensor is greater than the theoretical threshold, the sorting of the multiple tensors is adjusted, and the first address offsets of the multiple tensors are reallocated according to a preset rule.

[0090] Considering that the initial memory allocation of multiple tensors is often suboptimal, the memory allocation device can determine whether to reallocate the starting address offset based on whether any of the tensors has a tail address offset greater than a theoretical threshold. The meaning and calculation of the tail address offset are discussed previously and will not be further explained here. The theoretical threshold is the maximum sum of the memory space occupied by the tensors at each moment.

[0091] Specifically, after allocating the first address offsets to multiple tensors, the memory allocation device determines the tail address offset of each tensor based on the sum of the first address offset and the space size of each tensor. The memory allocation device can calculate the sum of the space sizes occupied by the tensors corresponding to each moment, obtain multiple sums, and take the maximum value as the theoretical threshold. Furthermore, the memory allocation device can detect whether the tail address offset of each tensor is greater than the theoretical threshold. If the tail address offset of any tensor is greater than the theoretical threshold, the sorting of the multiple tensors is adjusted, and the first address offsets are re-assigned to the multiple tensors after the adjusted sorting according to preset rules.

[0092] S204 , until the multiple tensors meet any one of the multiple tuning targets, determine the memory address of each tensor according to the first address offset and the last address offset of each tensor.

[0093] After the memory allocation device adjusts the sorting of the multiple tensors and reallocates the first address offsets for the multiple tensors according to a preset rule, it can determine whether to continue adjusting the sorting of the multiple tensors based on whether the multiple tensors meet the tuning target.

[0094] Specifically, the multiple tuning targets include the tail address offset of any tensor being less than or equal to the theoretical threshold, the same sorting result appearing, and the number of adjustments being greater than or equal to a preset number. The same sorting result is also called a fixed point tuning result. If multiple tensors do not meet any of the multiple tuning targets, continue to adjust the sorting of the multiple tensors, and reallocate the first address offsets for the multiple tensors according to the preset rules. Until the multiple tensors meet any of the multiple tuning targets, the memory allocation device can determine the first address of the memory pool of each tensor based on the first address of the memory pool and the first address offset of each tensor, determine the tail address of each tensor based on the first address of the memory pool and the tail address offset of each tensor, and determine the memory address of each tensor based on the first address of each tensor and the tail address of each tensor.

[0095] For example, for 3 tensors: tensor X, tensor Y, and tensor W, the preset number of times is 4 times. The first sorting result is: X, Y, W. After assigning the first address offset to X, Y, and W in turn, if the tail address offset of a tensor is greater than the theoretical threshold, the sorting of multiple tensors is adjusted. The second sorting result is: X, W, Y. After assigning the first address offset to X, W, and Y in turn, there is still a tensor with a tail address offset greater than the theoretical threshold, and the second sorting result is different from the first, and the preset number of times is not reached, then the sorting of multiple tensors is continued to be adjusted. The third sorting result is: X, Y, W. There is still a tensor with a tail address offset greater than the theoretical threshold and the preset number of times is not reached, but the third sorting result is the same as the first, and the adjustment can be stopped.

[0096] It should be noted that after the memory allocation device executes S202, if no tensor's tail address offset exceeds the theoretical threshold, indicating that the initial allocation result of the multiple tensors is already the optimal allocation result, S203 and S204 are not executed, and the memory address of each tensor can be directly determined based on the first and last address offsets of each tensor. The specific details of how to determine the memory address can be found in the previous section and will not be repeated here.

[0097] To more vividly illustrate the memory allocation method provided by the embodiments of the present application, in one possible embodiment, the memory allocation device can perform a geometric abstraction representation of the memory occupied by each tensor, with the memory equivalent to a rectangle and the memory pool equivalent to a container. The process of allocating memory for a tensor is equivalent to placing the rectangle into the container, and the memory address of each tensor is determined based on the final position of the rectangle in the container. The specific steps are as follows.

[0098] S1.1. Generate multiple memory rectangles corresponding to multiple memory operations, sort the multiple memory rectangles according to preset indicators, and arrange them in a memory container corresponding to the memory pool according to preset rules.

[0099] Among them, multiple memory operations represent operations in which multiple tensors occupy memory. The length of each memory rectangle represents the duration of each memory operation, the width of each memory rectangle represents the space size of each memory operation, and the length of the memory container represents the total duration of multiple memory operations.

[0100] Specifically, the memory allocation device analyzes the start and end times of each memory operation based on the time information of each tensor occupying the memory, and uses the time length between the start and end times as the length of each memory rectangle, and the size of the memory space occupied by each tensor as the width of each memory rectangle, thereby generating multiple memory rectangles corresponding to multiple memory operations.

[0101] In one possible embodiment, a serialization model can provide a memory operation sequence. A serialization model is a model stored in an array format, where the order in which nodes are arranged corresponds to the order in which they are executed. A memory operation sequence includes multiple memory operations executed sequentially. Each memory operation can have multiple parameters, such as a memory name, memory operation type, memory size, and address alignment. There are two types of memory operations: memory allocation and memory release. The address alignment is the first address offset.

[0102] After obtaining the multiple memory rectangles, the memory allocation device may sort the multiple memory rectangles according to a preset index, and arrange the sorted multiple memory rectangles in sequence in a memory container corresponding to the memory pool according to a preset rule.

[0103] The first address of the memory pool corresponds to the bottom of the memory container, and the minimum distance between each memory rectangle and the bottom is the first address offset of each tensor. Preset rules include that no two memory rectangles in the multiple memory rectangles can overlap. Preset rules can also include bottom alignment rules. For example, using a rectangular coordinate system for the memory container, with the horizontal axis representing time and the vertical axis representing memory offset, bottom alignment means placing the memory rectangles close together in the direction of the smaller memory offset.

[0104] The meaning of the preset indicator is involved. In one possible embodiment, the preset indicator includes at least one of the width of the memory rectangle, the height of the memory rectangle, and the time overlap length between the memory rectangles. The following describes the situation in detail.

[0105] In the first case, the preset indicator includes any one of the width of the memory rectangle, the height of the memory rectangle, and the time overlap length between the memory rectangles.

[0106] Case 1: The preset indicator is the width or height of the memory rectangle.

[0107] The memory allocation device sorts multiple memory rectangles by width or height. When at least two memory rectangles have the same width or height, the order of these at least two memory rectangles can be arbitrarily arranged. For example, if memory rectangles A and B have the same width, the order of arrangement from largest to smallest can be A, B, or B, A.

[0108] Case 2: The preset indicator is the time overlap length between memory rectangles.

[0109] The memory allocation device sorts each memory rectangle based on the sum of the temporal overlap lengths between each memory rectangle and the other memory rectangles. If two memory rectangles do not temporally overlap, the temporal overlap length between the two memory rectangles is 0. If at least two memory rectangles have the same sum of temporal overlap lengths, the order of these at least two memory rectangles can be arbitrarily arranged.

[0110] For example, for three memory rectangles: rectangle A, rectangle B, and rectangle C, the temporal overlap length between rectangle A and rectangle B is X1, the temporal overlap length between rectangle A and rectangle C is X2, and the temporal overlap length between rectangle B and rectangle C is X3. Then, the sum of the temporal overlap lengths between rectangle A and the other rectangles is X1 + X2, the sum of the temporal overlap lengths between rectangle B and the other rectangles is X1 + X3, and the sum of the temporal overlap lengths between rectangle C and the other rectangles is X2 + X3. If X1 + X2 > X1 + X3 > X2 + X3, then the order from largest to smallest is A, B, C. If X1 + X2 > X1 + X3 = X2 + X3, then the order from largest to smallest can be A, B, C, or A, C, B.

[0111] In the second case, the preset indicators include any two indicators among the width of the memory rectangle, the height of the memory rectangle, and the time overlap length between the memory rectangles.

[0112] Specifically, any two indicators are respectively a first indicator and a second indicator. The memory allocation device can sort according to the sum, product, or ratio of the first indicator and the second indicator.

[0113] For example, if the ratio of the width to the height of memory rectangle A is greater than the ratio of the width to the height of memory rectangle B, the order from largest to smallest is: A, B.

[0114] Alternatively, the memory allocation device may sort the plurality of memory rectangles according to a first index, and when there are at least two memory rectangles with the same first index, sort the at least two memory rectangles according to a second index, wherein the priority of the first index is higher than the priority of the second index.

[0115] For example, the priority of height is higher than the priority of width. For three memory rectangles: rectangle A, rectangle B, and rectangle C, the height of rectangle A is greater than the height of rectangle B, the height of rectangle B is equal to the height of rectangle C, and the width of rectangle C is greater than the width of rectangle B. Then the order from large to small is: A, C, B.

[0116] In the third case, the preset indicators include three indicators of the width of the memory rectangle, the height of the memory rectangle, and the time overlap length between the memory rectangles.

[0117] Specifically, the three indicators are the first indicator, the second indicator and the third indicator, and the memory allocation device can sort according to the sum or product of the first indicator, the second indicator and the third indicator.

[0118] Alternatively, the memory allocation device may sort the multiple memory rectangles according to a first index. If multiple first rectangles have the same first index among the multiple memory rectangles, the memory allocation device may sort the multiple first rectangles according to a second index. If multiple second rectangles have the same second index among the multiple first rectangles, the memory allocation device may sort the multiple second rectangles according to a third index. The priority of the first index is higher than that of the second index, and the priority of the second index is higher than that of the third index.

[0119] S1.2. If the width of the target rectangle exceeds the theoretical threshold, adjust the order of the multiple memory rectangles and rearrange them in the memory container according to a preset rule.

[0120] Considering that the initial placement of rectangles is often suboptimal, for example, the tops of some inner rectangles exceed the optimal height corresponding to a theoretical threshold, the memory allocation device can determine whether to adjust the position of the memory rectangle within the memory container based on whether the width of the target rectangle exceeds the theoretical threshold. The target rectangle is the smallest rectangle that contains the arranged memory rectangles. The width of the target rectangle represents the maximum value among the tail address offsets of multiple tensors.

[0121] Please refer to Figure 3 , is a schematic diagram of a target rectangle provided in an embodiment of the present application. There are a total of 9 rectangles corresponding to solid-line boxes, and the rectangle corresponding to the dotted-line box is the target rectangle.

[0122] Specifically, after placing multiple memory rectangles, the memory allocation device can determine the target rectangle and compare the width of the target rectangle with the theoretical threshold. If the width of the target rectangle exceeds the theoretical threshold, the memory allocation device adjusts the order of the multiple memory rectangles and rearranges the multiple memory rectangles after the adjustment in the memory container according to preset rules.

[0123] In order to speed up the tuning process, the present application provides a peak suppression tuning method that focuses on tuning medium and high-risk memory, with the aim of reducing the height of the peak memory rectangle in the memory container.

[0124] Specifically, the memory allocation device can classify multiple memory rectangles to obtain high-risk memory rectangles, medium-risk memory rectangles, low-risk memory rectangles and risk-free memory rectangles, adjust the sorting of high-risk memory rectangles and the sorting of medium-risk memory rectangles respectively, and merge the adjusted high-risk memory rectangles, the adjusted medium-risk memory rectangles, the low-risk memory rectangles and the risk-free memory rectangles in sequence.

[0125] Among them, the high-risk memory rectangle includes the first memory rectangle corresponding to the first tensor whose tail address offset is greater than the theoretical threshold, and the second memory rectangle that overlaps with the first memory rectangle in time. The medium-risk memory rectangle includes the third memory rectangle corresponding to the second tensor whose tail address offset is equal to the theoretical threshold, and the fourth memory rectangle that overlaps with the third memory rectangle in time. The low-risk memory rectangle includes the memory rectangle that overlaps with the high-risk memory rectangle and the medium-risk memory rectangle in time. The risk-free memory rectangle includes the memory rectangles among the multiple memory rectangles except the high-risk memory rectangle, the medium-risk memory rectangle and the low-risk memory rectangle.

[0126] This involves how to adjust the sorting of high-risk memory rectangles and medium-risk memory rectangles, which is introduced below.

[0127] The memory allocation device can divide high-risk memory rectangles into multiple groups and low-risk memory rectangles into multiple groups according to the length of time overlap, sort the memory rectangles of each group according to preset indicators, and arrange them in a memory container according to preset rules. If the width of the target rectangle exceeds a theoretical threshold, the sorting of the memory rectangles of each group is adjusted and rearranged in the memory container according to preset rules; until the memory rectangles of each group meet any one of multiple tuning targets, the memory rectangles of each group after adjustment and sorting are derived in turn to obtain adjusted high-risk memory rectangles and adjusted medium-risk memory rectangles.

[0128] It should be noted that the arrangement results of each group of memory rectangles can be accumulated in the memory container. For example, if the memory container is initially empty, the first group of memory rectangles is arranged in the empty memory container and tuning begins until the tuning target is met. The arrangement results of the first group of memory rectangles in the memory container are saved. Then, the second group of memory rectangles is arranged in the memory container containing the first group of memory rectangles and tuning begins again until the tuning target is met. The arrangement results of the first and second groups of memory rectangles in the memory container are saved. The same process is repeated for the third and fourth groups.

[0129] The embodiment of the present application starts from the bottleneck of memory usage and proposes a peak suppression tuning method, giving priority to processing the memory that has a direct impact on the usage rate, and clustering and grouping this part of the memory, taking into account both the algorithm complexity and the algorithm effect.

[0130] S1.3. Until the plurality of memory rectangles meet any one of the plurality of tuning targets, determine the first address offset and the last address offset of each tensor according to the position of each memory rectangle in the memory container.

[0131] Specifically, the multiple tuning targets include that the width of the target rectangle does not exceed the theoretical threshold, the same sorting result appears, and the number of adjustments is greater than or equal to the preset number. If the multiple memory rectangles do not meet any of the multiple tuning targets, then continue to adjust the sorting of the multiple memory rectangles and rearrange them in the memory container according to the preset rules. Until the multiple memory rectangles meet any of the multiple tuning targets, the memory allocation device can determine the first address offset of each tensor based on the minimum distance between each memory rectangle and the bottom of the memory container, determine the tail address offset of each tensor based on the maximum distance between each memory rectangle and the bottom of the memory container, and then determine the memory address of each tensor based on the first address of each tensor and the tail address of each tensor.

[0132] In a possible embodiment, after the plurality of tensors meet any one of the plurality of tuning targets, the memory allocation device may further visualize the plurality of memory rectangles and memory containers, so as to facilitate the user to view and check the memory allocation results.

[0133] In a possible embodiment, after determining the memory address of each tensor, the memory allocation device may detect whether there are overlapping memory addresses to ensure that the memory can be used normally.

[0134] Please refer to Figure 4 , is a structural schematic diagram of a memory allocation device provided in an embodiment of the present application, the memory allocation device including: a memory rectangle converter 401, a memory layout tuner 402, and a memory allocation debugger 403. Among them, the memory rectangle converter 401 is used to convert each memory operation into a memory rectangle. The memory layout tuner 402 is used to adjust the position of the memory rectangle in the memory container so that the height of the memory container is as low as possible, and output the memory allocation information after the tuning is completed. The memory allocation debugger 403 is used to check and visualize the memory allocation information. The memory allocation information includes memory name, time information, memory size, first address offset, etc.

[0135] Memory layout tuner 402 includes a memory rectangle sorter 404, a memory rectangle placer 405, a layout space optimizer 406, and a memory allocation information exporter 407. Memory rectangle sorter 404 is used to sort multiple memory rectangles. Memory rectangle placer 405 is used to sequentially arrange multiple memory rectangles in a memory container. Layout space optimizer 406 is used to optimize the spatial layout of multiple memory rectangles. Memory allocation information exporter 407 is used to output memory allocation information after optimization is complete.

[0136] Please refer to Figure 5 , the embodiment of the present application provides a process of a static memory allocation method Figure 2 , combined with Figure 4 and Figure 5 , introduces the static memory allocation method provided in the embodiment of the present application.

[0137] S501: Obtain a memory operation sequence.

[0138] The memory rectangle converter 401 can receive a memory operation sequence provided by the serialization model, wherein the memory operation sequence includes a plurality of memory operations executed in sequence. The meaning of the memory operation is described above and will not be repeated here.

[0139] S502: Convert each memory operation into a memory rectangle.

[0140] After acquiring the memory operation sequence, the memory rectangle converter 401 analyzes the time length and space size of each memory operation according to the memory operation sequence, and constructs a memory rectangle based on the time length and space size to obtain a memory rectangle list including multiple memory rectangles.

[0141] S503: Sort the multiple memory rectangles.

[0142] The memory rectangle sorter 404 receives the memory rectangle list output by the memory rectangle converter 401, sorts the memory rectangles in the memory rectangle list according to a preset index, and outputs the sorted memory rectangles. The meaning of the preset index is described above and will not be repeated here.

[0143] S504: Place the sorted memory rectangles in sequence in the memory container.

[0144] Memory rectangle placer 405 receives the sorted memory rectangle list output by memory rectangle sorter 404 and places the memory rectangles in a memory container according to a preset rule and order. The meaning of the sorting method, preset rules, and memory container is discussed above and will not be repeated here.

[0145] S505: Whether the tuning target is met.

[0146] It is determined whether the placement result of the memory rectangle placer 405 meets the tuning target. If so, S507 is executed. If not, S506 is executed. The meaning of the tuning target can be found in the above discussion and will not be repeated here.

[0147] S506: Optimize the spatial layout of multiple memory rectangles.

[0148] When the tuning target is not achieved, the layout space optimizer 406 receives the sorted memory rectangle list output by the memory rectangle sorter 404, optimizes the spatial layout of the multiple memory rectangles, and adjusts the unreasonable parts of the memory layout so that the tuning result is closer to the tuning target, and then continues to execute S505 until the tuning target is met.

[0149] S507: Check and visualize the memory allocation information.

[0150] When the tuning target is achieved, the memory allocation information output by the memory allocation information exporter 407 is received by the memory allocation debugger 403, which checks whether there is overlap in the memory addresses and visualizes the memory rectangles and memory containers. The method for determining the content of the memory address is discussed above and will not be repeated here.

[0151] Please refer to Figure 6 , which is a schematic diagram of the structure of a layout space optimizer according to an embodiment of the present application. Layout space optimizer 406 includes a memory rectangle classifier 601, a medium- and high-risk memory tuner 602, and a memory rectangle merger 603. The following describes the specific workflow of layout space optimizer 406 in conjunction with each module.

[0152] Memory rectangle classifier 601 receives the sorted memory rectangle list output by memory rectangle sorter 404 and classifies the memory rectangles in the sorted memory rectangle list into four categories: high-risk memory rectangles, medium-risk memory rectangles, low-risk memory rectangles, and no-risk memory rectangles. The meanings of high-risk memory rectangles, medium-risk memory rectangles, low-risk memory rectangles, and no-risk memory rectangles are discussed above and will not be further elaborated here.

[0153] The medium- and high-risk memory tuner 602 receives the high-risk memory rectangles and the medium-risk memory rectangles output by the memory rectangle classifier 601 , and adjusts the sorting of the high-risk memory rectangles and the sorting of the medium-risk memory rectangles respectively.

[0154] The memory rectangle merger 603 receives the adjusted high-risk memory rectangle and the adjusted medium-risk memory rectangle output by the medium- and high-risk memory tuner 602, as well as the low-risk memory rectangle and the risk-free memory rectangle output by the memory rectangle classifier 601, and then merges the adjusted high-risk memory rectangle, the adjusted medium-risk memory rectangle, the low-risk memory rectangle, and the risk-free memory rectangle in sequence according to the risk degradation order to obtain an updated memory rectangle list.

[0155] In a possible embodiment, the medium- and high-risk memory tuner 602 further includes a memory rectangle reorderer for reordering memory rectangles. Figure 7 , the embodiment of the present application provides a workflow diagram of a medium- and high-risk memory tuner, which is combined with Figure 4 and Figure 7 , introduces the specific workflow of the medium and high risk memory tuner 602.

[0156] S701: Divide medium- and high-risk memory rectangles into multiple groups.

[0157] The medium-high risk memory tuner 602 divides the high-risk memory rectangles into a plurality of groups, and divides the medium-risk memory rectangles into a plurality of groups.

[0158] S702: Sorting within the group.

[0159] The medium- and high-risk memory tuner 602 sorts the memory rectangles of each group through the memory rectangle sorter 404 .

[0160] S703: Place the memory rectangles in each group in the memory container in sequence.

[0161] The medium- and high-risk memory tuner 602 places the memory rectangles in each group in the memory container in sequence according to a preset rule through the memory rectangle placer 405 .

[0162] S704: Whether the optimization target is met.

[0163] The medium- and high-risk memory tuner 602 determines whether the placement result of the memory rectangles in each group meets the tuning target. If not, S705 is executed; if so, S706 is executed.

[0164] S705: Rearrange the memory rectangles of each group.

[0165] The medium- and high-risk memory tuner 602 rearranges the memory rectangles of each group through the memory rectangle rearranger to obtain a possibly better sorting result.

[0166] S706: Accumulate each set of tuning results into the memory container.

[0167] The medium- and high-risk memory tuner 602 accumulates each set of tuning results into the memory container in sequence. The meaning of accumulation is described above and will not be repeated here.

[0168] S707: Export a list of medium- and high-risk memory rectangles.

[0169] When all groups are processed, the medium- and high-risk memory tuner 602 exports a medium- and high-risk memory rectangle list from the memory container.

[0170] In summary, this application converts the static memory allocation problem into a two-dimensional rectangular box packing problem and solves the static memory allocation problem through an approximate mathematical problem. Each memory to be allocated is equivalent to a rectangle, and the memory pool is equivalent to a box container. The rectangles are placed in the box container. By adjusting the sorting and grouping strategies, setting tuning targets, etc., the maximum height of the box container is made as low as possible, thereby finding a more optimal memory allocation result and improving the utilization rate of memory resources.

[0171] Based on the same inventive concept, the present application also provides a static memory allocation device, which can be specifically set in the memory allocation device discussed above, please refer to Figure 8 , the device comprises:

[0172] An acquisition module 801 is used to acquire multiple tensors of a target model and determine the time information and space size of each tensor occupied by the memory;

[0173] An allocation module 802 is configured to sort the multiple tensors according to a preset index and assign a first address offset to the multiple tensors according to a preset rule; wherein the first address offset represents the offset between the first address of each tensor and the first address of the memory pool. The preset rule includes that the memory addresses of tensors with overlapping time information do not overlap, and the time information of tensors with overlapping memory addresses does not overlap. The memory address of each tensor is determined based on the first address offset and the spatial size of each tensor;

[0174] The allocation module 802 is further configured to adjust the order of the multiple tensors and reassign the first address offsets to the multiple tensors according to a preset rule if the tail address offset of any tensor is greater than a theoretical threshold. The tail address offset of each tensor is the sum of the first address offset and the space size of each tensor, and the theoretical threshold is the maximum value of the sum of the space sizes occupied by the tensors at each moment.

[0175] The determination module 803 is configured to determine the memory address of each tensor according to the first address offset and the last address offset of each tensor until the multiple tensors meet any one of the multiple tuning targets.

[0176] In a possible embodiment, the allocation module 802 is specifically configured to:

[0177] Generate multiple memory rectangles corresponding to multiple memory operations; where multiple memory operations represent operations in which multiple tensors occupy memory, the length of each memory rectangle represents the duration of each memory operation, and the width of each memory rectangle represents the space size of each memory operation;

[0178] Sort multiple memory rectangles according to preset indices and arrange them in a memory container corresponding to the memory pool according to preset rules; wherein the first address of the memory pool corresponds to the bottom of the memory container, and the minimum distance between each memory rectangle and the bottom is the first address offset of each tensor.

[0179] In a possible embodiment, the allocation module 802 is specifically configured to:

[0180] If the width of the target rectangle exceeds the theoretical threshold, the order of the multiple memory rectangles is adjusted and rearranged in the memory container according to preset rules; wherein the target rectangle is the smallest rectangle containing the multiple memory rectangles after arrangement, and the width of the target rectangle represents the maximum value of the tail address offsets of the multiple tensors.

[0181] In a possible embodiment, the determining module 803 is further configured to:

[0182] After the plurality of tensors meet any one of the plurality of tuning targets, a first address offset and a tail address offset of each tensor are determined according to a position of each memory rectangle in the memory container.

[0183] In a possible embodiment, the allocation module 802 is specifically configured to:

[0184] Classify multiple memory rectangles to obtain high-risk memory rectangles, medium-risk memory rectangles, low-risk memory rectangles, and risk-free memory rectangles; wherein the high-risk memory rectangle includes a first memory rectangle corresponding to a first tensor having a tail address offset greater than a theoretical threshold, and a second memory rectangle that overlaps with the first memory rectangle in time; the medium-risk memory rectangle includes a third memory rectangle corresponding to the second tensor having a tail address offset equal to the theoretical threshold, and a fourth memory rectangle that overlaps with the third memory rectangle in time; the low-risk memory rectangle includes a memory rectangle that overlaps with the high-risk memory rectangle and the medium-risk memory rectangle in time; and the risk-free memory rectangle includes memory rectangles other than the high-risk memory rectangle, the medium-risk memory rectangle, and the low-risk memory rectangle among the multiple memory rectangles;

[0185] Adjust the sorting of high-risk memory rectangles and medium-risk memory rectangles separately;

[0186] The adjusted high-risk memory rectangle, the adjusted medium-risk memory rectangle, the low-risk memory rectangle, and the no-risk memory rectangle are merged in sequence.

[0187] In a possible embodiment, the allocation module 802 is specifically configured to:

[0188] dividing the high-risk memory rectangles into multiple groups and the low-risk memory rectangles into multiple groups based on the length of temporal overlap;

[0189] Sort the memory rectangles of each group according to the preset index and arrange them in the memory container according to the preset rules;

[0190] If the width of the target rectangle exceeds the theoretical threshold, the order of the memory rectangles of each group is adjusted and rearranged in the memory container according to the preset rules;

[0191] Until the multiple memory rectangles meet any one of the multiple tuning targets, the memory rectangles of each group after adjustment and sorting are derived in sequence to obtain adjusted high-risk memory rectangles and adjusted medium-risk memory rectangles.

[0192] In a possible embodiment, the preset indicator includes at least one indicator of the length of the memory rectangle, the width of the memory rectangle, and the time overlap length between the memory rectangles.

[0193] In a possible embodiment, the multiple tuning targets include: the width of the target rectangle does not exceed a theoretical threshold; the same sorting result occurs; and the number of adjustments is greater than or equal to a preset number.

[0194] It should be noted that although several modules or submodules of the device are mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of the present invention, the features and functions of two or more modules described above can be embodied in a single unit. Conversely, the features and functions of a module described above can be further divided and embodied by multiple modules.

[0195] It should be noted that Figure 8 The device can also be used to implement any of the static memory allocation methods discussed above, which will not be described here.

[0196] Based on the same inventive concept, an electronic device is also provided in the embodiment of the present application. The device is equivalent to the memory allocation device discussed above. Please refer to Figure 9 , the device includes:

[0197] Memory 902, used to store program instructions;

[0198] Processor 901 is used to call the program instructions stored in the memory 902 and execute the program instructions obtained. Figure 2 and Figure 5 Any of the static memory allocation methods. The processor 901 can also implement Figure 8 The functions of each module in the device shown.

[0199] The specific connection medium between the processor 901 and the memory 902 is not limited in the embodiment of the present application. Figure 9 In the example, the processor 901 and the memory 902 are connected via a bus 900. Figure 9 The bus 900 can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, Figure 9 The diagram is represented by only one thick line, but this does not mean that there is only one bus or one type of bus. Alternatively, the processor 901 may also be referred to as a controller, without limitation to the name.

[0200] Among them, the processor 901 is the control center of the device, which can use various interfaces and lines to connect the various parts of the entire control device, and monitor the device as a whole by running or executing instructions stored in the memory 902 and calling data stored in the memory 902, the various functions of the device and processing data.

[0201] In one possible design, processor 901 may include one or more processing units. Processor 901 may integrate an application processor and a modem processor. The application processor primarily processes the operating system, user interface, and application programs, while the modem processor primarily handles wireless communications. It is understood that the modem processor may not be integrated into processor 901. In some embodiments, processor 901 and memory 902 may be implemented on the same chip. In some embodiments, they may also be implemented on separate chips.

[0202] The processor 901 can be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, and can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of the target monitoring method disclosed in the embodiments of the present application can be directly embodied as a hardware processor for execution, or can be executed by a combination of hardware and software modules in the processor.

[0203] The memory 902 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The memory 902 may include at least one type of storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory, a random access memory (Random Access Memory, RAM), a static random access memory (Static Random Access Memory, SRAM), a programmable read-only memory (Programmable Read Only Memory, PROM), a read-only memory (Read Only Memory, ROM), an electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), a magnetic memory, a disk, an optical disk, etc. The memory 902 is any other medium that can be used to carry or store a desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory 902 in the embodiment of the present application can also be a circuit or any other device that can realize a storage function, for storing program instructions and / or data.

[0204] By designing and programming the processor 901, the code corresponding to the target monitoring method described in the above embodiment can be fixed into the chip, so that the chip can execute the target monitoring method when it is running. Figure 2 and Figure 5 The steps of the static memory allocation method are as follows: How to design and program the processor 901 is a technique well known to those skilled in the art and will not be described in detail here.

[0205] Based on the same inventive concept, an embodiment of the present application provides a computer-readable storage medium storing a computer program, wherein the computer program includes program instructions. When executed by a computer, the program instructions cause the computer to execute any of the static memory allocation methods discussed above. Because the principles for solving the problem solved by the computer-readable storage medium are similar to those of the static memory allocation method, the implementation of the computer-readable storage medium can be referred to as the implementation of the method, and any repetitions will not be repeated.

[0206] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0207] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0208] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0209] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0210] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A static memory allocation method, characterized in that: include: Get multiple tensors of the target model and determine the time information and space size of each tensor in memory; Sorting the multiple tensors according to a preset index, and assigning a first address offset to the multiple tensors according to a preset rule; wherein the first address offset represents the offset between the first address of each tensor and the first address of the memory pool, the preset rule includes that the memory addresses of tensors with overlapping time information do not overlap, and the time information of tensors with overlapping memory addresses does not overlap, and the memory address of each tensor is determined according to the first address offset and the space size of each tensor; If the tail address offset of any tensor is greater than the theoretical threshold, the order of the multiple tensors is adjusted, and the first address offsets are reassigned to the multiple tensors according to the preset rule; wherein the tail address offset of each tensor is the sum of the first address offset and the space size of each tensor, and the theoretical threshold is the maximum value of the sum of the space sizes occupied by the tensors at each moment; Until the plurality of tensors meet any one of a plurality of tuning targets, determining a memory address of each tensor according to a first address offset and a tail address offset of each tensor; The step of sorting the plurality of tensors according to a preset index and allocating a first address offset to the plurality of tensors according to a preset rule includes: Generate multiple memory rectangles corresponding to multiple memory operations; wherein the multiple memory operations represent operations of the multiple tensors occupying memory, the length of each memory rectangle represents the time length of each memory operation, and the width of each memory rectangle represents the space size of each memory operation; The multiple memory rectangles are sorted according to preset indicators and arranged in a memory container corresponding to the memory pool according to preset rules; wherein the first address of the memory pool corresponds to the bottom of the memory container, and the minimum distance between each memory rectangle and the bottom is the first address offset of each tensor.

2. The method according to claim 1, wherein If the tail address offset of any tensor is greater than the theoretical threshold, the order of the multiple tensors is adjusted, and the first address offsets of the multiple tensors are reassigned according to the preset rule, including: If the width of the target rectangle exceeds a theoretical threshold, the order of the multiple memory rectangles is adjusted and rearranged in the memory container according to the preset rule; wherein the target rectangle is the smallest rectangle containing the multiple arranged memory rectangles, and the width of the target rectangle represents the maximum value among the tail address offsets of the multiple tensors.

3. The method according to claim 2, wherein After the plurality of tensors satisfy any one of a plurality of tuning objectives, the method further includes: According to the position of each memory rectangle in the memory container, the first address offset and the last address offset of each tensor are determined.

4. The method according to claim 2, wherein Adjusting the order of the plurality of memory rectangles includes: Classify the multiple memory rectangles to obtain high-risk memory rectangles, medium-risk memory rectangles, low-risk memory rectangles, and risk-free memory rectangles; wherein, the high-risk memory rectangle includes a first memory rectangle corresponding to a first tensor whose tail address offset is greater than the theoretical threshold, and a second memory rectangle that temporally overlaps with the first memory rectangle; the medium-risk memory rectangle includes a third memory rectangle corresponding to a second tensor whose tail address offset is equal to the theoretical threshold, and a fourth memory rectangle that temporally overlaps with the third memory rectangle; the low-risk memory rectangle includes a memory rectangle that temporally overlaps with the high-risk memory rectangle and the medium-risk memory rectangle; and the risk-free memory rectangle includes memory rectangles among the multiple memory rectangles excluding the high-risk memory rectangle, the medium-risk memory rectangle, and the low-risk memory rectangle; Adjusting the sorting of the high-risk memory rectangles and the sorting of the medium-risk memory rectangles respectively; The adjusted high-risk memory rectangle, the adjusted medium-risk memory rectangle, the low-risk memory rectangle, and the risk-free memory rectangle are merged in sequence.

5. The method according to claim 4, wherein Adjusting the sorting of the high-risk memory rectangles and the medium-risk memory rectangles respectively includes: dividing the high-risk memory rectangles into a plurality of groups, and dividing the low-risk memory rectangles into a plurality of groups according to a time overlap length; Sort the memory rectangles of each group according to the preset index and arrange them in the memory container according to the preset rule; If the width of the target rectangle exceeds the theoretical threshold, adjusting the order of the memory rectangles of each group and rearranging them in the memory container according to the preset rule; Until the memory rectangle of each group meets any one of the multiple tuning targets, the memory rectangles of each group after adjustment and sorting are derived in sequence to obtain the adjusted high-risk memory rectangle and the adjusted medium-risk memory rectangle.

6. The method according to any one of claims 1 to 5, wherein: The preset indicator includes at least one indicator of the length of the memory rectangle, the width of the memory rectangle, and the time overlap length between the memory rectangles.

7. The method according to any one of claims 1 to 5, characterized in that The multiple tuning targets include: no tail address offset of any tensor is greater than the theoretical threshold, the same sorting result occurs, and the number of adjustments is greater than or equal to a preset number.

8. A static memory allocation device, characterized in that: include: The acquisition module is used to obtain multiple tensors of the target model and determine the time information and space size of each tensor in memory; an allocation module, configured to sort the multiple tensors according to a preset index and allocate a first address offset to the multiple tensors according to a preset rule; wherein the first address offset represents the offset between the first address of each tensor and the first address of the memory pool, the preset rule includes that the memory addresses of tensors with overlapping time information do not overlap, and the time information of tensors with overlapping memory addresses does not overlap, and the memory address of each tensor is determined based on the first address offset and the spatial size of each tensor; The allocation module is further configured to adjust the order of the multiple tensors and reassign the first address offsets to the multiple tensors according to the preset rule if the tail address offset of any tensor is greater than a theoretical threshold; wherein the tail address offset of each tensor is the sum of the first address offset and the space size of each tensor, and the theoretical threshold is the maximum value of the sum of the space sizes occupied by the tensors at each moment; a determination module, configured to determine a memory address of each tensor according to a first address offset and a last address offset of each tensor until the plurality of tensors meet any one of a plurality of tuning targets; The allocation module is specifically used to: generate multiple memory rectangles corresponding to multiple memory operations; wherein the multiple memory operations represent the operations of the multiple tensors occupying memory, the length of each memory rectangle represents the time length of each memory operation, and the width of each memory rectangle represents the space size of each memory operation; sort the multiple memory rectangles according to preset indicators, and arrange them in the memory container corresponding to the memory pool according to preset rules; wherein the first address of the memory pool corresponds to the bottom of the memory container, and the minimum distance between each memory rectangle and the bottom is the first address offset of each tensor.

9. An electronic device, characterized in that: include: a memory for storing program instructions; A processor is configured to call program instructions stored in the memory and execute the method according to any one of claims 1 to 7 according to the obtained program instructions.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions. When the program instructions are executed by a computer, the computer is caused to perform the method according to any one of claims 1 to 7.

Images